Class RaceTheGrade

java.lang.Object

ai.redouble.examples.benchmark.RaceTheGrade

public class RaceTheGrade
extends Object

Is the grade right for this job, and which model at that grade does it best, for what cost and how fast? The benchmark answers with evidence: it runs the job once the way the deployment would (the reference), then the same job with the same input on every model of the grade the deployment can call, and a judge - the strongest model the deployment serves - compares each run with the reference in full, not told which model ran either side. A candidate that skipped a lookup loses points even when its reply reads well.

Benchmark is a doer over a factory, because every run needs a fresh agent, and any job whose model is chosen by grade can be raced this way, agents and one-call tools alike. It returns what the reference answered, so a workflow that wraps a step in a benchmark receives exactly what the step would have returned; the measurements come beside it, one row per run with cost, speed and the judge's verdict, from report().

Since:

0.1 (2026-09-27)

Author:

Andrey Santrosyan

Constructor Summary

Constructors

Constructor	Description
RaceTheGrade()

Method Summary

Modifier and Type	Method	Description
static void	main(String[] args)

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RaceTheGrade

public RaceTheGrade()

Method Details

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception