Class BenchmarkRace

java.lang.Object
ai.redouble.demo.BenchmarkRace
All Implemented Interfaces:
JobObserver<JobEvent>
public final class BenchmarkRace extends Object implements JobObserver<JobEvent>
One benchmark's race, streamed to the browser as it runs: an observer on the runtime's message bus, scoped to the race's workflow, that turns every event of the raced runs and their judges - and of the calls those runs make, attributed by walking each job's parent chain up to the run the Benchmark named - into one line of newline-delimited JSON on the response, the way the chat endpoints stream a conversation's progress over a socket. The lines borrow the chat wire's frame types: a status line carrying the plan (one row per run, in race order), a progress line per event - a terminal one carrying the row's spend so far, summed from the responses it carries and priced from the catalog; a judge's completion carrying its score and reason - and workflow_complete with the answer and the report, or workflow_failed with the failure. A browser that leaves stops the lines and nothing else: the race runs to its end on the runtime.

The logic is the demo's, shared by every host: a host supplies how a line reaches the client (sink), how the stream is closed (onEnd), and its own JSON serializer (toJson), so the wire is the framework's while the trace is one.

Since:
0.1 (2026-09-25)
Author:
Andrey Santrosyan

  • Constructor Details

    • BenchmarkRace

      public BenchmarkRace(Function<Object,String> toJson, String workflowId, String benchmarkJobId, Consumer<String> sink, Runnable onEnd)
      Parameters:
      toJson - the host's JSON serializer
      sink - where a rendered line goes, one call per line
      onEnd - closes the stream, called once when the last line is written

  • Method Details

    • plan

      public void plan(List<ModelSpec> raced, int runs)
      The first line: every row the race will have, in race order, before any of them has an event.

    • getPredicate

      public Predicate<JobEvent> getPredicate()
      Description copied from interface: JobObserver
      Returns a predicate for filtering events at the MessageBus level. Called once during subscription.
      Specified by:
      getPredicate in interface JobObserver<JobEvent>
      Returns:
      predicate to filter events, or null to accept all events of type T

    • observe

      public void observe(JobEvent event)
      Description copied from interface: JobObserver
      Process a job event. Runs in a separate virtual thread. Must handle its own resources and exceptions.
      Specified by:
      observe in interface JobObserver<JobEvent>
      Parameters:
      event - the job event to process

    • complete

      public void complete(BenchmarkOutcome outcome)
      The last line: the answer and the report, then the stream closes.

    • fail

      public void fail(Throwable failure)
      The last line when the benchmark itself failed, then the stream closes.