Class DemoBenchmark

java.lang.Object

ai.redouble.nucleo.harness.AbstractJob<DemoAnswer>

ai.redouble.nucleo.tools.AbstractTool<DemoQuery, DemoAnswer>

ai.redouble.nucleo.tools.AbstractOrchestrator<DemoQuery, DemoAnswer>

ai.redouble.nucleo.tools.AbstractDoer<DemoQuery, DemoAnswer>

ai.redouble.nucleo.tools.benchmark.Benchmark<DemoQuery, DemoAnswer>

ai.redouble.demo.DemoBenchmark

All Implemented Interfaces:

GuardedExecution, ScopeAuthority, Identifiable, Job<DemoAnswer>, Doer<DemoQuery, DemoAnswer>, Orchestrator<DemoQuery, DemoAnswer>, Tool<DemoQuery, DemoAnswer>

public final class DemoBenchmark
extends Benchmark<DemoQuery, DemoAnswer>

The demo's benchmark of DemoAgent: the reference answers on the strongest model the deployment serves - resolved through Grade.CEILING, the same way the judge is - while every raced copy is re-pinned to its candidate, and the judges wait behind the same person the raced agents do. A named class, so its job ids and its log lines (ai.redouble.demo.DemoBenchmark.progress for the runs-finished count) name it.

Since:

0.1 (2026-09-22)

Author:

Andrey Santrosyan

Field Summary

Fields inherited from class Benchmark

CANDIDATE, JUDGE, REFERENCE, REPORT, RUN_MODEL, RUN_NUMBER, RUN_ROLE

Fields inherited from class AbstractOrchestrator

META_ITERATION

Fields inherited from class AbstractTool

input

Fields inherited from class AbstractJob

timeout, upstreamRetries

Fields inherited from interface Job

DEFAULT_UPSTREAM_RETRIES

Fields inherited from interface Tool

OBS_INPUT, OBS_OUTPUT

Constructor Summary

Constructors

Constructor	Description
DemoBenchmark(Identifiable workflow, int runs)

Method Summary

Modifier and Type	Method	Description
protected JudgeTool	newJudge()	The judge for one of this benchmark's answers.

Methods inherited from class Benchmark

candidates, execute, report, setCandidates, setRubric, setScorer

Methods inherited from class AbstractDoer

nextStep, submitInCurrentStep, submitInCurrentStep, submitInCurrentStep, submitInStep

Methods inherited from class AbstractOrchestrator

execute, getJobType, getRequirements, getScopeGuard, getTimeout, sealScopeGuard, setScopeGuard, setTimeout, submitInIteration, submitInIteration, submitInIteration

Methods inherited from class AbstractTool

getInput, postExecute, preExecute, setInput

Methods inherited from class AbstractJob

getId, getName, getParentJobId, getPriority, getUpstreamRetries, getUserId, getWorkflowId, randomIdSuffix, setUpstreamRetries, toString

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface GuardedExecution

declareAdmissionGuardrails, declareAuthGuardrails, declareContentGuardrails

Methods inherited from interface Identifiable

getParentJobId, getUserId, getWorkflowId

Methods inherited from interface Job

getDisplayName, getId, getJobId, getJobType, getName, getPriority, getRequirements, getTimeout, getUpstreamRetries, postExecute, preExecute

Methods inherited from interface ScopeAuthority

getScopeGuard, sealScopeGuard, setScopeGuard

Methods inherited from interface Tool

execute, getInput, guardedInputTarget, setInput

Constructor Details

DemoBenchmark

public DemoBenchmark(Identifiable workflow,
 int runs)

Parameters:

workflow - the workflow the benchmark and every agent it builds belong to

runs - how many times each candidate answers

Method Details

newJudge

protected JudgeTool newJudge()

Description copied from class: Benchmark

The judge for one of this benchmark's answers. A test overrides it to judge without a transport.

Overrides:

newJudge in class Benchmark<DemoQuery, DemoAnswer>