AxJudge

Defined in: https://github.com/ax-llm/ax/blob/a8847bd2906efff202fde10d776fddd20fd2ff57/src/ax/dsp/judge.ts#L148

AxJudge - Polymorphic evaluation engine that automatically selects the best strategy.

Example

const judge = new AxJudge(signature, { ai: teacherAI });

// Absolute mode (with ground truth)
const result1 = await judge.evaluate(
  { question: 'Capital of France?' },
  { answer: 'Paris' },
  { answer: 'Paris' }  // expected
);

// Relativistic mode (compare student vs teacher)
const result2 = await judge.evaluate(
  { question: 'Explain quantum computing' },
  studentResponse,
  teacherResponse
);

// Reference-free mode (no comparison data)
const result3 = await judge.evaluate(
  { question: 'Write a poem' },
  { poem: 'Roses are red...' }
);

Type Parameters

Constructors

Constructor

new AxJudge<IN, OUT>(signature: AxSignature<IN, OUT>, options: AxJudgeOptions): AxJudge<IN, OUT>;

Defined in: https://github.com/ax-llm/ax/blob/a8847bd2906efff202fde10d776fddd20fd2ff57/src/ax/dsp/judge.ts#L153

Parameters

Returns

AxJudge<IN, OUT>

Methods

evaluate()

evaluate(
   input: IN, 
   studentOutput: OUT, 
referenceOutput?: OUT): Promise<AxJudgeResult>;

Defined in: https://github.com/ax-llm/ax/blob/a8847bd2906efff202fde10d776fddd20fd2ff57/src/ax/dsp/judge.ts#L166

The main entry point. Automatically routes to the best strategy.

Parameters

Returns

Promise<AxJudgeResult>

getSignature()

getSignature(): AxSignature<IN, OUT>;

Defined in: https://github.com/ax-llm/ax/blob/a8847bd2906efff202fde10d776fddd20fd2ff57/src/ax/dsp/judge.ts#L451

Get the signature being evaluated.

Returns

AxSignature<IN, OUT>

toMetricFn()

toMetricFn(): AxMetricFn;

Defined in: https://github.com/ax-llm/ax/blob/a8847bd2906efff202fde10d776fddd20fd2ff57/src/ax/dsp/judge.ts#L413

Convert this judge to a metric function for use with optimizers. Uses relativistic mode when teacher output is available as expected.

Returns

AxMetricFn