AxOptimizer

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L984

Interface for optimizing AI programs through automated prompt tuning.

Optimizers improve program performance by finding optimal demonstrations (few-shot examples) that guide the AI toward better outputs. The optimization process:

1. Runs the program on training examples
2. Evaluates outputs using the metric function
3. Selects high-scoring input/output pairs as demonstrations
4. Iterates to find the best demonstration set

Example

const optimizer = new AxBootstrapFewShot({ maxDemos: 4 });

const result = await optimizer.compile(
  program,
  trainingExamples,
  ({ prediction, example }) => prediction.answer === example.expectedAnswer ? 1 : 0
);

// Apply optimized demos to program
program.setDemos(result.demos);

Methods

cancel()?

optional cancel(): Promise<void>;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1105

Cancel ongoing optimization gracefully

Returns

Promise<void>

Promise that resolves when cancellation is complete

compile()

compile<IN, OUT>(
   program: Readonly<AxGen<IN, OUT>>, 
   examples: readonly AxTypedExample<IN>[], 
   metricFn: AxMetricFn, 
options?: AxCompileOptions): Promise<AxOptimizerResult<OUT>>;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1058

Optimize a program using the provided training examples and metric function.

The optimizer runs the program on examples, scores the outputs, and builds a set of demonstrations that improve program performance. The process is automatic - you provide the data and metric, the optimizer does the rest.

Metric Function: The metric function evaluates how well the program’s output matches expectations. It receives the prediction and original example, and should return a score between 0 (worst) and 1 (best).

Type Parameters

Parameters

Returns

Promise<AxOptimizerResult<OUT>>

Promise resolving to optimization results including:

• demos: Optimized demonstrations to use with the program
• stats: Training/validation scores and iteration counts

Examples

const result = await optimizer.compile(
  qa,
  examples,
  ({ prediction, example }) => {
    return prediction.answer.toLowerCase() === example.expectedAnswer.toLowerCase() ? 1 : 0;
  }
);

const result = await optimizer.compile(
  summarizer,
  examples,
  async ({ prediction, example }) => {
    const similarity = await computeCosineSimilarity(
      await embed(prediction.summary),
      await embed(example.referenceSummary)
    );
    return similarity; // 0-1 based on embedding similarity
  }
);

const result = await optimizer.compile(
  extractor,
  examples,
  ({ prediction, example }) => {
    const expectedKeywords = example.keywords;
    const foundKeywords = prediction.keywords;
    const matches = expectedKeywords.filter(k => foundKeywords.includes(k));
    return matches.length / expectedKeywords.length; // Fraction found
  }
);

compilePareto()?

optional compilePareto<IN, OUT>(
   program: Readonly<AxGen<IN, OUT>>, 
   examples: readonly AxTypedExample<IN>[], 
   metricFn: AxMultiMetricFn, 
options?: AxCompileOptions): Promise<AxParetoResult<OUT>>;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1088

Multi-objective optimization using Pareto frontier

Type Parameters

Parameters

Returns

Promise<AxParetoResult<OUT>>

Pareto optimization result

compileStream()?

optional compileStream<IN, OUT>(
   program: Readonly<AxGen<IN, OUT>>, 
   examples: readonly AxTypedExample<IN>[], 
   metricFn: AxMetricFn, 
options?: AxCompileOptions): AsyncIterableIterator<AxOptimizationProgress>;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1073

Optimize a program with real-time streaming updates

Type Parameters

Parameters

Returns

AsyncIterableIterator<AxOptimizationProgress>

Async iterator yielding optimization progress

getConfiguration()?

optional getConfiguration(): Record<string, unknown>;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1116

Get optimizer-specific configuration

Returns

Record<string, unknown>

Current optimizer configuration

getStats()

getStats(): AxOptimizationStats;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1099

Get current optimization statistics

Returns

AxOptimizationStats

Current optimization statistics

reset()?

optional reset(): void;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1110

Reset optimizer state for reuse with different programs

Returns

void

updateConfiguration()?

optional updateConfiguration(config: Readonly<Record<string, unknown>>): void;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1122

Update optimizer configuration

Parameters

Returns

void

validateProgram()?

optional validateProgram<IN, OUT>(program: Readonly<AxGen<IN, OUT>>): object;

Defined in: https://github.com/ax-llm/ax/blob/5b28f9093bb70863b59459bb6df5062d005bce41/src/ax/dsp/optimizer.ts#L1129

Validate that the optimizer can handle the given program

Type Parameters

Parameters

Returns

object

Validation result with any issues found