Unit 8 · Measure and improve AI quality

Turn verified runs into reusable guidance

You accumulate situational guidance from live feedback or a verified task set. Evolution keeps grounded advice that improves performance without unacceptable held-out regression.

playbook()9 focused minutesNot started

Worked example

See the idea in context

AxPlaybook pb = Ax.playbook(program, Map.of("studentAI", studentAI));
pb.evolve(examples, metric, Map.of());   // grow a playbook offline from examples
pb.update(Map.of("example", ex, "prediction", pred, "feedback", "..."));
pb.applyTo(program);

Run itIn your own project

// Gradle (build.gradle):
implementation 'dev.axllm:ax:22.0.4'
// Maven (pom.xml):
<dependency>
  <groupId>dev.axllm</groupId>
  <artifactId>ax</artifactId>
  <version>22.0.4</version>
</dependency>

import dev.axllm.ax.Ax;

var llm = Ax.ai("openai", Map.of("apiKey", System.getenv("OPENAI_API_KEY")));
var classify = Ax.ax("review:string -> sentiment:class \"positive, negative, neutral\"");

Set OPENAI_APIKEY in your environment before running provider-backed code.

In the ax repo

From a clone of the ax repo:

npm run example -- java src/examples/java/optimization/AxgenOptimizationExample.java

Active practice

Show that you can use it

Answer 2 in a row to learn this · attempt 1

Keep exploring

See the idea in context

Show that you can use it

Source-backed follow-up