Unit 8 · Measure and improve AI quality

Turn verified runs into reusable guidance

You accumulate situational guidance from live feedback or a verified task set. Evolution keeps grounded advice that improves performance without unacceptable held-out regression.

playbook()9 focused minutesNot started

Worked example

See the idea in context

let mut pb = axllm::playbook(program, student, None, json!({}));
pb.evolve(&examples, &metric, json!({}));  // grow a playbook offline from examples
let rendered = pb.render();                // the playbook is now applied to the program
let state = pb.to_json();                  // persist; restore later with playbook(...).load(state)

Run itIn your own project

cargo add axllm

use axllm::{ai, ax};
use serde_json::json;

let llm = ai("openai", json!({"apiKey": std::env::var("OPENAI_API_KEY")?}))?;
let classify = ax("review:string -> sentiment:class \"positive, negative, neutral\"")?;

Set OPENAI_APIKEY in your environment before running provider-backed code.

In the ax repo

From a clone of the ax repo:

npm run example -- rust src/examples/rust/optimization/axgen_optimization.rs

Active practice

Show that you can use it

Answer 2 in a row to learn this · attempt 1

Keep exploring

See the idea in context

Show that you can use it

Source-backed follow-up