Unit 8 · Measure and improve AI quality

Turn verified runs into reusable guidance

You accumulate situational guidance from live feedback or a verified task set. Evolution keeps grounded advice that improves performance without unacceptable held-out regression.

playbook()9 focused minutesNot started

Worked example

See the idea in context

pb := axllm.Playbook(program, map[string]axllm.Value{"studentAI": studentAI})
pb.Evolve(ctx, examples, metric, nil)   // grow a playbook offline from examples
pb.Update(ctx, map[string]axllm.Value{"example": ex, "prediction": pred, "feedback": "..."})
pb.ApplyTo(program)

Run itIn your own project

go get github.com/ax-llm/ax/packages/go

package main

import (
    "context"
    "fmt"
    "os"

    axllm "github.com/ax-llm/ax/packages/go"
)

func main() {
    client := axllm.NewAI("openai", map[string]axllm.Value{"apiKey": os.Getenv("OPENAI_API_KEY")})
    classify := axllm.NewAx("review:string -> sentiment:class \"positive, negative, neutral\"", nil)
    result, err := classify.Forward(context.Background(), client, map[string]axllm.Value{
        "review": "Useful and boring in the best way."
    }, nil)
    if err != nil {
        panic(err)
    }

    fmt.Println("sentiment:", result.(map[string]axllm.Value)["sentiment"])
}

Set OPENAI_APIKEY in your environment before running provider-backed code.

In the ax repo

From a clone of the ax repo:

npm run example -- go src/examples/go/optimization/axgen_optimization.go

Active practice

Show that you can use it

Answer 2 in a row to learn this · attempt 1

Keep exploring

See the idea in context

Show that you can use it

Source-backed follow-up