Unit 8 · Measure and improve AI quality

Turn verified runs into reusable guidance

You accumulate situational guidance from live feedback or a verified task set. Evolution keeps grounded advice that improves performance without unacceptable held-out regression.

playbook()9 focused minutesNot started

Worked example

See the idea in context

auto pb = axllm::playbook(program, student_ai);
pb.evolve(examples, metric);   // grow a playbook offline from examples
auto rendered = pb.render();   // the playbook is now applied to the program
auto state = pb.to_json();     // persist; restore later with playbook(...).load(state)

Run itIn your own project

include(FetchContent)
FetchContent_Declare(axllm GIT_REPOSITORY https://github.com/ax-llm/ax GIT_TAG main SOURCE_SUBDIR packages/cpp)
FetchContent_MakeAvailable(axllm)
target_link_libraries(your_app PRIVATE axllm::axllm)

#include <axllm/axllm.hpp>

auto llm = axllm::ai("openai", axllm::object({{"apiKey", std::getenv("OPENAI_API_KEY")}}));
auto classify = axllm::ax("review:string -> sentiment:class \"positive, negative, neutral\"");

Set OPENAI_APIKEY in your environment before running provider-backed code.

In the ax repo

From a clone of the ax repo:

npm run example -- cpp src/examples/cpp/optimization/axgen_optimization.cpp

Active practice

Show that you can use it

Answer 2 in a row to learn this · attempt 1

Keep exploring

See the idea in context

Show that you can use it

Source-backed follow-up