Test AI behavior on real examplesYou record realistic inputs, criteria, expected or forbidden actions, predictions, and traces. Use deterministic metrics when possible and judges when quality needs holistic review.cppacademyacademy/topics/evals-metrics-judgeswebsite/content-src/academy/course.mjsacademyTest AI behavior on real examples
You record realistic inputs, criteria, expected or forbidden actions, predictions, and traces. Use deterministic metrics when possible and judges when quality needs holistic review.
9 focused minutesNot started
Unit example (nearest native match)
See the idea in context
auto engine = axllm::AxGEPA(reflectionClient, axllm::object({}));
auto result = engine.optimize(request, evaluator);
Use a realistic input
The refund request represents the kind of task the agent will face.
Write the success rule
criteria explains that eligibility must be verified before action.
Record observable behavior
expectedActions lets the evaluation check tool selection, not only final prose.
Run itIn your own project
cmake_minimum_required(VERSION 3.20)
project(ax_quick_start LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
include(FetchContent)
FetchContent_Declare(axllm GIT_REPOSITORY https://github.com/ax-llm/ax GIT_TAG main SOURCE_SUBDIR packages/cpp)
FetchContent_MakeAvailable(axllm)
add_executable(quick_start quick_start.cpp)
target_link_libraries(quick_start PRIVATE axllm::axllm)
#include <axllm/axllm.hpp>
#include <cstdlib>
#include <iostream>
int main() {
auto llm = axllm::ai("openai", axllm::object({{"apiKey", std::getenv("OPENAI_API_KEY")}}));
auto classify = axllm::ax("review:string -> sentiment:class \"positive, negative, neutral\"");
auto result = classify.forward(*llm, axllm::object({
{"review", "Useful and boring in the best way."}
}));
auto sentiment = axllm::Core::get(result, "sentiment");
std::cout << "sentiment: " << std::get<std::string>(sentiment.data) << "\n";
}
Set OPENAI_APIKEY in your environment before running provider-backed code.
In the ax repo
From a clone of the ax repo:
npm run example -- cpp src/examples/cpp/optimization/axgen_optimization.cpp