See failures, cost, and latency in production
You add traces, usage and cost accounting, cache policy, cancellation, bounded retries, and safe logs. Debug output becomes evidence for tests and operations.
Unit example (nearest native match)
See the idea in context
axllm::set_rate_limiter(global_limiter);
axllm::set_tracer(tracer);
axllm::set_meter(meter);
workflow.forward(client, input, axllm::Value::object(), axllm::AxRuntimeHooks{call_limiter, tracer, meter});
axllm::set_rate_limiter({}); axllm::set_tracer({}); axllm::set_meter({});- Trace the run
tracer connects model and tool activity to the surrounding request.
- Make cancellation possible
abortSignal lets callers stop work that is no longer useful.
- Control repeated work
contextCache makes reuse an explicit operational policy.
Run itIn your own project
cmake_minimum_required(VERSION 3.20)
project(ax_quick_start LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 17)
include(FetchContent)
FetchContent_Declare(axllm GIT_REPOSITORY https://github.com/ax-llm/ax GIT_TAG main SOURCE_SUBDIR packages/cpp)
FetchContent_MakeAvailable(axllm)
add_executable(quick_start quick_start.cpp)
target_link_libraries(quick_start PRIVATE axllm::axllm)
#include <axllm/axllm.hpp>
#include <cstdlib>
#include <iostream>
int main() {
auto llm = axllm::ai("openai", axllm::object({{"apiKey", std::getenv("OPENAI_API_KEY")}}));
auto classify = axllm::ax("review:string -> sentiment:class \"positive, negative, neutral\"");
auto result = classify.forward(*llm, axllm::object({
{"review", "Useful and boring in the best way."}
}));
auto sentiment = axllm::Core::get(result, "sentiment");
std::cout << "sentiment: " << std::get<std::string>(sentiment.data) << "\n";
}Set OPENAI_APIKEY in your environment before running provider-backed code.
From a clone of the ax repo:
npm run example -- cpp src/examples/cpp/long-agents/smart_defaults_agent.cppActive practice
Answer 2 in a row to learn this · attempt 1