AxAgent Observability Rules (@ax-llm/ax)

This skill helps an LLM generate correct AxAgent observability code using @ax-llm/ax. Use when the user asks about axGlobals.onUsage, usageContext, centralized or multi-tenant usage accounting, actorTurnCallback, onContextEvent, agentStatusCallback, onFunctionCall, reportSuccess, reportFailure, getChatLog(), getUsage(), resetUsage(), debug traces, progress updates, or telemetry for AxAgent runs.

Install

Install only this skill for TypeScript:

Shell

npx skills add https://ax-llm.github.io/ax/typescript/ --skill 'ax-agent-observability'

Published skill file: ax-agent-observability/SKILL.md.

Source

Source: src/ax/skills/ax-agent-observability.md
Version: 23.0.5

Skill Instructions

Use this skill when an agent needs runtime visibility, progress reporting, tracing, usage accounting, or chat-log access. For ordinary agent setup use ax-agent. For RLM runtime policy use ax-agent-rlm. For memories and dynamic skill loading use ax-agent-memory-skills.

Choose The Smallest Hook

Need a quick prompt/runtime trace during development -> start with debug: true.
Need structured per-turn code, raw runtime result, formatted output, provider thoughts, or actor stage -> use actorTurnCallback.
Need context-pressure and compaction telemetry -> use onContextEvent.
Need real-time task progress emitted by actor code -> use agentStatusCallback.
Need every runtime function call before execution -> use onFunctionCall.
Need model prompts/responses after a run -> use getChatLog().
Need centralized chat/embed usage across APIs, users, agents, and services -> use axGlobals.onUsage plus usageContext.
Need token usage by actor/responder -> use getUsage() and resetUsage().
Need usage split by context and task stages -> use getStagedUsage().
Need Ax program traces -> use getTraces().
Do not add multiple hooks unless the user clearly needs each output stream.

Global Runtime Defaults

OpenTelemetry and debug defaults come from the shared Ax runtime surface:

TypeScript

import { axGlobals, axCreateDefaultColorLogger } from '@ax-llm/ax';
import { trace } from '@opentelemetry/api';

axGlobals.tracer = trace.getTracer('agent-app');
axGlobals.debug = true;
axGlobals.logger = axCreateDefaultColorLogger();
axGlobals.onUsage = (event) => usageQueue.enqueue(event);

These globals are live defaults for future AI, AxGen, AxFlow, and agent-internal model calls. Per-call or explicitly configured options still override axGlobals. Use AxAgent callbacks below when the caller needs structured agent-turn events rather than OpenTelemetry spans or debug logs.

Centralized Usage Observer

Use the process-wide usage observer for application accounting across many agents, API routes, tenants, and users. Keep getUsage() for inspecting one agent instance after a run.

TypeScript

import { axGlobals } from '@ax-llm/ax';

axGlobals.onUsage = (event) => {
  usageQueue.enqueue(event); // Must return immediately.
};

await supportAgent.forward(
  llm,
  { query: request.body.query },
  {
    usageContext: {
      tenantId: auth.tenantId,
      userId: auth.userId,
      requestId: request.id,
      runId: crypto.randomUUID(),
      feature: 'support-chat',
      attributes: { environment: 'production' },
    },
  }
);

Rules:

The observer receives one immutable normalized event for each completed chat or embedding call that reports provider usage. A fully consumed stream emits once; an unconsumed or cancelled stream may not emit.
Events include operation, ai, model, normalized tokens, streaming, optional context, and available session or remote request IDs.
Put stable attribution such as application or environment in AI service-level usageContext. Put tenant, user, request, run, and feature attribution in per-call or per-forward usageContext.
Per-call context overrides service defaults. attributes are shallow-merged.
The observer is best-effort and fail-open. Ax does not await it and ignores observer failures, so synchronously enqueue and persist or aggregate out of band.
The registration is process-wide. A new assignment replaces the previous observer; set axGlobals.onUsage = undefined during test teardown or shutdown when appropriate.
In multi-process or serverless deployments, send events to a shared durable pipeline. Do not treat an in-memory total as application-wide accounting.
Keep identifiers opaque and attributes low-cardinality. Avoid prompts, responses, secrets, and other sensitive payloads.
Token events do not estimate currency cost. Apply a versioned provider/model pricing table downstream.

For direct AI calls and the complete event shape, also read ax-ai.

Actor Turn Callback

Use actorTurnCallback when the caller needs structured telemetry for each actor turn.

What it gives you:

code: the normalized JavaScript code the actor produced
stage: which actor produced the turn (distiller or executor)
result: the raw untruncated runtime return value from executing that code
output: the formatted action-log output string after Ax normalizes and truncates it for prompt replay
thought: the actor model’s thought field when showThoughts is enabled and the provider returns one
executorResult: the full actor payload returned by the current actor stage, kept under this historical field name for compatibility
isError: whether the execution path for that turn was treated as an error
usage: token usage for this actor turn only
model: model used for this turn when explicitly set through executorModelPolicy
chatLogMessages: raw ChatML conversation for this turn, populated only when an actor turn callback is set

Use it for:

debug UIs that want to show code plus raw runtime results
tracing and analytics
capturing thought for internal diagnostics when supported by the provider
storing per-turn execution artifacts without scraping the prompt/action log

Important:

output is not raw stdout; it is the formatted replay string used in the action log.
result is the raw runtime result before Ax applies type-aware serialization and budget-proportional truncation.
thought is optional and only appears when the underlying AxGen call had showThoughts enabled and the provider actually returned a thought field.
actionLogEntryCount and guidanceLogEntryCount reflect the live log sizes after the turn is processed, including resumed runs.
actorTurnCallback fires for the configured agent instance. Child agents passed through functions: [...] should define their own callback if you need their internal actor turns; use onFunctionCall on the parent to observe the parent-side child-agent invocation.

Good pattern:

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  actorTurnCallback: ({
    stage,
    turn,
    actionLogEntryCount,
    guidanceLogEntryCount,
    code,
    result,
    output,
    thought,
    isError,
    usage,
    model,
  }) => {
    console.log({
      turn,
      stage,
      model,
      actionLogEntryCount,
      guidanceLogEntryCount,
      isError,
      code,
      rawResult: result,
      replayOutput: output,
      thought,
      usage,
    });
  },
  executorOptions: {
    model: 'gpt-5.4-mini',
    showThoughts: true,
  },
});

Callback type:

TypeScript

actorTurnCallback?: (turn: {
  stage: 'distiller' | 'executor';
  turn: number;
  actionLogEntryCount: number;
  guidanceLogEntryCount: number;
  executorResult: Record<string, unknown>;
  code: string;
  result: unknown;
  output: string;
  isError: boolean;
  thought?: string;
  usage?: AxProgramUsage[];
  model?: string;
  chatLogMessages?: ReadonlyArray<{ role: string; content: string }>;
}) => void | Promise<void>;

actorTurnCallback?: (turn: {
  stage: 'distiller' | 'executor';
  turn: number;
  actionLogEntryCount: number;
  guidanceLogEntryCount: number;
  executorResult: Record<string, unknown>;
  code: string;
  result: unknown;
  output: string;
  isError: boolean;
  thought?: string;
  usage?: AxProgramUsage[];
  model?: string;
  chatLogMessages?: ReadonlyArray<{ role: string; content: string }>;
}) => void | Promise<void>; // deprecated alias

Context Event Observability

Use onContextEvent when the caller needs structured telemetry about prompt pressure and compaction. It does not change model behavior directly; it is for logs, evals, and dashboards.

Events:

budget_check: character-based prompt pressure before an actor turn, with detailed metrics kept out of the actor prompt
checkpoint_created / checkpoint_cleared: checkpoint lifecycle events with covered turns and reason
tombstone_created: compact resolved-error summary creation
relevance_ranking: emitted once per ranked domain per forward when relevanceRanking is enabled; carries domain ('modules' | 'skills' | 'memories'), the shortlist ({ id, score }[], most relevant first), and suppressed (true when the low-confidence guard emitted no hint)
field_auto_promoted: emitted once per field per run when autoUpgrade keeps an oversized undeclared input value runtime-only; carries fieldName, originalChars, and promptPreviewChars (undefined when no inline preview was kept)

To measure whether the advisory hint helps, join per forward: relevance_ranking.shortlist ids against what the actor then loaded — for modules the internal discover calls (onFunctionCall with kind: 'internal', name: 'discover', args.request) plus the module part of external qualifiedNames; for skills onLoadedSkills / used(id); for memories onLoadedMemories / used(id).

Rules:

contextPressure in the actor prompt is intentionally compact (ok, watch, critical plus one short instruction).
Budget metrics are character-based for provider neutrality and are exposed through onContextEvent, not the actor prompt.
Callback errors are swallowed so telemetry cannot break the agent run.
Do not scrape actor prompts for pressure metrics.

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  contextPolicy: { preset: 'checkpointed', budget: 'balanced' },
  onContextEvent: (event) => {
    if (event.kind === 'budget_check') {
      console.log(event.pressure, event.mutablePromptChars);
    }
  },
});

Type:

TypeScript

onContextEvent?: (event: AxAgentContextEvent) => void | Promise<void>;

Agent Status Callback

Use agentStatusCallback when the caller wants real-time progress updates from the actor. When set, the actor can call await reportSuccess(message) and await reportFailure(message) in its JavaScript turns.

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  agentStatusCallback: (message, status) => {
    console.log(`[${status}] ${message}`);
  },
});

Rules:

agentStatusCallback receives (message: string, status: 'success' | 'failed').
When set, the actor prompt automatically includes reportSuccess(message) and reportFailure(message) as available runtime functions.
The actor is instructed to keep the user updated on task progress.
reportSuccess and reportFailure are reserved runtime names when the callback is configured.
Child agents inherit the callback via the RLM config.

Type:

TypeScript

agentStatusCallback?: (
  message: string,
  status: 'success' | 'failed'
) => void | Promise<void>;

On Function Call

Use onFunctionCall when the caller wants to observe every function call the actor makes from the JS runtime. It fires before the underlying function runs.

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  functions: [helperAgent, lookupOrderTool],
  onFunctionCall: ({ name, qualifiedName, args, kind }) => {
    console.log(`[${kind}] ${qualifiedName}`, args);
  },
});

Rules:

Receives { name, qualifiedName, args, kind }.
name is the bare function name, e.g. 'lookupOrder'.
qualifiedName is the namespaced name as the actor sees it, e.g. 'tools.lookupOrder'; for un-namespaced runtime globals it equals name.
args is the resolved positional/named arguments object (Record<string, unknown>).
kind is 'external' for caller-registered functions.
kind is 'internal' for agent-injected globals: child agents, discover, recall, and used.
Fires once per call, before the function executes.
Errors thrown inside the callback are swallowed so they cannot break the actor loop.
This is independent from the DSP-layer onFunctionCall on AxProgramForwardOptions; that hook is for LLM tool-calls and never fires under AxAgent because AxAgent injects functions as runtime globals.

Type:

TypeScript

onFunctionCall?: (call: {
  name: string;
  qualifiedName: string;
  args: Record<string, unknown>;
  kind: 'internal' | 'external';
}) => void | Promise<void>;

Chat Log, Usage, And Traces

AxAgent exposes actor and responder sub-programs. getChatLog() returns the same flat AxChatLogEntry[] shape as AxGen and AxFlow; use each entry’s optional name field to distinguish distiller, executor, and responder. getUsage() returns token usage split by actor/responder.

getChatLog()

Returns the full normalized chat history after any .forward() call. Each entry is one ai.chat() round-trip. Actor stages accumulate one entry per turn; the responder typically has one entry.

TypeScript

const log = myAgent.getChatLog();

for (const entry of log) {
  console.log(entry.name, entry.model);
  for (const msg of entry.messages) {
    console.log(`[${msg.role}]`, msg.content);
  }
}

Each AxChatLogEntry captures the full prompt sent to the model and its response:

TypeScript

type AxChatLogMessage =
  | { role: 'system'; content: string }
  | { role: 'user'; content: string }
  | { role: 'assistant'; content: string }
  | { role: 'tool'; name: string; content: string };

type AxChatLogEntry = {
  name?: string; // e.g. "distiller", "executor", "responder"
  model: string;
  messages: AxChatLogMessage[];
  modelUsage?: AxProgramUsage;
  stage?: 'ctx' | 'task';
};

getUsage()

Returns token usage split by actor/responder. Each sub-array contains one AxProgramUsage entry per model/run, merged by (ai, model) key.

TypeScript

const usage = myAgent.getUsage();
// { actor: AxProgramUsage[], responder: AxProgramUsage[] }

console.log('Actor tokens:', usage.actor[0]?.tokens);
console.log('Responder tokens:', usage.responder[0]?.tokens);

getStagedUsage()

Returns usage split by pipeline stage. The ctx stage has the distiller actor only; the task stage has the executor actor plus responder.

TypeScript

const staged = myAgent.getStagedUsage();
console.log(staged.ctx?.actor);
console.log(staged.task.actor);
console.log(staged.task.responder);

getTraces()

Returns Ax program traces for the agent pipeline. Use it when the caller needs trace data rather than chat messages or token summaries.

TypeScript

const traces = myAgent.getTraces();

resetUsage()

Resets both actor and responder usage at once:

TypeScript

myAgent.resetUsage();

Type signatures:

TypeScript

// AxAgent
agent.getChatLog(): readonly AxChatLogEntry[]
agent.getUsage(): { actor: AxProgramUsage[]; responder: AxProgramUsage[] }
agent.getStagedUsage(): { ctx?: AxAgentUsage; task: AxAgentUsage }
agent.getTraces(): AxProgramTrace[]
agent.resetUsage(): void

// AxGen / AxFlow
gen.getChatLog(): readonly AxChatLogEntry[]
gen.getUsage(): AxProgramUsage[]

Do Not Generate

Do not add both debug: true and actorTurnCallback unless the user wants both unstructured prompt/runtime visibility and structured telemetry.
Do not scrape actor prompts or action logs when a callback exposes the data directly.
Do not let observability callback failures break the agent run; Ax swallows callback errors for telemetry hooks.
Do not use DSP-layer onFunctionCall when the user wants AxAgent runtime function calls.
Do not enable showThoughts unless the user needs provider thought diagnostics and the provider supports it.
Do not use getUsage() as the centralized source of truth across shared agents or processes.
Do not perform database or network work inline in axGlobals.onUsage; enqueue and return.