AxAgent Observability Rules (@ax-llm/ax)

This skill helps an LLM generate correct AxAgent observability code using @ax-llm/ax. Use when the user asks about actorTurnCallback, onContextEvent, agentStatusCallback, onFunctionCall, reportSuccess, reportFailure, getChatLog(), getUsage(), resetUsage(), debug traces, progress updates, or telemetry for AxAgent runs.

Install

Install only this skill for TypeScript:

Shell

npx skills add https://ax-llm.github.io/ax/typescript/ --skill 'ax-agent-observability'

Published skill file: ax-agent-observability/SKILL.md.

Source

Source: src/ax/skills/ax-agent-observability.md
Version: 22.0.3

Skill Instructions

Use this skill when an agent needs runtime visibility, progress reporting, tracing, usage accounting, or chat-log access. For ordinary agent setup use ax-agent. For RLM runtime policy use ax-agent-rlm. For memories and dynamic skill loading use ax-agent-memory-skills.

Choose The Smallest Hook

Need a quick prompt/runtime trace during development -> start with debug: true.
Need structured per-turn code, raw runtime result, formatted output, provider thoughts, or actor stage -> use actorTurnCallback.
Need context-pressure and compaction telemetry -> use onContextEvent.
Need real-time task progress emitted by actor code -> use agentStatusCallback.
Need every runtime function call before execution -> use onFunctionCall.
Need model prompts/responses after a run -> use getChatLog().
Need token usage by actor/responder -> use getUsage() and resetUsage().
Need usage split by context and task stages -> use getStagedUsage().
Need Ax program traces -> use getTraces().
Do not add multiple hooks unless the user clearly needs each output stream.

Global Runtime Defaults

OpenTelemetry and debug defaults come from the shared Ax runtime surface:

TypeScript

import { axGlobals, axCreateDefaultColorLogger } from '@ax-llm/ax';
import { trace } from '@opentelemetry/api';

axGlobals.tracer = trace.getTracer('agent-app');
axGlobals.debug = true;
axGlobals.logger = axCreateDefaultColorLogger();

These globals are live defaults for future AI, AxGen, AxFlow, and agent-internal model calls. Per-call or explicitly configured options still override axGlobals. Use AxAgent callbacks below when the caller needs structured agent-turn events rather than OpenTelemetry spans or debug logs.

Actor Turn Callback

Use actorTurnCallback when the caller needs structured telemetry for each actor turn.

What it gives you:

code: the normalized JavaScript code the actor produced
stage: which actor produced the turn (distiller or executor)
result: the raw untruncated runtime return value from executing that code
output: the formatted action-log output string after Ax normalizes and truncates it for prompt replay
thought: the actor model’s thought field when showThoughts is enabled and the provider returns one
executorResult: the full actor payload returned by the current actor stage, kept under this historical field name for compatibility
isError: whether the execution path for that turn was treated as an error
usage: token usage for this actor turn only
model: model used for this turn when explicitly set through executorModelPolicy
chatLogMessages: raw ChatML conversation for this turn, populated only when an actor turn callback is set

Use it for:

debug UIs that want to show code plus raw runtime results
tracing and analytics
capturing thought for internal diagnostics when supported by the provider
storing per-turn execution artifacts without scraping the prompt/action log

Important:

output is not raw stdout; it is the formatted replay string used in the action log.
result is the raw runtime result before Ax applies type-aware serialization and budget-proportional truncation.
thought is optional and only appears when the underlying AxGen call had showThoughts enabled and the provider actually returned a thought field.
actionLogEntryCount and guidanceLogEntryCount reflect the live log sizes after the turn is processed, including resumed runs.
actorTurnCallback fires for the configured agent instance. Child agents passed through functions: [...] should define their own callback if you need their internal actor turns; use onFunctionCall on the parent to observe the parent-side child-agent invocation.

Good pattern:

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  actorTurnCallback: ({
    stage,
    turn,
    actionLogEntryCount,
    guidanceLogEntryCount,
    code,
    result,
    output,
    thought,
    isError,
    usage,
    model,
  }) => {
    console.log({
      turn,
      stage,
      model,
      actionLogEntryCount,
      guidanceLogEntryCount,
      isError,
      code,
      rawResult: result,
      replayOutput: output,
      thought,
      usage,
    });
  },
  executorOptions: {
    model: 'gpt-5.4-mini',
    showThoughts: true,
  },
});

Callback type:

TypeScript

actorTurnCallback?: (turn: {
  stage: 'distiller' | 'executor';
  turn: number;
  actionLogEntryCount: number;
  guidanceLogEntryCount: number;
  executorResult: Record<string, unknown>;
  code: string;
  result: unknown;
  output: string;
  isError: boolean;
  thought?: string;
  usage?: AxProgramUsage[];
  model?: string;
  chatLogMessages?: ReadonlyArray<{ role: string; content: string }>;
}) => void | Promise<void>;

actorTurnCallback?: (turn: {
  stage: 'distiller' | 'executor';
  turn: number;
  actionLogEntryCount: number;
  guidanceLogEntryCount: number;
  executorResult: Record<string, unknown>;
  code: string;
  result: unknown;
  output: string;
  isError: boolean;
  thought?: string;
  usage?: AxProgramUsage[];
  model?: string;
  chatLogMessages?: ReadonlyArray<{ role: string; content: string }>;
}) => void | Promise<void>; // deprecated alias

Context Event Observability

Use onContextEvent when the caller needs structured telemetry about prompt pressure and compaction. It does not change model behavior directly; it is for logs, evals, and dashboards.

Events:

budget_check: character-based prompt pressure before an actor turn, with detailed metrics kept out of the actor prompt
checkpoint_created / checkpoint_cleared: checkpoint lifecycle events with covered turns and reason
tombstone_created: compact resolved-error summary creation

Rules:

contextPressure in the actor prompt is intentionally compact (ok, watch, critical plus one short instruction).
Budget metrics are character-based for provider neutrality and are exposed through onContextEvent, not the actor prompt.
Callback errors are swallowed so telemetry cannot break the agent run.
Do not scrape actor prompts for pressure metrics.

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  contextPolicy: { preset: 'checkpointed', budget: 'balanced' },
  onContextEvent: (event) => {
    if (event.kind === 'budget_check') {
      console.log(event.pressure, event.mutablePromptChars);
    }
  },
});

Type:

TypeScript

onContextEvent?: (event: AxAgentContextEvent) => void | Promise<void>;

Agent Status Callback

Use agentStatusCallback when the caller wants real-time progress updates from the actor. When set, the actor can call await reportSuccess(message) and await reportFailure(message) in its JavaScript turns.

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  agentStatusCallback: (message, status) => {
    console.log(`[${status}] ${message}`);
  },
});

Rules:

agentStatusCallback receives (message: string, status: 'success' | 'failed').
When set, the actor prompt automatically includes reportSuccess(message) and reportFailure(message) as available runtime functions.
The actor is instructed to keep the user updated on task progress.
reportSuccess and reportFailure are reserved runtime names when the callback is configured.
Child agents inherit the callback via the RLM config.

Type:

TypeScript

agentStatusCallback?: (
  message: string,
  status: 'success' | 'failed'
) => void | Promise<void>;

On Function Call

Use onFunctionCall when the caller wants to observe every function call the actor makes from the JS runtime. It fires before the underlying function runs.

TypeScript

const supportAgent = agent('query:string -> answer:string', {
  contextFields: ['query'],
  runtime,
  functions: [helperAgent, lookupOrderTool],
  onFunctionCall: ({ name, qualifiedName, args, kind }) => {
    console.log(`[${kind}] ${qualifiedName}`, args);
  },
});

Rules:

Receives { name, qualifiedName, args, kind }.
name is the bare function name, e.g. 'lookupOrder'.
qualifiedName is the namespaced name as the actor sees it, e.g. 'tools.lookupOrder'; for un-namespaced runtime globals it equals name.
args is the resolved positional/named arguments object (Record<string, unknown>).
kind is 'external' for caller-registered functions.
kind is 'internal' for agent-injected globals: child agents, discover, recall, and used.
Fires once per call, before the function executes.
Errors thrown inside the callback are swallowed so they cannot break the actor loop.
This is independent from the DSP-layer onFunctionCall on AxProgramForwardOptions; that hook is for LLM tool-calls and never fires under AxAgent because AxAgent injects functions as runtime globals.

Type:

TypeScript

onFunctionCall?: (call: {
  name: string;
  qualifiedName: string;
  args: Record<string, unknown>;
  kind: 'internal' | 'external';
}) => void | Promise<void>;

Chat Log, Usage, And Traces

AxAgent exposes actor and responder sub-programs. getChatLog() returns the same flat AxChatLogEntry[] shape as AxGen and AxFlow; use each entry’s optional name field to distinguish distiller, executor, and responder. getUsage() returns token usage split by actor/responder.

getChatLog()

Returns the full normalized chat history after any .forward() call. Each entry is one ai.chat() round-trip. Actor stages accumulate one entry per turn; the responder typically has one entry.

TypeScript

const log = myAgent.getChatLog();

for (const entry of log) {
  console.log(entry.name, entry.model);
  for (const msg of entry.messages) {
    console.log(`[${msg.role}]`, msg.content);
  }
}

Each AxChatLogEntry captures the full prompt sent to the model and its response:

TypeScript

type AxChatLogMessage =
  | { role: 'system'; content: string }
  | { role: 'user'; content: string }
  | { role: 'assistant'; content: string }
  | { role: 'tool'; name: string; content: string };

type AxChatLogEntry = {
  name?: string; // e.g. "distiller", "executor", "responder"
  model: string;
  messages: AxChatLogMessage[];
  modelUsage?: AxProgramUsage;
  stage?: 'ctx' | 'task';
};

getUsage()

Returns token usage split by actor/responder. Each sub-array contains one AxProgramUsage entry per model/run, merged by (ai, model) key.

TypeScript

const usage = myAgent.getUsage();
// { actor: AxProgramUsage[], responder: AxProgramUsage[] }

console.log('Actor tokens:', usage.actor[0]?.tokens);
console.log('Responder tokens:', usage.responder[0]?.tokens);

getStagedUsage()

Returns usage split by pipeline stage. The ctx stage has the distiller actor only; the task stage has the executor actor plus responder.

TypeScript

const staged = myAgent.getStagedUsage();
console.log(staged.ctx?.actor);
console.log(staged.task.actor);
console.log(staged.task.responder);

getTraces()

Returns Ax program traces for the agent pipeline. Use it when the caller needs trace data rather than chat messages or token summaries.

TypeScript

const traces = myAgent.getTraces();

resetUsage()

Resets both actor and responder usage at once:

TypeScript

myAgent.resetUsage();

Type signatures:

TypeScript

// AxAgent
agent.getChatLog(): readonly AxChatLogEntry[]
agent.getUsage(): { actor: AxProgramUsage[]; responder: AxProgramUsage[] }
agent.getStagedUsage(): { ctx?: AxAgentUsage; task: AxAgentUsage }
agent.getTraces(): AxProgramTrace[]
agent.resetUsage(): void

// AxGen / AxFlow
gen.getChatLog(): readonly AxChatLogEntry[]
gen.getUsage(): AxProgramUsage[]

Do Not Generate

Do not add both debug: true and actorTurnCallback unless the user wants both unstructured prompt/runtime visibility and structured telemetry.
Do not scrape actor prompts or action logs when a callback exposes the data directly.
Do not let observability callback failures break the agent run; Ax swallows callback errors for telemetry hooks.
Do not use DSP-layer onFunctionCall when the user wants AxAgent runtime function calls.
Do not enable showThoughts unless the user needs provider thought diagnostics and the provider supports it.