AxAIServiceOptions

type AxAIServiceOptions = object;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1054

Runtime options for AI service requests.

These options control how requests are made to the AI service, including debugging, rate limiting, streaming, function calling, and extended thinking.

Example

const options: AxAIServiceOptions = {
  stream: true,
  thinkingTokenBudget: 'medium',
  debug: true
};
await gen.forward(ai, values, options);

Properties

abortSignal?

optional abortSignal: AbortSignal;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1104

AbortSignal for cancelling in-flight requests.

asyncMode?

optional asyncMode: "auto" | "off";

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1056

Automatically use native async capabilities when available.

beta?

optional beta: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1230

Provider-specific hint to opt in to beta API paths when available.

Currently used by Google Gemini on Vertex AI to route requests through v1beta1 instead of the default stable v1 endpoint.

contextCache?

optional contextCache: AxContextCacheOptions;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1261

Context caching options for large prompt prefixes.

When enabled, large prompt prefixes can be cached for cost savings and lower latency on subsequent requests.

Currently supported by: Google Gemini/Vertex AI

control?

optional control: AxRunControl;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1057

corsProxy?

optional corsProxy: string;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1243

CORS proxy URL for browser environments.

When running in a browser, API calls may be blocked by CORS. Specify a proxy URL to route requests through.

Example

'https://cors-anywhere.herokuapp.com/'

customLabels?

optional customLabels: Record<string, string>;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1271

Custom labels for OpenTelemetry metrics.

These labels are merged with axGlobals.customLabels (service-level options override global settings).

Example

{ environment: 'production', feature: 'search' }

debug?

optional debug: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1068

Enable debug logging for troubleshooting.

When true, logs detailed information about prompts, responses, and the generation pipeline. Useful for understanding AI behavior.

debugHideSystemPrompt?

optional debugHideSystemPrompt: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1137

Hide system prompt in debug output (for cleaner logs).

excludeContentFromTrace?

optional excludeContentFromTrace: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1101

Exclude message content from OpenTelemetry traces (for privacy).

executionPath?

optional executionPath: string;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1059

Stable execution scope for targeted run updates.

fetch?

optional fetch: typeof fetch;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1082

Custom fetch implementation (useful for proxies or custom HTTP handling).

functionCallMode?

optional functionCallMode: "auto" | "native" | "prompt";

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1161

How to handle function/tool calling.

• 'auto' - Let the provider decide the best approach (default)
• 'native' - Use the provider’s native function calling API. Fails if the model doesn’t support it.
• 'prompt' - Simulate function calling via prompt engineering. Works with any model but may be less reliable.

Default

'auto'

functionCallSource?

optional functionCallSource: "ax" | "caller";

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1061

Internal

Identifies whether a forced tool choice came from Ax itself.

includeRequestBodyInErrors?

optional includeRequestBodyInErrors: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1282

Whether to include the request body in AxAIServiceError messages.

When false, the request body is omitted from thrown errors. Useful when requests may contain sensitive data (API keys, PII) or large base64-encoded content that would bloat error logs.

Default

true

logger?

optional logger: AxLoggerFunction;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1107

Custom logger function for debug output.

meter?

optional meter: Meter;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1091

OpenTelemetry meter for metrics collection.

promptCacheKey?

optional promptCacheKey: string;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1123

Stable per-conversation key for OpenAI prompt caching.

Routes the request to the shard its cache lives on, so it must stay the same for every turn of a conversation and differ between unrelated ones. OpenAI advises staying under roughly 15 requests/minute per key.

Falls back to sessionId when unset.

Currently used by: OpenAI GPT-5.6+ and Meta Responses/Chat Completions.

promptCacheRetention?

optional promptCacheRetention: "in_memory" | "24h";

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1126

Meta Responses prompt-cache retention policy.

rateLimiter?

optional rateLimiter: AxRateLimiterFunction;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1079

Custom rate limiter function to control request throughput.

retry?

optional retry: Partial<RetryConfig>;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1251

Retry configuration for failed requests.

Controls automatic retry behavior for transient errors (rate limits, timeouts, server errors).

serviceTier?

optional serviceTier: AxServiceTier;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1204

Portable inference scheduling policy.

auto delegates to provider routing when supported and otherwise omits the provider field. Explicit tiers are rejected when the selected provider/model has not verified them.

sessionId?

optional sessionId: string;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1110

Session identifier for conversation tracking and memory isolation.

showThoughts?

optional showThoughts: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1214

Include the model’s thinking/reasoning in the output.

When true and thinkingTokenBudget is set, the model’s internal reasoning is included in the response. Useful for debugging and understanding AI behavior.

Default

false

stepIndex?

optional stepIndex: number;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1233

Internal: Current step index for multi-step operations.

stream?

optional stream: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1148

Enable streaming responses.

When true, the AI returns responses as a stream of chunks, enabling real-time display of generated text.

thinkingTokenBudget?

optional thinkingTokenBudget: "minimal" | "low" | "medium" | "high" | "highest" | "none";

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1189

Token budget for extended thinking (chain-of-thought reasoning).

Extended thinking allows models to “think through” complex problems before responding. Higher budgets allow deeper reasoning but cost more.

Approximate token allocations:

• 'none' - Disabled (default)
• 'minimal' - ~1,000 tokens (~750 words of thinking)
• 'low' - ~4,000 tokens
• 'medium' - ~10,000 tokens
• 'high' - ~20,000 tokens
• 'highest' - ~32,000+ tokens (provider maximum)

Provider support:

• Anthropic Claude: Full support with claude-sonnet-4 and above
• OpenAI: Supported with o1/o3 models (uses reasoning_effort)
• Google: Supported with Gemini 2.0 Flash Thinking
• DeepSeek: Supported with DeepSeek V4 models

Example

// Enable medium thinking for complex reasoning
await gen.forward(ai, values, { thinkingTokenBudget: 'medium' });

timeout?

optional timeout: number;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1098

Request timeout in milliseconds.

Default

300000 (5 minutes)

traceContext?

optional traceContext: Context;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1140

OpenTelemetry trace context for distributed tracing.

tracer?

optional tracer: Tracer;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1088

OpenTelemetry tracer for distributed tracing.

usageContext?

optional usageContext: AxUsageContext;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1134

Request-scoped attribution included in normalized usage events.

Per-call values override service defaults. Custom attributes are shallow-merged with per-call keys taking precedence.

useExpensiveModel?

optional useExpensiveModel: "yes";

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1222

Hint to use a more capable (and expensive) model for complex tasks.

Some providers offer tiered models. Setting this to ‘yes’ requests the higher-capability tier when available.

verbose?

optional verbose: boolean;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1076

Enable low-level HTTP request/response logging.

More verbose than debug. Shows raw HTTP traffic including headers. Useful for debugging API issues.

webSocket?

optional webSocket: any;

Defined in: https://github.com/ax-llm/ax/blob/e690b4823b8309e258594064abe941d4cff6ef89/src/ax/ai/types.ts#L1085

Custom WebSocket constructor for providers that use realtime WebSocket transports.