AxAIServiceImpl

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1421

Type Parameters

Methods

buildCacheCreateOp()?

optional buildCacheCreateOp(req: Readonly<AxInternalChatRequest<TModel>>, options: Readonly<AxAIServiceOptions>): 
  | AxContextCacheOperation
  | undefined;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1520

Optional: Build a context cache creation operation. Called when a new cache needs to be created from the request.

Parameters

Returns

| AxContextCacheOperation | undefined

buildCacheDeleteOp()?

optional buildCacheDeleteOp(cacheName: string): AxContextCacheOperation;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1536

Optional: Build a context cache deletion operation.

Parameters

Returns

AxContextCacheOperation

buildCacheUpdateTTLOp()?

optional buildCacheUpdateTTLOp(cacheName: string, ttlSeconds: number): AxContextCacheOperation;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1528

Optional: Build a context cache TTL update operation.

Parameters

Returns

AxContextCacheOperation

classifyStreamErrorStatus()?

optional classifyStreamErrorStatus(resp: Readonly<TChatResponseDelta>): number | undefined;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1455

Optional: classify a raw streaming delta that carries a transient error into the HTTP status it corresponds to (e.g. Anthropic’s HTTP-200 overloaded_error SSE event → 529). The base layer applies the same retryable-status policy used for real HTTP status errors, so a streaming overload is retried-with-backoff before any failover — matching the non-streaming path. Return undefined for normal deltas (the common case).

Parameters

Returns

number | undefined

createChatReq()

createChatReq(req: Readonly<AxInternalChatRequest<TModel>>, config?: Readonly<AxAIServiceOptions>): 
  | [AxAPI, TChatRequest]
| Promise<[AxAPI, TChatRequest]>;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1433

Parameters

Returns

| [AxAPI, TChatRequest] | Promise<[AxAPI, TChatRequest]>

createChatResp()

createChatResp(resp: Readonly<TChatResponse>, request?: Readonly<AxInternalChatRequest<TModel>>): AxChatResponse;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1438

Parameters

Returns

AxChatResponse

createChatStreamResp()?

optional createChatStreamResp(resp: Readonly<TChatResponseDelta>, state: object): AxChatResponse;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1443

Parameters

Returns

AxChatResponse

createEmbedReq()?

optional createEmbedReq(req: Readonly<AxInternalEmbedRequest<TEmbedModel>>, config?: Readonly<AxAIServiceOptions>): 
  | [AxAPI, TEmbedRequest]
| Promise<[AxAPI, TEmbedRequest]>;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1459

Parameters

Returns

| [AxAPI, TEmbedRequest] | Promise<[AxAPI, TEmbedRequest]>

createEmbedResp()?

optional createEmbedResp(resp: Readonly<TEmbedResponse>): AxEmbedResponse;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1464

Parameters

Returns

AxEmbedResponse

getContextCacheToolState()?

optional getContextCacheToolState(req: Readonly<AxInternalChatRequest<TModel>>, options: Readonly<AxAIServiceOptions>): 
  | {
  functionCall?:   | "auto"
     | "none"
     | "required"
     | {
     function: {
        name: string;
     };
     type: "function";
   };
  functions?: Readonly<{
     cache?: boolean;
     description: string;
     execution?: "blocking" | "background";
     name: string;
     parameters?: AxFunctionJSONSchema;
  }>[];
}
  | undefined;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1506

Optional: Return tool/function-call state that becomes part of the provider’s immutable explicit-cache prefix.

This is used for cache identity and token estimation when a provider treats tool state as cached prefix state even if the generic request does not mark functions with cache: true.

Parameters

Returns

| { functionCall?: | "auto" | "none" | "required" | { function: { name: string; }; type: "function"; }; functions?: Readonly<{ cache?: boolean; description: string; execution?: "blocking" | "background"; name: string; parameters?: AxFunctionJSONSchema; }>[]; } | undefined

getModelConfig()

getModelConfig(): AxModelConfig;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1483

Returns

AxModelConfig

getTokenUsage()

getTokenUsage(): AxTokenUsage | undefined;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1485

Returns

AxTokenUsage | undefined

prepareCachedChatReq()?

optional prepareCachedChatReq(
   req: Readonly<AxInternalChatRequest<TModel>>, 
   options: Readonly<AxAIServiceOptions>, 
existingCacheName?: string): Promise<AxPreparedChatRequest<TChatRequest>>;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1492

Optional: Prepare a chat request with context cache support. Providers implement this to support explicit context caching. Returns cache operations to execute and the modified request.

Parameters

Returns

Promise<AxPreparedChatRequest<TChatRequest>>

resolveReasoningEffort()?

optional resolveReasoningEffort(model: TModel, config: Readonly<AxAIServiceOptions>): string | undefined;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1472

Optional: the reasoning effort a chat request for model will send, or undefined when it sends none. Lets the base layer tell whether the model reasons on that request, which decides whether it accepts sampling parameters (see AxModelInfo.supported.samplingWithoutReasoning).

Parameters

Returns

string | undefined

samplingModelInfo()?

optional samplingModelInfo(model: TModel): Readonly<AxModelInfo> | undefined;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1481

Optional: model info for the sampling filter when the AI has none for model (the OpenAI o-series on profiles that carry no model info).

Parameters

Returns

Readonly<AxModelInfo> | undefined

supportsContextCache()?

optional supportsContextCache(model: TModel): boolean;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1542

Optional: Check if explicit context caching is supported (e.g., Gemini). Explicit caching creates a separate cache resource with an ID.

Parameters

Returns

boolean

supportsImplicitCaching()?

optional supportsImplicitCaching(model: TModel): boolean;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1548

Optional: Check if implicit context caching is supported (e.g., Anthropic). Implicit caching marks content in the request; provider handles caching automatically.

Parameters

Returns

boolean

validateChatReq()?

optional validateChatReq(req: Readonly<AxInternalChatRequest<TModel>>): void;

Defined in: https://github.com/ax-llm/ax/blob/3d5aa8dac28a2637e4150f022dcd02e6fc3309f0/src/ax/ai/types.ts#L1431

Validate original tools/history before automatic prompt-mode emulation.

Parameters

Returns

void