AxAIServiceImpl

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1089

Type Parameters

Methods

buildCacheCreateOp()?

optional buildCacheCreateOp(req: Readonly<AxInternalChatRequest<TModel>>, options: Readonly<AxAIServiceOptions>): 
  | undefined
  | AxContextCacheOperation;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1165

Optional: Build a context cache creation operation. Called when a new cache needs to be created from the request.

Parameters

Returns

| undefined | AxContextCacheOperation

buildCacheDeleteOp()?

optional buildCacheDeleteOp(cacheName: string): AxContextCacheOperation;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1181

Optional: Build a context cache deletion operation.

Parameters

Returns

AxContextCacheOperation

buildCacheUpdateTTLOp()?

optional buildCacheUpdateTTLOp(cacheName: string, ttlSeconds: number): AxContextCacheOperation;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1173

Optional: Build a context cache TTL update operation.

Parameters

Returns

AxContextCacheOperation

classifyStreamErrorStatus()?

optional classifyStreamErrorStatus(resp: Readonly<TChatResponseDelta>): undefined | number;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1117

Optional: classify a raw streaming delta that carries a transient error into the HTTP status it corresponds to (e.g. Anthropic’s HTTP-200 overloaded_error SSE event → 529). The base layer applies the same retryable-status policy used for real HTTP status errors, so a streaming overload is retried-with-backoff before any failover — matching the non-streaming path. Return undefined for normal deltas (the common case).

Parameters

Returns

undefined | number

createChatReq()

createChatReq(req: Readonly<AxInternalChatRequest<TModel>>, config?: Readonly<AxAIServiceOptions>): 
  | [AxAPI, TChatRequest]
| Promise<[AxAPI, TChatRequest]>;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1098

Parameters

Returns

| [AxAPI, TChatRequest] | Promise<[AxAPI, TChatRequest]>

createChatResp()

createChatResp(resp: Readonly<TChatResponse>): AxChatResponse;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1103

Parameters

Returns

AxChatResponse

createChatStreamResp()?

optional createChatStreamResp(resp: Readonly<TChatResponseDelta>, state: object): AxChatResponse;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1105

Parameters

Returns

AxChatResponse

createEmbedReq()?

optional createEmbedReq(req: Readonly<AxInternalEmbedRequest<TEmbedModel>>, config?: Readonly<AxAIServiceOptions>): 
  | [AxAPI, TEmbedRequest]
| Promise<[AxAPI, TEmbedRequest]>;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1121

Parameters

Returns

| [AxAPI, TEmbedRequest] | Promise<[AxAPI, TEmbedRequest]>

createEmbedResp()?

optional createEmbedResp(resp: Readonly<TEmbedResponse>): AxEmbedResponse;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1126

Parameters

Returns

AxEmbedResponse

getContextCacheToolState()?

optional getContextCacheToolState(req: Readonly<AxInternalChatRequest<TModel>>, options: Readonly<AxAIServiceOptions>): 
  | undefined
  | {
  functionCall?:   | "auto"
     | "none"
     | "required"
     | {
     function: {
        name: string;
     };
     type: "function";
   };
  functions?: Readonly<{
     cache?: boolean;
     description: string;
     name: string;
     parameters?: AxFunctionJSONSchema;
  }>[];
};

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1151

Optional: Return tool/function-call state that becomes part of the provider’s immutable explicit-cache prefix.

This is used for cache identity and token estimation when a provider treats tool state as cached prefix state even if the generic request does not mark functions with cache: true.

Parameters

Returns

| undefined | { functionCall?: | "auto" | "none" | "required" | { function: { name: string; }; type: "function"; }; functions?: Readonly<{ cache?: boolean; description: string; name: string; parameters?: AxFunctionJSONSchema; }>[]; }

getModelConfig()

getModelConfig(): AxModelConfig;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1128

Returns

AxModelConfig

getTokenUsage()

getTokenUsage(): undefined | AxTokenUsage;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1130

Returns

undefined | AxTokenUsage

prepareCachedChatReq()?

optional prepareCachedChatReq(
   req: Readonly<AxInternalChatRequest<TModel>>, 
   options: Readonly<AxAIServiceOptions>, 
existingCacheName?: string): Promise<AxPreparedChatRequest<TChatRequest>>;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1137

Optional: Prepare a chat request with context cache support. Providers implement this to support explicit context caching. Returns cache operations to execute and the modified request.

Parameters

Returns

Promise<AxPreparedChatRequest<TChatRequest>>

supportsContextCache()?

optional supportsContextCache(model: TModel): boolean;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1187

Optional: Check if explicit context caching is supported (e.g., Gemini). Explicit caching creates a separate cache resource with an ID.

Parameters

Returns

boolean

supportsImplicitCaching()?

optional supportsImplicitCaching(model: TModel): boolean;

Defined in: https://github.com/ax-llm/ax/blob/3306475085414d36aee4411ad2466e08e53bef8a/src/ax/ai/types.ts#L1193

Optional: Check if implicit context caching is supported (e.g., Anthropic). Implicit caching marks content in the request; provider handles caching automatically.

Parameters

Returns

boolean