ai() LLM Models

Use ai() to create provider clients and keep model traffic behind one Ax request shape.

import { AxAIOpenAIModel, ai } from '@ax-llm/ax';

const openai = ai({
  name: 'openai',
  apiKey: process.env.OPENAI_APIKEY!,
  config: { model: AxAIOpenAIModel.GPT4OMini },
});

What It Does

ai() selects a provider implementation from configuration and returns a client that Ax programs can call. The client handles chat, streaming, embeddings, media where supported, usage normalization, provider options, model keys, routing hooks, tracing, and runtime defaults.

flowchart LR
  A["Model key or alias"] --> B["Model catalog"]
  B --> C["Capability filter"]
  C --> D["Provider client"]
  D --> E["Request mapping"]
  E --> F["Provider API"]
  F --> G["Response normalization"]
  G --> H["Usage + trace"]

Core Call Shape

Create the client once near the application boundary, then pass it into forward(), streamingForward(), agents, flows, or optimizers.

client = ai(provider options)
result = program.forward(client, inputs)

Common Patterns

• Use a provider name and environment-backed API key.
• Set a default model in provider config when the app has one obvious model.
• Define model aliases when callers should choose fast, smart, or cheap instead of provider model IDs.
• Use OpenAI-compatible apiURL for compatible providers.
• Use model catalog helpers before runtime when the UI needs provider/model selectors.
• Use routers or balancers when provider fallback is part of the product.

Adaptive balancing

AxBalancer keeps its existing ordered failover behavior by default. Set strategy.type to adaptive to rank equivalent providers per chat request using learned reliability, successful latency, a deadline, and estimated cost. Configure badOutcomeCost in the same currency or unit as the route cost estimate.

import { AxBalancer, AxInMemoryBalancerStatsStore } from '@ax-llm/ax';

const statsStore = new AxInMemoryBalancerStatsStore();
const routeKeys = new Map<string, string>([
  [openai.getId(), 'openai-primary'],
  [anthropic.getId(), 'anthropic-primary'],
]);

const llm = AxBalancer.create([openai, anthropic] as const, {
  strategy: {
    type: 'adaptive',
    deadlineMs: 6_000,
    badOutcomeCost: 0.02,
    expectedTokens: { promptTokens: 1_200, completionTokens: 300 },
    namespace: 'support-summary-v1',
    routeKey: (service) => {
      const key = routeKeys.get(service.getId());
      if (!key) throw new Error('Missing stable route key.');
      return key;
    },
    slice: ({ options }) =>
      options?.customLabels?.workflow ?? 'default-workflow',
    statsStore,
    onRoutingEvent: (event) => {
      if (event.type === 'selected' || event.type === 'fallback') {
        console.log('route:', event);
      }
    },
  },
});

Use the native stats-store option for authoritative decision state. The built-in in-memory store can be shared by balancers in one process; multi-process applications can implement AxBalancerStatsStore with an atomic Redis or database update. The routing-event hook is best-effort telemetry, not routing state. Stable route keys are required with a shared store, and namespace plus slice keep unrelated traffic from learning from each other.

Adaptive balancing does not inspect prompt meaning or decide which model is best for a task. The application defines acceptable substitutes through shared logical aliases.

Provider clients

OpenAI

import { AxAIOpenAIModel, ai } from '@ax-llm/ax';

const openai = ai({
  name: 'openai',
  apiKey: process.env.OPENAI_APIKEY!,
  config: { model: AxAIOpenAIModel.GPT4OMini },
});

OpenAI Responses

const responses = ai({
  name: 'openai-responses',
  apiKey: process.env.OPENAI_APIKEY!,
  config: { model: 'gpt-4.1-mini' },
});

Claude / Anthropic

import { AxAIAnthropicModel, ai } from '@ax-llm/ax';

const claude = ai({
  name: 'anthropic',
  apiKey: process.env.ANTHROPIC_APIKEY!,
  config: { model: AxAIAnthropicModel.Claude48Opus },
});

Gemini

import { AxAIGoogleGeminiModel, ai } from '@ax-llm/ax';

const gemini = ai({
  name: 'google-gemini',
  apiKey: process.env.GOOGLE_APIKEY!,
  config: { model: AxAIGoogleGeminiModel.Gemini25Flash },
});

OpenAI-Compatible Providers

Use apiURL when a provider shares the OpenAI wire shape but uses a different host or model naming scheme.

const compatible = ai({
  name: 'openai',
  apiKey: process.env.PROVIDER_API_KEY!,
  apiURL: 'https://provider.example/v1',
  config: { model: 'provider/model-name' },
});

Embeddings and audio

const { embeddings } = await openai.embed({
  texts: ['typed LLM programs', 'runtime agents'],
  embedModel: 'text-embedding-3-small',
});

const transcript = await openai.transcribe({
  audio: { data: base64Wav, format: 'wav' },
  model: 'gpt-4o-mini-transcribe',
  language: 'en',
});

const speech = await openai.speak({ text: transcript.text, model: 'gpt-4o-mini-tts', voice: 'alloy' });

Practical Notes

• Prefer provider factories over direct provider classes in new code.
• Use model catalog and provider-scoring helpers when choosing between providers.
• Use a multi-service router to dispatch caller-selected model keys; use a balancer for fallback or adaptive operational routing across equivalent services.
• Keep public provider examples separate from internal conformance fixtures.
• Trace provider requests, token usage, estimated cost, and routing decisions in production.

See ai() API.