Auto routing

Send model auto and the gateway picks a tier for each request before billing; read cruxy.routed_tier to see which tier served it.

Send model: "auto" instead of a tier id and the gateway's server-side router picks the tier from the request content. It resolves auto to kavi, vaani, or mira before billing and before the upstream call, so auto never reaches a model, and every response names the tier that served it.

curl https://api.cruxy.ai/api/v1/chat \
  -H "Authorization: Bearer $CRUXY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "stream": false,
    "messages": [
      { "role": "user", "content": "Summarize the causes of the monsoon in three sentences." }
    ]
  }'

const res = await fetch('https://api.cruxy.ai/api/v1/chat', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.CRUXY_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'auto',
    stream: false,
    messages: [
      { role: 'user', content: 'Summarize the causes of the monsoon in three sentences.' },
    ],
  }),
})
 
const completion = await res.json()
console.log(completion.cruxy.routed_tier) // the tier that served the call
console.log(completion.cruxy.reasoning_effort) // the effort it ran at
console.log(completion.message.content)

Read the tier that served

The routed tier comes back on every response, in a field that depends on the endpoint:

usage.cost.tier also names the tier that served the request. The router can choose differently for two similar requests, so log routed_tier with each request_id and branch on the value you read, never on the tier you expected.

{
  "cruxy": {
    "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "routed_tier": "kavi",
    "reasoning_effort": "high"
  }
}

What auto does for you

• Picks a healthy tier. The router chooses a tier that is available before the call is sent. Once sent, a call never moves to another tier. When every tier is unavailable, the request is 503 all_models_unavailable; retry it with backoff.
• Chooses the reasoning effort. With reasoning_effort omitted, auto chooses the effort per request and echoes it as cruxy.reasoning_effort (routing.reasoning_effort on the native wire). Send reasoning_effort to set it yourself.
• Routes images to a tier that reads them. A turn whose messages attach an image goes to a tier with supports_vision. Every tier reads images today.
• Moves to mira instead of failing a budget. When a budget would refuse the request and mira can serve it, the gateway moves an auto request to mira rather than answering 429 budget_exhausted. On the native wire, routing.mode is auto_degraded. Rate limits and budgets explains the budgets.

Size max_tokens for any tier

max_tokens is clamped to the cap of the tier the router picks: 8192 on kavi, 4096 on vaani, and 2048 on mira. If you set a small max_tokens on an auto request, also send reasoning_effort. Otherwise the router may pick kavi, whose default thinking can spend the whole budget and end the turn with finish_reason: "length" and empty content; see Reasoning effort.

When to name a tier instead

• You need the same behavior on every call: one output cap, one non-streaming deadline, one reasoning default.
• You are measuring. An evaluation or benchmark compares like with like, so pin the tier it measures.
• You count tokens first. POST /api/v1/tokens/count rejects auto with 400; count against the tier you plan to send.

Choosing a tier compares the three tiers.

Next steps

• Choosing a tier: what each tier is for.
• Reasoning effort: set the effort instead of letting auto choose.
• Streaming: read the routed tier from the role primer.
• Errors and retries: retry 503 all_models_unavailable.