Errors and retries

Read the error envelope, look up every error code the API defines, and decide which failures to retry and how long to wait.

A failed request returns a non-2xx status and a JSON error body. Read the HTTP status first, then branch on the body's code: the code is stable, while the wording of the message can change.

The error envelope

{
  "code": "invalid_request",
  "error": "invalid JSON body",
  "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}

Some codes add fields next to these three. Each is optional, and present only on the codes listed:

Ignore fields you do not recognize.

Three surfaces use a different shape:

• POST /v1/chat/completions uses the OpenAI envelope, {"error": {"message", "type", "param", "code"}}. Its code is the same Cruxy code listed here; type is a coarse category derived from the status (invalid_request_error, authentication_error, permission_error, not_found_error, rate_limit_error, or server_error); param is always null.
• Device login answers a pending or refused poll with an OAuth-style body; see Get your API key.
• A POST /api/v2/chat stream that fails after it opens ends with an error event carrying code, error, and request_id.

Handle errors and retry safely

This request sends an Idempotency-Key, so once it succeeds, running it again with the same key replays the stored response instead of calling the model and billing again. The gateway stores only a success, so a retry after an error runs the request again under the same key. The TypeScript version retries the codes the API marks retryable, waiting Retry-After when it is sent, and retries a request that got no response with the same key:

IDEMPOTENCY_KEY=$(uuidgen)
 
# -i prints the status line and headers, including Cruxy-Version and the
# X-RateLimit-* headers. Run it twice: once the first call succeeds, the
# second replays it.
curl -i https://api.cruxy.ai/api/v1/chat \
  -H "Authorization: Bearer $CRUXY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $IDEMPOTENCY_KEY" \
  -d '{
    "model": "vaani",
    "stream": false,
    "messages": [
      { "role": "user", "content": "Hello" }
    ]
  }'

class CruxyError extends Error {
  constructor(
    readonly status: number,
    readonly code: string,
    readonly requestId: string,
    message: string,
  ) {
    super(`${status} ${code}: ${message} (request_id ${requestId})`)
  }
}
 
// Your retry budget and timeout. Tune them for your workload; they are not API limits.
// TIMEOUT_MS sits above the longest tier deadline (90 s), so a slow model
// reaches you as 504 upstream_timeout rather than as your own abort.
const MAX_ATTEMPTS = 4
const BASE_DELAY_MS = 500
const TIMEOUT_MS = 120_000
 
// Transient failures the API marks retryable. upstream_timeout is left out:
// the gateway already made its one attempt, and the same request usually
// times out again.
const RETRYABLE = new Set([
  'rate_limited',
  'credits_unavailable',
  'headless_concurrency_limit',
  'limits_unavailable',
  'upstream_overloaded',
  'model_unavailable',
  'all_models_unavailable',
  'server_shutting_down',
  'internal_error',
])
 
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms))
// Exponential backoff with full jitter.
const backoff = (attempt: number) => Math.random() * BASE_DELAY_MS * 2 ** attempt
 
async function chat(body: Record<string, unknown>) {
  // One key per logical request, reused on every retry of it.
  const idempotencyKey = crypto.randomUUID()
 
  for (let attempt = 1; ; attempt++) {
    let res: Response
    try {
      res = await fetch('https://api.cruxy.ai/api/v1/chat', {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${process.env.CRUXY_API_KEY}`,
          'Content-Type': 'application/json',
          'Idempotency-Key': idempotencyKey,
        },
        body: JSON.stringify({ ...body, stream: false }),
        signal: AbortSignal.timeout(TIMEOUT_MS),
      })
    } catch (err) {
      // No response: the call may or may not have run. If it succeeded, the
      // same key replays its stored response rather than running and billing
      // it a second time; if it failed, the same key runs it again.
      if (attempt >= MAX_ATTEMPTS) throw err
      await sleep(backoff(attempt))
      continue
    }
 
    if (res.ok) return res.json()
 
    const { code, error, request_id } = await res.json()
 
    // An error is never stored, so the same key runs the request again.
    if (RETRYABLE.has(code) && attempt < MAX_ATTEMPTS) {
      const retryAfter = Number(res.headers.get('Retry-After'))
      await sleep(retryAfter > 0 ? retryAfter * 1000 : backoff(attempt))
      continue
    }
 
    throw new CruxyError(res.status, code, request_id, error)
  }
}
 
const completion = await chat({
  model: 'vaani',
  messages: [{ role: 'user', content: 'Hello' }],
})
console.log(completion.message.content)

Which failures to retry

Rate limits and budgets explains the difference between a rate limit and a budget, and how to read your remaining headroom.

Timeouts

With stream: false, the gateway makes one attempt at the model, bounded by the tier's deadline:

An attempt that runs past the deadline is cancelled and answered 504 upstream_timeout, and nothing is billed. The gateway does not retry a timeout: a turn that needed longer needs longer again. Send stream: true instead, since a stream may run for up to five minutes, or ask for less with a lower max_tokens or reasoning_effort. Reliability collects the deadlines, retries, and stream checks in one place.

Other upstream failures, such as an overloaded or failing model or a dropped connection, are retried inside the gateway a bounded number of times, with backoff, before it answers 503 upstream_overloaded or model_unavailable. On stream: true that happens only while the stream opens, never after a byte has reached you. A call never moves to another tier once it is sent: model: "auto" picks a healthy tier before the call, and an explicit tier that keeps failing is refused at once with 503 model_unavailable while it cools down.

Back off

• Honour Retry-After. Every 429 carries it, and so does a 503 limits_unavailable. On rate_limited, X-RateLimit-Reset gives the same moment as an absolute Unix time in seconds. On a budget or spend-cap refusal, Retry-After counts down to reset_at instead, while the X-RateLimit-* headers still describe the per-minute request window.
• Otherwise back off exponentially, with jitter. Double the wait after each failed attempt and randomize it, so many clients that failed together do not retry together.
• Cap the attempts and the total wait. A retry budget keeps a persistent failure from turning into a stuck request.
• Log request_id for every failure, retried or not.

Make retries idempotent

Idempotency-Key makes a retry safe. The first call with a key is served, and when it succeeds its response is stored for 24 hours. A later or concurrent call with the same key and the same body replays that response byte for byte, without calling the model again and without billing again; a concurrent duplicate waits for the first call to finish.

Only a success is stored. An error, a 429 or any 5xx included, releases the key, so a retry with the same key and body runs the request again. Reusing the key is the intended way to retry a 429 or a 5xx.

• Scope: a key belongs to your API key and to the endpoint. On a conversation append, it belongs to the conversation.
• Conflicts: the same key with a different body is 409 idempotency_conflict.
• Large responses: a response over 256 KiB is served but not stored, so a retry runs again. It is still billed once.
• Length: keys are at most 255 characters.

These endpoints accept it:

• POST /api/v1/chat, POST /api/v2/chat, and POST /v1/chat/completions
• POST /api/v1/embeddings, POST /api/v1/rerank, and POST /api/v1/moderations
• POST /api/v1/conversations/{id}/messages, which stores its 201 and replays it without appending a second message

Error codes

Every code the API defines, grouped by the status it is returned with.

400 Bad Request

The request itself is wrong. Fix it; sending it again unchanged fails the same way.

401 Unauthorized

403 Forbidden

The credential is genuine, but it cannot make this call. Waiting never widens a credential, so retrying cannot succeed.

404 Not Found

The target does not exist, or you cannot see it. The API gives the same answer for an id that is missing and one you cannot see, so a 404 never confirms that an id exists.

409 Conflict

The request conflicts with existing state. Only idempotency_conflict has a retry path.

413 Payload Too Large

422 Unprocessable Content

429 Too Many Requests

Every 429 carries Retry-After, and nothing is billed.

5xx Server errors

On chat, every model failure adds tier, and none is billed.

Stream errors

A stream that fails after it opens has already sent its 200, so the code arrives in the stream, not the status:

• POST /api/v2/chat ends with an error event whose code is one of upstream_error, upstream_overloaded, upstream_timeout, model_unavailable, server_shutting_down, or internal_error.
• POST /v1/chat/completions ends with a data: {"error": {...}} frame.
• POST /api/v1/chat sends no error frame: the stream ends without data: [DONE]; see Streaming.

Codes in request logs

client_closed_request (499) records that the caller disconnected before the gateway had an answer. Nobody receives it; it appears only in request logs.

Next steps

• Reliability: deadlines, retries, idempotency, and platform status together.
• Request IDs and metadata: find the request_id and quote it when you report a problem.
• Rate limits and budgets: read your headroom before you hit a 429.
• Streaming: detect a stream that failed partway.
• Get your API key: fix 401 and 403 at the key.