Handle tool calls

Read the tool calls the model returns, run each function, send every result back by tool_call_id, and repeat until finish_reason is stop.

When the model decides to call one of the tools you defined, the turn ends with finish_reason: "tool_calls" and the assistant message carries tool_calls. You run each function, then send a new request with the results; the model reads them and continues. Define tools covers the request that offers the tools.

Read the tool calls

On POST /api/v1/chat with stream: false, the assistant reply is the top-level message. A turn that calls a tool looks like this, trimmed to the fields that matter here:

{
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" }
      }
    ]
  },
  "finish_reason": "tool_calls"
}

Each entry in tool_calls has three fields:

arguments is a JSON-encoded string, not an object. Parse it before you dispatch:

const args = JSON.parse(call.function.arguments) // { city: "Paris" }

Send the results back

1. 1

   Append the assistant message whole

   Push the message the gateway returned onto your conversation, tool_calls included. The model needs its own call in the history next to the result that answers it.

2. 2

   Append one tool message per call

   For every entry in tool_calls, run the function and append a message with role: "tool", the call's id as tool_call_id, and the result as a string in content.

3. 3

   Call chat again with the same tools

   Send the whole conversation back with the same tools. The model reads the results and either answers or calls another tool.

This request carries a finished tool exchange back to the model:

curl https://api.cruxy.ai/api/v1/chat \
  -H "Authorization: Bearer $CRUXY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "kavi",
    "stream": false,
    "messages": [
      { "role": "user", "content": "What is the weather in Paris?" },
      {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" }
          }
        ]
      },
      {
        "role": "tool",
        "tool_call_id": "call_abc123",
        "content": "{\"city\":\"Paris\",\"tempC\":18,\"sky\":\"clear\"}"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a city",
          "parameters": {
            "type": "object",
            "properties": { "city": { "type": "string" } },
            "required": ["city"]
          }
        }
      }
    ]
  }'

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get the current weather for a city',
      parameters: {
        type: 'object',
        properties: { city: { type: 'string' } },
        required: ['city'],
      },
    },
  },
]
 
// The assistant turn the gateway returned, kept whole.
const assistantTurn = {
  role: 'assistant',
  content: '',
  tool_calls: [
    {
      id: 'call_abc123',
      type: 'function',
      function: { name: 'get_weather', arguments: '{"city":"Paris"}' },
    },
  ],
}
 
const messages = [
  { role: 'user', content: 'What is the weather in Paris?' },
  assistantTurn,
  // Your result, tied back to the call by its id.
  {
    role: 'tool',
    tool_call_id: 'call_abc123',
    content: JSON.stringify({ city: 'Paris', tempC: 18, sky: 'clear' }),
  },
]
 
const res = await fetch('https://api.cruxy.ai/api/v1/chat', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.CRUXY_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ model: 'kavi', stream: false, messages, tools }),
})
 
const completion = await res.json()
console.log(completion.finish_reason) // "stop" once the model has its answer
console.log(completion.message.content)

Loop on finish_reason

finish_reason tells you what to do with each turn:

Keep calling while finish_reason is tool_calls, and stop when it returns stop. Put a ceiling on the number of rounds too, so a model that keeps calling tools cannot run your loop forever.

Several calls in one turn

tool_calls is an array, so one turn can ask for several calls. Run all of them and append one tool message for each before you call chat again. The tool_call_id on each result is what pairs it with its call.

If a function fails, still answer its call: put a short description of the failure in that tool message's content, so every call in the turn has a result.

Send the tools on every request

Keep sending the same tools for as long as the conversation holds tool exchanges. When the history replays a tool call to a function that the request's tools does not declare, the gateway sends that exchange to the model as prose instead of as a tool call, and reports it in cruxy.undeclared_tool_replay:

The field is absent when nothing was rewritten, and nothing you store or send is changed: only what the model reads that turn.

Tool calls on a stream

With stream: true, the calls arrive in pieces. Deltas carry tool_calls fragments instead of content; each fragment has an index and a piece of function.arguments, and may carry the call's id, type, and function.name. Assemble the fragments per index, concatenating the arguments pieces. The finish frame with finish_reason: "tool_calls" means the calls are complete; answer them as above. Tool-call deltas shows the frames. On the native wire, POST /api/v2/chat, the fragments ride delta events and the terminal finish event carries finish_reason.

Next steps

• Tutorial: build a tool-using agent: this loop as one working program.
• Define tools: the tool shape, tool_choice, and schema bounds.
• Streaming: the full frame order.