Tool Use

Tools let AI models call functions in your application, access external data, and take actions. Instead of just generating text, models can request specific operations and use the results to provide better answers.

Tool Types

The type field of each entry in a request’s tools array selects the kind of tool:

Function tools are the most common—they let you connect models to your APIs, databases, and services.

Function Tools

Define a function tool with a name, description, and JSON schema for parameters:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get current weather for a location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {"type": "string", "description": "City name"},
        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
      },
      "required": ["location"]
    }
  }
}

You can write the parameter schema by hand or generate it from your own types with any JSON Schema library.

Adding Tools to Requests

Pass tools in the tools array of a Responses API request:

curl https://api.modelrelay.ai/api/v1/responses \
  -H "X-ModelRelay-Api-Key: mr_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5-5",
    "input": [
      {"type": "message", "role": "system", "content": [{"type": "text", "text": "You are a helpful assistant with access to weather data."}]},
      {"type": "message", "role": "user", "content": [{"type": "text", "text": "What is the weather in Tokyo?"}]}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string", "description": "City name"}},
            "required": ["location"]
          }
        }
      }
    ]
  }'

The OpenAI-compatible (/v1/chat/completions, /v1/responses) and Anthropic-compatible (/v1/messages) endpoints accept tools in their own wire formats. See SDK compatibility.

Tool Choice

Control when the model uses tools with tool_choice:

{"tool_choice": {"type": "auto"}}
{"tool_choice": {"type": "required"}}
{"tool_choice": {"type": "required", "function": "get_weather"}}
{"tool_choice": {"type": "none"}}

Claude Opus 5.5 and Sonnet 5.5 support automatic tool selection, but reject forced tool choices (required or a named function).

Handling Tool Calls

When the model calls tools, the assistant message in output carries a tool_calls array. Each call has an id, a type, and a function with the tool name and its arguments as a JSON string:

{
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [],
      "tool_calls": [
        {
          "id": "toolu_01A09q90qw90lq917835lq9",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\":\"Tokyo\"}"
          }
        }
      ]
    }
  ],
  "stop_reason": "tool_use",
  ...
}

The model may request several calls in one response. Run each one and return every result before asking the model to continue.

Parsing Arguments

function.arguments is a JSON string. Parse it and validate it against the schema you declared before running the tool; a model can still produce arguments that are malformed or out of range:

const args = JSON.parse(call.function.arguments);
if (typeof args.location !== "string" || args.location === "") {
  // Return an error as the tool result so the model can correct itself
}

Tool Results

After running a tool, send the conversation back with two extra messages: the assistant message that contains the tool calls, and a tool message for each result. Set tool_call_id to the id of the call it answers:

curl https://api.modelrelay.ai/api/v1/responses \
  -H "X-ModelRelay-Api-Key: mr_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5-5",
    "input": [
      {"type": "message", "role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
      {"type": "message", "role": "user", "content": [{"type": "text", "text": "What is the weather in Tokyo?"}]},
      {
        "type": "message",
        "role": "assistant",
        "tool_calls": [
          {
            "id": "toolu_01A09q90qw90lq917835lq9",
            "type": "function",
            "function": {"name": "get_weather", "arguments": "{\"location\":\"Tokyo\"}"}
          }
        ]
      },
      {
        "type": "message",
        "role": "tool",
        "tool_call_id": "toolu_01A09q90qw90lq917835lq9",
        "content": [{"type": "text", "text": "{\"temperature\":22,\"conditions\":\"sunny\"}"}]
      }
    ],
    "tools": [ ... same tools as before ... ]
  }'

Keep sending the same tools on each turn so the model can call them again.

Echo the tool call id verbatim

Replay each tool call with the exact id ModelRelay returned (id in tool_calls on the native Responses API and Chat Completions, call_id on the OpenAI-compatible /v1/responses). Do not mint your own id, truncate it, or normalize it — pair the call and its result by copying the id through unchanged.

The id is opaque and can be long. For Gemini 3 models it carries the model’s thought signature, which Google requires on every function call replayed in the history. A rewritten id drops that signature, and the next request fails:

Function call is missing a thought_signature in functionCall parts. This is
required for tools to work correctly ... function call `default_api:get_weather`,
position 2

The same rule applies to the tool result: its id must match the call it answers.

Complete Tool Loop

A full tool use loop iterates until the model stops requesting tools:

flowchart TD
    A[User Query] --> B[Send Request with Tools]
    B --> C{Model Response}
    C -->|Has Tool Calls| D[Execute Tool Functions]
    D --> E[Append Tool Results to Messages]
    E --> B
    C -->|No Tool Calls| F[Return Final Response]

This TypeScript loop uses plain fetch. runTool is your own function that executes a call and returns its result as a string:

async function respond(input: any[], tools: any[], maxTurns = 5) {
  for (let turn = 0; turn < maxTurns; turn++) {
    const res = await fetch("https://api.modelrelay.ai/api/v1/responses", {
      method: "POST",
      headers: {
        "X-ModelRelay-Api-Key": process.env.MODELRELAY_API_KEY!,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ model: "claude-sonnet-5-5", input, tools }),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`);
    const response = await res.json();

    const message = response.output.find((item: any) => item.role === "assistant");
    const calls = message?.tool_calls ?? [];
    if (calls.length === 0) return response;

    // Echo the assistant's tool calls, then answer each one
    input.push({ type: "message", role: "assistant", content: message.content ?? [], tool_calls: calls });
    for (const call of calls) {
      const result = await runTool(call.function.name, JSON.parse(call.function.arguments));
      input.push({
        type: "message",
        role: "tool",
        tool_call_id: call.id,
        content: [{ type: "text", text: result }],
      });
    }
  }
  throw new Error("Tool loop did not finish");
}

Server-Side Tools

Some tools run on ModelRelay instead of in your application. When the model calls one, ModelRelay executes it and feeds the result back to the model within the same request.

Web Search Tools

For web search and page fetching, use the server-side web_search and web_fetch tools, which provide consistent behavior across all models.

Tool Hooks

Tool hooks are webhooks that let you validate, block, modify, or execute tool calls on your own infrastructure during the tool loop.

X/Twitter Search (Grok Only)

Search posts on X/Twitter with xAI’s Grok models. All x_search fields are optional; dates are RFC 3339 timestamps:

curl https://api.modelrelay.ai/api/v1/responses \
  -H "X-ModelRelay-Api-Key: mr_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "grok-4.7",
    "input": [
      {"type": "message", "role": "user", "content": [{"type": "text", "text": "What are AI companies saying about safety?"}]}
    ],
    "tools": [
      {
        "type": "x_search",
        "x_search": {
          "allowed_handles": ["openai", "anthropic", "google"],
          "from_date": "2026-01-01T00:00:00Z",
          "to_date": "2026-06-30T00:00:00Z"
        }
      }
    ]
  }'

Code Execution

Let the model run code in a provider-hosted sandbox:

curl https://api.modelrelay.ai/api/v1/responses \
  -H "X-ModelRelay-Api-Key: mr_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6.1-sol",
    "input": [
      {"type": "message", "role": "user", "content": [{"type": "text", "text": "Calculate the first 20 Fibonacci numbers"}]}
    ],
    "tools": [{"type": "code_execution", "code_execution": {}}]
  }'

Code execution is available on OpenAI and xAI models that list the code_execution capability. Requesting it on a model without that capability is rejected with 400 and code MODEL_CAPABILITY_UNSUPPORTED. Supported languages and limits are set by the provider.

Streaming Tool Calls

When streaming, tool calls arrive as tool_use_start, tool_use_delta, and tool_use_stop events. The tool_use_stop event carries the finished calls in tool_calls, in the same shape as the non-streaming response. See Streaming with Tool Calls.

Handling Tool Errors

When a tool fails or its arguments are invalid, do not drop the call. Return a tool message for it whose content describes the error, so the model can correct its arguments or choose another approach:

{
  "type": "message",
  "role": "tool",
  "tool_call_id": "toolu_01A09q90qw90lq917835lq9",
  "content": [{"type": "text", "text": "{\"error\":\"location must be a city name\"}"}]
}

Cap the number of retries in your loop so a model that keeps producing invalid arguments cannot run up cost.

Best Practices

1. Write clear tool descriptions — Models use descriptions to decide when to call tools. Be specific about what each tool does and when to use it.

2. Validate arguments — Always validate tool arguments before execution.

3. Handle errors gracefully — Return structured error messages that help the model understand what went wrong and retry appropriately.

4. Limit tool iterations — Set a maximum number of tool loop iterations to prevent runaway costs. Most tasks complete in 2-5 iterations.

5. Consider streaming for long operations — Stream responses when tools might take time, so users see progress.

Next Steps

• Streaming — Real-time response streaming with tool calls
• Built-in Tools — Server-side web, image, and Python tools
• Tool Hooks — Run tool calls on your own infrastructure
• Structured Output — Get typed JSON responses
• Error Handling — Handle tool and request errors