SDK Compatibility

ModelRelay provides drop-in compatibility endpoints for the OpenAI and Anthropic APIs. If you have existing code using these providers’ SDKs, you can point them at ModelRelay with a base URL change to get unified billing, observability, and access to any model.

Why Use Compatibility Endpoints?

• Minimal code changes - Point existing code at ModelRelay with a base URL change
• Unified billing - Track usage across all providers through ModelRelay
• Provider flexibility - Use any model through the same API interface
• Gradual migration - Move to ModelRelay without rewriting your application

OpenAI Responses API

Endpoint: POST /v1/responses

Point the OpenAI SDK at ModelRelay by changing the base URL and using your ModelRelay API key.

Configuration

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MODELRELAY_API_KEY,
  baseURL: "https://api.modelrelay.ai/v1",
});

const response = await client.responses.create({
  model: "claude-sonnet-5-5", // Use any ModelRelay model
  input: "What is the capital of France?",
});

console.log(response.output_text);

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MODELRELAY_API_KEY"],
    base_url="https://api.modelrelay.ai/v1",
)

response = client.responses.create(
    model="claude-sonnet-5-5",  # Use any ModelRelay model
    input="What is the capital of France?",
)

print(response.output_text)

curl -X POST https://api.modelrelay.ai/v1/responses \
  -H "Authorization: Bearer $MODELRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5-5",
    "input": "What is the capital of France?"
  }'

Namespace tools and validation

/v1/responses accepts function tools grouped under a namespace, using the OpenAI namespace schema:

{"type":"namespace","name":"crm","description":"Customer tools","tools":[
  {"type":"function","name":"lookup","parameters":{"type":"object","properties":{},"additionalProperties":false}}
]}

Each member becomes a function tool for the selected provider. A returned call uses the member’s original name and a separate namespace, in both ordinary responses and streaming response.output_item.* and response.completed events:

{"type":"function_call","call_id":"call_1","namespace":"crm","name":"lookup","arguments":"{}"}

Replay those fields unchanged in later input. Return the result as function_call_output with the same call_id. Top-level functions keep their existing names and omit namespace. Namespace descriptions, function descriptions, parameter schemas, and strictness are preserved. Duplicate or conflicting function names are rejected. defer_loading: true is rejected with INVALID_TOOL_DEFINITION and an indexed .defer_loading param because Responses tool search is not supported. Namespace requests are also rejected with INVALID_TOOL_DEFINITION when the project has enabled tool hooks: the current hook contract carries one flat tool name and cannot safely express namespace identity. Top-level function hooks keep their existing behavior.

Across Responses, Chat Completions, and Messages, an unsupported tool type returns HTTP 400 with error.code: "UNSUPPORTED_TOOL_TYPE"; invalid definitions use "INVALID_TOOL_DEFINITION". error.param identifies the tool index and field, including nested members such as tools[3].tools[1].type. The message names the declared type when one was provided. Array-level shape errors use the enclosing array’s path (tools, or tools[i].tools for namespace members) and carry no declared type. Messages retains its existing error.error_code and adds code and param for these errors; code and error_code always contain the same typed tool error code. For example:

{"error":{"type":"invalid_request_error","code":"UNSUPPORTED_TOOL_TYPE","param":"tools[3].type","message":"unsupported tool type; must be a function tool (tool type \"web_search\")"}}

Namespace members must be function tools. Hosted tools such as web_search, file_search, computer_use_preview, and code_interpreter are rejected explicitly; they are never dropped. Chat Completions accepts function tools only. Messages accepts client tools with an omitted type or type: "custom" and rejects hosted tool types. Codex’s captured default request includes namespaces and web_search: its namespaces are supported, and web search still needs to be disabled in Codex for the request to succeed.

Streaming

Enable streaming by setting stream: true:

const stream = await client.responses.create({
  model: "claude-sonnet-5-5",
  input: "Write a haiku about programming.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}

stream = client.responses.create(
    model="claude-sonnet-5-5",
    input="Write a haiku about programming.",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

Authentication

Use Authorization: Bearer <mr_sk_...> with your ModelRelay secret key. The compatibility layer maps this to X-ModelRelay-Api-Key internally.

Field Mapping

input_image.image_url must be a data:image/...;base64,... URL. HTTP(S) image URLs and OpenAI file_id values are rejected rather than fetched or resolved. detail is accepted and ignored. Image input still requires a model that advertises vision and a rate card that prices image tokens.

Some routes accept only certain image encodings, and that is separate from whether they accept images at all. Where the upstream publishes a constraint, ModelRelay declares it and converts your image into a format the route takes — WebP and GIF are re-encoded as PNG, losslessly — rather than forwarding bytes the host will reject. Cerebras (qwen-3.8-27b) publishes PNG and JPEG only, so a WebP screenshot works there and arrives as PNG. Two consequences worth knowing: PNG is larger than WebP, which counts against a host’s request size limit, and an image in a format with no decoder (or a corrupt one) is refused before dispatch with an error naming the format you sent, the ones the route accepts, and why the conversion failed. Animated WebP is not decodable and is refused; an animated GIF converts to its first frame.

function_call_output.output accepts either a string or the same content-part array a message carries, so a tool that returns an image hands it back directly:

{
  "type": "function_call_output",
  "call_id": "call_1",
  "output": [
    { "type": "input_text", "text": "Read image file [image/png]" },
    { "type": "input_image", "image_url": "data:image/png;base64,..." }
  ]
}

An image in a tool result counts as image input for routing and pricing exactly as one in a user message does, so it needs the same vision route and image rate card. Where the upstream transport carries images inside its own tool result — Anthropic, Gemini, the OpenAI Responses API — the image stays there. Where it does not, notably Chat Completions hosts such as Cerebras and Together, ModelRelay moves the image to a user turn immediately after the tool results and labels it with the call_id it came from. What you send back on the next turn is unaffected: the conversation ModelRelay returns is the shape you sent.

OpenAI Chat Completions API

Endpoint: POST /v1/chat/completions

Existing client.chat.completions.create(...) calls can use ModelRelay for streaming and non-streaming completions, function calling, and JSON schema output. Set the base URL to https://api.modelrelay.ai/v1 and use a ModelRelay key. Model IDs must exist in ModelRelay or match an explicitly supported OpenRouter alias. User message content may include image_url data URLs.

Use stream: true for SSE and stream_options: { "include_usage": true } for an OpenAI-style terminal usage chunk with empty choices. Tool calls and results use standard tool_calls and tool_call_id fields, so SDK-assembled assistant messages can be replayed for function-calling conversations. Structured output uses response_format.type: "json_schema"; json_object mode is also supported by OpenAI, OpenAI Chat Completions transports, Google, and xAI, and rejected by Anthropic.

OpenRouter extensions include reasoning.effort, reasoning.exclude, provider selection through only, order, ignore, allow_fallbacks, zdr, and data_collection, and provider routing through sort, max_price, preferred_min_throughput, preferred_max_latency, and the :nitro and :floor model suffixes. The serving provider comes back as provider on the completion and its chunks. This is partial OpenRouter compatibility: plugins, parameter routing constraints, opaque reasoning round trips, and unconditional OpenRouter-style streaming usage are unsupported. Assistant reasoning history (reasoning_content, reasoning, reasoning_details) is accepted and ignored rather than forwarded. Audio/refusal metadata, HTTP(S) image URLs, unrecognized generation controls, and ambiguous message shapes return errors. temperature is rejected for reasoning models except those served by Fireworks, Together, NEAR AI, Cerebras, and Inco, and cannot be combined with reasoning_effort. See the Chat Completions API reference for supported fields and exact behavior.

Model discovery follows OpenRouter too: GET /v1/models lists one entry per model, and GET /v1/models/{author}/{slug}/endpoints (or /v1/models/{model_id}/endpoints) lists each provider route that serves it with that route’s own price, limits and recent latency and throughput. See List a Model’s Endpoints.

Codex

Codex uses the OpenAI Responses protocol. Configure a custom provider in your Codex configuration and set MODELRELAY_API_KEY in the environment:

model_provider = "modelrelay"
model = "glm-5.3-flash"
model_reasoning_effort = "low"
web_search = "disabled"

[features]
apps = false
multi_agent = false
tool_suggest = false

[model_providers.modelrelay]
name = "ModelRelay"
base_url = "https://api.modelrelay.ai/v1"
env_key = "MODELRELAY_API_KEY"
wire_api = "responses"

The repository’s scripts/smoke-codex.sh runs a small Codex file-edit task in an isolated temporary directory through ModelRelay, defaulting to glm-5.3-flash with apps, multiple agents, and tool suggestions disabled. It requires an installed Codex CLI and a funded ModelRelay API key with access to the selected model. This config limits the smoke task to standard function tools: newer Codex protocol features such as additional_tools and tool namespaces are not supported. This exercises the Responses endpoint; the Chat Completions endpoint has separate SDK streaming and tool-cycle tests.

Anthropic Messages API

Endpoint: POST /v1/messages

Point the Anthropic SDK at ModelRelay by setting the base URL to https://api.modelrelay.ai (the SDK appends /v1/messages itself) and using your ModelRelay API key.

Configuration

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.MODELRELAY_API_KEY,
  baseURL: "https://api.modelrelay.ai",
});

const message = await client.messages.create({
  model: "claude-sonnet-5-5",
  max_tokens: 1024,
  messages: [
    { role: "user", content: "What is the capital of France?" }
  ],
});

console.log(message.content[0].text);

import os
import anthropic

client = anthropic.Anthropic(
    api_key=os.environ["MODELRELAY_API_KEY"],
    base_url="https://api.modelrelay.ai",
)

message = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
)

print(message.content[0].text)

curl -X POST https://api.modelrelay.ai/v1/messages \
  -H "x-api-key: $MODELRELAY_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'

Streaming

Enable streaming by setting stream: true:

const stream = await client.messages.stream({
  model: "claude-sonnet-5-5",
  max_tokens: 1024,
  messages: [
    { role: "user", content: "Write a haiku about programming." }
  ],
});

for await (const event of stream) {
  if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
    process.stdout.write(event.delta.text);
  }
}

with client.messages.stream(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Write a haiku about programming."}
    ],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Authentication

The Anthropic compatibility endpoint supports two authentication styles:

• x-api-key: <mr_sk_...> - Anthropic SDK default
• Authorization: Bearer <mr_sk_...> - Alternative style

Both are mapped to X-ModelRelay-Api-Key internally.

Every executed Messages request returns X-ModelRelay-Execution-Id. The UUID
identifies the immutable content-free pricing and settlement evidence available
from GET /api/v1/responses/executions/{execution_id}/evidence.

Field Mapping

Mid-conversation system messages

The Messages compatibility endpoint supports system role items inside the ordered messages array, with no beta header. Their position is preserved for both streaming and non-streaming requests. The top-level system field continues to provide the leading system prompt. For effort and tool changes in these messages, see the Messages API reference.

System-level priority is preserved when the selected model and message placement support native mid-conversation system messages. Claude Fable 5.1, Opus 5.5, Opus 5, Sonnet 5.5, and Haiku 5.5 support them, including dated IDs. Consecutive system messages count as one: they must follow a user turn and either end the message array or be followed by an assistant turn. For other request shapes, including a user / system / user sequence, and on other models, ModelRelay sends the system item’s text at the same position as a user-role message. That keeps the instruction after earlier conversation content, but gives it user-level rather than system-level priority; the provider may also combine consecutive user-role messages.

Chat Completions, Responses, and CLI requests keep their existing system-message behavior.

Content Block Types

The Anthropic adapter supports all standard content block types:

Model Flexibility

The compatibility endpoints allow you to use any model available in ModelRelay, not just models from the corresponding provider:

// Use OpenAI SDK to call an Anthropic model
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MODELRELAY_API_KEY,
  baseURL: "https://api.modelrelay.ai/v1",
});

await client.responses.create({
  model: "claude-opus-5-5", // Anthropic model via OpenAI API format
  input: "Hello!",
});

// Use Anthropic SDK to call an OpenAI model
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.MODELRELAY_API_KEY,
  baseURL: "https://api.modelrelay.ai",
});

await client.messages.create({
  model: "gpt-6.1-sol", // OpenAI model via Anthropic API format
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello!" }],
});

Error Handling

Errors are returned in the format expected by each API:

OpenAI Format

{
  "error": {
    "message": "Invalid API key",
    "type": "authentication_error",
    "code": "invalid_api_key"
  }
}

Anthropic Format

{
  "type": "error",
  "error": {
    "type": "authentication_error",
    "message": "Invalid API key"
  }
}

Limitations

The compatibility endpoints provide high-fidelity API translation, but some provider-specific features may not be available:

• Provider-specific extensions - Check the endpoint reference; unsupported Chat Completions generation and routing fields are rejected
• Batch endpoints - Submit individual Responses requests instead
• Chat Completions scope - Function tools and structured output are supported; multiple choices, audio, and legacy function calling are not; replayed reasoning history is accepted but ignored
• Image file_id and remote URLs - Compatibility vision input is data URLs only; OpenAI file IDs and HTTP(S) image URLs are rejected

For full access to ModelRelay features like state handles and built-in tools,
use the native API.

Next Steps

• First Request - Make your first request
• Responses API - Full API reference
• Streaming - Real-time response streaming
• Tool Use - Function calling with tools

Request-level ZDR routing

Compatibility requests use "provider": {"zdr": true}; native Responses use
top-level "zdr": true. SDK builders expose
Go .ZDR(true), TypeScript/Swift .zdr(true), and Python ResponsesRequest.zdr.
With upstream OpenAI or Anthropic Python SDKs, use
extra_body={"provider": {"zdr": True}} for the extension.

ZDR filters every eligible inference route, including pins and fallbacks, using
the route policy published by the catalog. False, null, or omission cannot
remove account/key require_zdr. Request-only ZDR leaves provider caching,
explicit gateway caching, state and tools under their normal policies; account
and key requirements still forbid gateway content storage. Provider free-text
diagnostics are redacted on effectively ZDR text calls; metadata remains.
See Zero Data Retention.