Error Handling

ModelRelay returns structured errors with consistent formats across all endpoints. This guide covers error response formats, error codes, retry strategies, and best practices for production applications.

Error Response Format

Native endpoints (/api/v1/...) return errors in this envelope:

{
  "error": "api_error",
  "layer": "request",
  "cause": "validation",
  "status_code": 404,
  "code": "NOT_FOUND",
  "message": "Customer not found",
  "retryable": false,
  "request_id": "req_abc123"
}

The OpenAI-compatible endpoints (/v1/chat/completions, /v1/responses, /v1/models, /v1/audio/*, /v1/decisions) return OpenAI’s shape instead, {"error": {"message", "type", "param", "code"}}, and /v1/messages returns Anthropic’s error shape. See Chat Completions errors.

HTTP Status Codes

Upstream failures

The API does not send 502 Bad Gateway or 504 Gateway Timeout on the wire.
Our CDN replaces any response with those statuses with its own plain-text page,
which would drop the JSON error body, its code, and its retryable flag. An
upstream provider error is therefore sent as 500 and an upstream timeout as
503, with the original status in the X-ModelRelay-Upstream-Status header:

HTTP/2 500
Content-Type: application/json
X-ModelRelay-Upstream-Status: 502

Error bodies that carry a status field (such as status_code) still report
the original 502 or 504. Decide retries from the body’s retryable flag
when it is present.

Error Codes

General Errors

Provider Errors

Failures attributed to a model provider or to the relay’s own deadline carry these codes:

A request naming a model that is not in the catalog returns 400 naming the model; list valid IDs with GET /v1/models.

Billing admission refusals

Every request refused before dispatch because of a spending limit carries a
code naming the limit. The status tells you whose problem it is.

402 means the caller owes money and can act on it. 503 means ModelRelay
could not authorize the spend for reasons the caller does not control — it is
an operator-side condition, it is marked retryable, and it clears when the
project owner acts.

The same limit can produce either status depending on who is asking, because
the answer to “can you do something about this?” differs:

The last four name a limit belonging to the project owner. An end customer
holds no such ledger and can do nothing about it, so they receive 503 with a
neutral message rather than a demand for payment they cannot satisfy. The
project owner calling for themselves (a project API key, or an owner bearer
token with no customer reference) still receives 402 with the actionable
funding guidance, because for them it genuinely is a payment problem.

If you are building a consumer application, this is the guarantee you need:
a 402 reaching your app always names a limit your end user holds, so it is
safe to surface as an upgrade or top-up prompt. An operator-side funding
problem arrives as a 503 and should be shown as a temporary service error.
Never derive a billing diagnosis from 402 alone — branch on code.

Model Errors

Reading Errors in Your Client

Check the HTTP status, then parse the JSON body and branch on code and retryable. Some responses are not JSON (for example, the plain-text 429 described below), so parse defensively:

class ApiRequestError extends Error {
  constructor(
    readonly status: number,
    readonly code: string | undefined,
    readonly retryable: boolean,
    readonly requestId: string | undefined,
    readonly retryAfterSeconds: number | undefined,
    message: string,
  ) {
    super(message);
  }
}

async function callResponses(body: unknown): Promise<any> {
  const res = await fetch("https://api.modelrelay.ai/api/v1/responses", {
    method: "POST",
    headers: {
      "X-ModelRelay-Api-Key": process.env.MODELRELAY_API_KEY!,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  });
  if (res.ok) return res.json();

  const text = await res.text();
  let err: any = {};
  try {
    err = JSON.parse(text);
  } catch {
    // Not a JSON envelope (for example, a plain-text 429)
  }
  const retryAfter = res.headers.get("Retry-After");
  throw new ApiRequestError(
    res.status,
    err.code,
    err.retryable ?? (res.status === 429),
    err.request_id,
    retryAfter ? Number(retryAfter) : undefined,
    err.message ?? text,
  );
}

Then handle failures by code:

try {
  const response = await callResponses(request);
} catch (error) {
  if (!(error instanceof ApiRequestError)) throw error; // network failure
  console.error(`Request ${error.requestId} failed: ${error.status} ${error.code}`);

  switch (error.code) {
    case "UNAUTHORIZED":
      // Check the API key
      break;
    case "VALIDATION_ERROR":
    case "INVALID_INPUT":
      // Fix the request; retrying will not help
      break;
    case "WALLET_EMPTY":
    case "SUBSCRIPTION_INACTIVE":
    case "TIER_ALLOWANCE_EXHAUSTED":
      // Customer-side billing limit; see Billing admission refusals
      break;
    default:
      if (error.retryable) {
        // Retry with backoff
      }
  }
}

Rate Limiting

When you exceed rate limits, the API returns a 429 status with a Retry-After header indicating how many seconds to wait. On native endpoints the body is plain text; the OpenAI-compatible endpoints wrap it in OpenAI’s error shape (type: "rate_limit_error"):

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: text/plain; charset=utf-8

rate limit exceeded

A provider’s own rate limit also surfaces as 429, with Retry-After when the provider supplied one.

Handling Rate Limits

Wait for the Retry-After interval when it is present, and fall back to exponential backoff when it is not. This builds on callResponses above:

async function callWithRetry(body: unknown, maxRetries = 3): Promise<any> {
  for (let attempt = 0; ; attempt++) {
    try {
      return await callResponses(body);
    } catch (error) {
      if (!(error instanceof ApiRequestError) || !error.retryable || attempt >= maxRetries) {
        throw error;
      }
      // Prefer Retry-After; otherwise back off 1s, 2s, 4s...
      const delayMs = error.retryAfterSeconds !== undefined
        ? error.retryAfterSeconds * 1000
        : Math.pow(2, attempt) * 1000;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
    }
  }
}

Retry Strategies

Retryable vs Non-Retryable Errors

Retry these errors (with exponential backoff):

• 429 Rate Limit
• 500 Internal Server Error (including upstream provider errors)
• 503 Service Unavailable (including upstream timeouts)
• Network timeouts
• Connection errors

Don’t retry these errors:

• 400 Bad Request (fix your request)
• 401 Unauthorized (fix credentials)
• 402 Payment Required (a billing limit; branch on code)
• 403 Forbidden (permissions issue)
• 404 Not Found (resource doesn’t exist)
• 409 Conflict (resolve conflict first)

When the error body carries retryable, decide from it rather than from the status alone.

Exponential Backoff

Use exponential backoff with jitter to avoid thundering herd problems:

function calculateBackoff(attempt: number, baseMs = 1000, maxMs = 30000): number {
  // Exponential: 1s, 2s, 4s, 8s, 16s...
  const exponential = Math.min(baseMs * Math.pow(2, attempt), maxMs);

  // Add jitter: ±25%
  const jitter = exponential * 0.25 * (Math.random() * 2 - 1);

  return Math.round(exponential + jitter);
}

Timeout Handling

Request Timeouts

Set a client-side timeout so a hung connection does not block your application. With fetch, pass AbortSignal.timeout:

const res = await fetch("https://api.modelrelay.ai/api/v1/responses", {
  method: "POST",
  headers: {
    "X-ModelRelay-Api-Key": process.env.MODELRELAY_API_KEY!,
    "Content-Type": "application/json",
  },
  body: JSON.stringify(request),
  signal: AbortSignal.timeout(60_000), // 60s for a non-streaming request
});

Reasoning models at high effort can take much longer than chat models to finish, so size the timeout to the model and effort you request.

If ModelRelay’s own deadline expires while it waits for the provider, the request fails with RELAY_TIMEOUT (see Provider Errors).

Stream Timeouts

For streaming responses, track timeouts separately:

ModelRelay sends keepalives during long gaps, so measure the idle timeout against content events rather than raw bytes. A stream failure after the stream has opened arrives as an error event with the same code, retryable, and request_id fields as the error envelope. See Streaming.

Best Practices

1. Branch on code, Not Status Alone

Several codes share a status. 402 and 503 in particular carry billing refusals that need different handling, so inspect code and retryable before deciding what to do.

2. Log Request IDs

Always log the request_id from error responses. This helps support debug issues:

console.error(`Request ${error.requestId} failed: ${error.message}`);

3. Set Appropriate Timeouts

Configure timeouts based on your use case:

• Chat applications: Lower TTFT timeout, moderate total timeout
• Batch processing: Higher timeouts, more retries
• Real-time systems: Aggressive timeouts, fast failover

4. Implement Circuit Breakers

For high-throughput systems, use circuit breakers to prevent cascade failures:

class CircuitBreaker {
  private failures = 0;
  private lastFailure = 0;
  private readonly threshold = 5;
  private readonly resetMs = 30000;

  async call<T>(fn: () => Promise<T>): Promise<T> {
    if (this.isOpen()) {
      throw new Error("Circuit breaker open");
    }

    try {
      const result = await fn();
      this.failures = 0;
      return result;
    } catch (error) {
      this.failures++;
      this.lastFailure = Date.now();
      throw error;
    }
  }

  private isOpen(): boolean {
    if (this.failures < this.threshold) return false;
    if (Date.now() - this.lastFailure > this.resetMs) {
      this.failures = 0;
      return false;
    }
    return true;
  }
}

5. Handle Quota Exhaustion

When customers exceed their quota, provide clear feedback:

// A 402 always names a limit the calling customer holds (see Billing admission refusals)
if (error instanceof ApiRequestError && error.status === 402) {
  if (error.code === "WALLET_EMPTY") {
    showTopUpPrompt(customer);
  } else {
    // SUBSCRIPTION_INACTIVE, TIER_ALLOWANCE_EXHAUSTED
    showUpgradeModal(customer);
  }
}

Next Steps

• Streaming - Handle streaming responses
• Webhooks - React to events asynchronously
• Customer Billing - Manage quotas and subscriptions