Structured Output

Structured output constrains AI models to return valid JSON matching a schema you define. Instead of parsing free-form text, you get type-safe data that’s validated automatically.

When to Use Structured Output

• Data extraction — Pull entities from unstructured text
• Form generation — Create structured records from natural language
• API responses — Generate valid payloads for downstream services
• Classification — Get typed category/label responses

Requesting Structured Output

Set output_format on a Responses API request. With type: "json_schema", supply a name and a JSON Schema in json_schema.schema:

curl https://api.modelrelay.ai/api/v1/responses \
  -H "X-ModelRelay-Api-Key: mr_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5-5",
    "input": [
      {"type": "message", "role": "user", "content": [{"type": "text", "text": "Extract: John Doe is 30 years old, email john@example.com"}]}
    ],
    "output_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string", "description": "Full name"},
            "age": {"type": "integer"},
            "email": {"type": ["string", "null"]},
            "tags": {"type": "array", "items": {"type": "string"}}
          },
          "required": ["name", "age", "email", "tags"]
        }
      }
    }
  }'

The JSON arrives as the text of the assistant message in output. Parse it from output[0].content[0].text:

{
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {"type": "text", "text": "{\"name\":\"John Doe\",\"age\":30,\"email\":\"john@example.com\",\"tags\":[]}"}
      ]
    }
  ],
  ...
}

If the selected provider route cannot produce structured output, the request is rejected before it reaches the provider.

OpenAI- and Anthropic-compatible endpoints

POST /v1/chat/completions accepts the OpenAI response_format field, and POST /v1/responses accepts text.format, so existing OpenAI SDK code that requests JSON schema output works unchanged. See SDK compatibility.

Defining Schemas

Any library that emits JSON Schema works: write the schema by hand, or generate it from your language’s types (for example, Zod with z.toJSONSchema() in TypeScript, or Pydantic’s model_json_schema() in Python). Field names and description strings act as instructions to the model, so make them specific.

Model Compatibility Notes

The server normalizes every json_schema output format to the strictest provider requirements so the same schema works on every provider:

• Objects are closed. A missing or true additionalProperties becomes false.
• Every property is required. The server rewrites required to list every declared property, so represent optional fields as nullable (e.g., string | null) rather than omitting them.
• Map/dynamic objects (map[string]any, unbounded keys) are not supported. An additionalProperties subschema returns 400 naming the schema path, for example output_format.json_schema.schema invalid at $.properties.tags.additionalProperties: must be false (maps are not supported).

The json_object output format skips schema normalization and only asks for valid JSON; not every provider supports it.

Streaming Structured Output

Structured output can be streamed with the same Accept headers as text (see Streaming). When output_format requests JSON, the stream runs in structured-patch mode instead of text-delta mode:

• start reports "stream_mode": "structured-patch".
• update events carry patch, a JSON Patch (RFC 6902) array to apply to the previous value, starting from {}, and complete_fields, the top-level fields whose values are complete.
• completion carries the final object in payload, along with complete_fields, usage, and stop_reason.

{"type":"start","stream_mode":"structured-patch","stream_version":"v2","request_id":"...","provider":"anthropic","model":"claude-sonnet-5-5"}
{"type":"update","patch":[{"op":"add","path":"/name","value":"John Doe"},{"op":"add","path":"/age","value":30}],"complete_fields":["name","age"]}
{"type":"completion","payload":{"name":"John Doe","age":30},"complete_fields":["name","age"],"stop_reason":"end_turn","usage":{...}}

ModelRelay emits a structured update once the model has produced a complete top-level JSON value; partial objects are not streamed field by field. If you only need the final object, read payload from the completion event. If the stream ends without a parseable JSON value, it ends with an error event whose code is INVALID_OUTPUT.

Validating the Result

ModelRelay normalizes your schema and asks the provider to constrain its output to it, but it does not re-validate the returned object against your schema. Parse and validate the JSON in your application before using it:

import { z } from "zod";

const Person = z.object({
  name: z.string(),
  age: z.number().int(),
  email: z.string().nullable(),
  tags: z.array(z.string()),
});

const text = response.output[0].content[0].text;
const person = Person.parse(JSON.parse(text));

Structured output can fail in two ways:

To recover, check stop_reason first: a max_tokens stop means the output was truncated, so raise max_output_tokens. Otherwise you can send a follow-up request that appends the model’s previous output and a user message describing the error, asking it to return corrected JSON.

Complex Schemas

Structured output works with nested objects, enums, and arrays:

{
  "type": "json_schema",
  "json_schema": {
    "name": "analysis",
    "schema": {
      "type": "object",
      "properties": {
        "sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]},
        "confidence": {"type": "number"},
        "topics": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": {"type": "string"},
              "relevance": {"type": "number"}
            },
            "required": ["name", "relevance"]
          }
        },
        "summary": {"type": ["string", "null"]}
      },
      "required": ["sentiment", "confidence", "topics", "summary"]
    }
  }
}

Best Practices

1. Use descriptive field names — The model uses field names as hints for what to generate

2. Add descriptions — Field descriptions help the model understand expected values

3. Keep schemas focused — Extract one concept at a time rather than requesting everything in one schema

4. Model optional fields as nullable — Every property becomes required, so use ["string", "null"] instead of leaving a field out of required

5. Leave room in max_output_tokens — Truncated output is not valid JSON

6. Validate before use — Decode errors (bad JSON) and validation errors (schema mismatch) need different handling

Next Steps

• Streaming — Text streaming basics
• Tool Use — Let models call functions
• Error Handling — Handle errors gracefully