Structured Output

Structured output constrains AI models to return valid JSON matching a schema you define. Instead of parsing free-form text, you get type-safe data that’s validated automatically.

When to Use Structured Output

Data extraction — Pull entities from unstructured text
Form generation — Create structured records from natural language
API responses — Generate valid payloads for downstream services
Classification — Get typed category/label responses

Defining Schemas

Each SDK generates JSON Schema from native type definitions:

import { z } from "zod";

const PersonSchema = z.object({
  name: z.string().describe("Full name"),
  age: z.number().int().positive(),
  email: z.string().email().optional(),
  tags: z.array(z.string()),
});

type Person = z.infer<typeof PersonSchema>;

TypeScript uses Zod for schema definition. The SDK converts Zod schemas to JSON Schema automatically.

type Person struct {
    Name  string   `json:"name" description:"Full name"`
    Age   int      `json:"age" minimum:"0"`
    Email string   `json:"email,omitempty" format:"email"`
    Tags  []string `json:"tags"`
}

Go uses struct tags for schema generation. Supported tags:

Tag	Description
`description:"..."`	Field description
`enum:"a,b,c"`	Allowed values
`default:"..."`	Default value
`minimum:"N"`	Min for numbers
`maximum:"N"`	Max for numbers
`minLength:"N"`	Min string length
`maxLength:"N"`	Max string length
`pattern:"regex"`	Regex pattern
`format:"email"`	Format hint

use schemars::JsonSchema;
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, JsonSchema)]
struct Person {
    /// Full name
    name: String,
    age: u32,
    email: Option<String>,
    tags: Vec<String>,
}

Rust uses schemars for schema generation. Doc comments become descriptions.

Model Compatibility Notes

Structured output schemas are validated against the strictest provider requirements to keep workflows portable:

All object schemas must set additionalProperties: false.
All properties must appear in required. Represent optional fields as nullable (e.g., string | null) rather than omitting them.
Map/dynamic objects (map[string]any, unbounded keys) are not supported in structured outputs.

These constraints align with the strictest provider requirements and prevent runtime incompatibilities across providers.

Basic Structured Output

Request structured output and get a validated, typed response:

import { ModelRelay } from "@modelrelay/sdk";
import { z } from "zod";

const mr = ModelRelay.fromSecretKey(process.env.MODELRELAY_API_KEY!);

// Simple one-call API (recommended)
const person = await mr.responses.object<z.infer<typeof PersonSchema>>({
  model: "claude-sonnet-4-5",
  schema: PersonSchema,
  prompt: "Extract: John Doe is 30 years old, email john@example.com",
});

console.log(person.name);  // "John Doe"
console.log(person.age);   // 30

For metadata (attempts, request ID), use objectWithMetadata:

const result = await mr.responses.objectWithMetadata<z.infer<typeof PersonSchema>>({
  model: "claude-sonnet-4-5",
  schema: PersonSchema,
  prompt: "Extract: John Doe is 30 years old",
  maxRetries: 2,
});

console.log(result.value.name);  // "John Doe"
console.log(result.attempts);    // 1 (no retries needed)
console.log(result.requestId);   // Server request ID

import sdk "github.com/modelrelay/sdk-go"

req, opts, _ := client.Responses.New().
    Model(sdk.NewModelID("claude-sonnet-4-5")).
    User("Extract: John Doe is 30 years old, email john@example.com").
    Build()

result, err := sdk.Structured[Person](ctx, client.Responses, req, opts,
    sdk.StructuredOptions{})
if err != nil {
    log.Fatal(err)
}

fmt.Println(result.Value.Name)  // "John Doe"
fmt.Println(result.Value.Age)   // 30
fmt.Println(result.Attempts)    // 1

use modelrelay::{Client, ResponseBuilder};

let client = Client::from_api_key(std::env::var("MODELRELAY_API_KEY")?)?
    .build()?;

let result = ResponseBuilder::new()
    .model("claude-sonnet-4-5")
    .user("Extract: John Doe is 30 years old, email john@example.com")
    .structured::<Person>()
    .send(&client.responses())
    .await?;

println!("{}", result.value.name);  // "John Doe"
println!("{}", result.value.age);   // 30
println!("{}", result.attempts);    // 1

The result includes:

value — The parsed and validated data
attempts — How many tries (1 = first try succeeded)
requestId — Server request ID for debugging

Streaming Structured Output

Stream structured responses as they’re generated. The model outputs JSON incrementally, and you receive updates via JSON Patch (RFC 6902):

sequenceDiagram
    participant App
    participant ModelRelay
    participant Model

    App->>ModelRelay: streamStructured(schema, prompt)
    ModelRelay->>Model: Generate JSON

    Model-->>ModelRelay: {"name": "John...
    ModelRelay-->>App: update: {name: "John"}, complete: [name]

    Model-->>ModelRelay: ", "age": 30}
    ModelRelay-->>App: update: {name: "John", age: 30}, complete: [name, age]

    Model-->>ModelRelay: [done]
    ModelRelay-->>App: completion: {name: "John", age: 30}

const stream = await mr.responses.streamStructured(
  PersonSchema,
  mr.responses
    .new()
    .model("claude-sonnet-4-5")
    .user("Generate a person profile for a software developer")
    .build()
);

for await (const event of stream) {
  if (event.type === "update") {
    console.log("Partial:", event.payload);
    console.log("Complete fields:", [...event.completeFields]);
  } else if (event.type === "completion") {
    console.log("Final:", event.payload);
  }
}

req, opts, _ := client.Responses.New().
    Model(sdk.NewModelID("claude-sonnet-4-5")).
    User("Generate a person profile for a software developer").
    Build()

stream, err := sdk.StreamStructured[Person](ctx, client.Responses, req, opts, "person")
if err != nil {
    log.Fatal(err)
}
defer stream.Close()

for {
    event, ok, err := stream.Next()
    if err != nil {
        log.Fatal(err)
    }
    if !ok {
        break
    }

    switch event.Type {
    case sdk.StructuredRecordTypeUpdate:
        fmt.Printf("Partial: %+v\n", event.Payload)
        fmt.Printf("Complete: %v\n", event.CompleteFields)
    case sdk.StructuredRecordTypeCompletion:
        fmt.Printf("Final: %+v\n", event.Payload)
    }
}

use futures_util::StreamExt;
use modelrelay::{ResponseBuilder, StructuredRecordKind};

let mut stream = ResponseBuilder::new()
    .model("claude-sonnet-4-5")
    .user("Generate a person profile for a software developer")
    .structured::<Person>()
    .stream(&client.responses())
    .await?;

while let Some(event) = stream.next().await {
    let event = event?;
    match event.kind {
        StructuredRecordKind::Update => {
            println!("Partial: {:?}", event.payload);
            println!("Complete: {:?}", event.complete_fields);
        }
        StructuredRecordKind::Completion => {
            println!("Final: {:?}", event.payload);
        }
    }
}

Collecting the Final Result

If you only need the final value:

const stream = await mr.responses.streamStructured(PersonSchema, req);
const result = await stream.collect();
console.log(result);  // Person object

stream, _ := sdk.StreamStructured[Person](ctx, client.Responses, req, opts, "person")
result, err := stream.Collect(ctx)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("%+v\n", result)  // Person struct

let stream = ResponseBuilder::new()
    .model("claude-sonnet-4-5")
    .user("Generate a person profile")
    .structured::<Person>()
    .stream(&client.responses())
    .await?;

let result = stream.collect().await?;
println!("{:?}", result);  // Person struct

Complete Fields Tracking

During streaming, completeFields tells you which fields have received their closing delimiter. Use this for progressive UI updates:

const stream = await mr.responses.streamStructured(PersonSchema, req);

for await (const event of stream) {
  if (event.type === "update") {
    // Show completed fields immediately
    if (event.completeFields.has("name")) {
      renderName(event.payload.name);
    }
    if (event.completeFields.has("age")) {
      renderAge(event.payload.age);
    }
    // Show loading spinner for incomplete fields
    if (!event.completeFields.has("tags")) {
      showSpinner("tags");
    }
  }
}

This enables field-by-field display instead of waiting for the entire response.

Validation and Retry

By default, structured output validates the response against your schema. If validation fails, you can retry automatically:

const result = await mr.responses.structured(
  PersonSchema,
  req,
  { maxRetries: 2 }  // Retry up to 2 times on validation failure
);

console.log(`Succeeded after ${result.attempts} attempt(s)`);

result, err := sdk.Structured[Person](ctx, client.Responses, req, opts,
    sdk.StructuredOptions{MaxRetries: 2})
if err != nil {
    log.Fatal(err)
}

fmt.Printf("Succeeded after %d attempt(s)\n", result.Attempts)

let result = ResponseBuilder::new()
    .model("claude-sonnet-4-5")
    .user("Extract person from text...")
    .structured::<Person>()
    .max_retries(2)
    .send(&client.responses())
    .await?;

println!("Succeeded after {} attempt(s)", result.attempts);

The retry mechanism:

Parses the model’s JSON response
Validates against the schema
On failure, appends the error to the conversation and asks the model to fix it
Repeats until validation passes or retries exhausted

Note: Streaming structured output does not support automatic retries. Validation errors surface only on stream completion.

Custom Retry Handlers

Override the default retry behavior:

import { RetryHandler } from "@modelrelay/sdk";

const customHandler: RetryHandler = {
  onValidationError(attempt, rawJson, error, originalInput) {
    // Log the failure
    console.log(`Attempt ${attempt} failed:`, error);

    // Stop after 3 attempts
    if (attempt >= 3) return null;

    // Build error message
    const errorMsg =
      error.kind === "decode"
        ? error.message
        : error.issues.map((i) => `${i.path}: ${i.message}`).join("; ");

    // Return messages to append to conversation
    return [
      {
        type: "message",
        role: "user",
        content: [
          {
            type: "text",
            text: `Response invalid: ${errorMsg}. Please provide valid JSON.`,
          },
        ],
      },
    ];
  },
};

const result = await mr.responses.structured(PersonSchema, req, {
  maxRetries: 5,
  retryHandler: customHandler,
});

type MyRetryHandler struct{}

func (h MyRetryHandler) OnValidationError(
    attempt int,
    rawJSON string,
    err sdk.StructuredErrorDetail,
    originalInput []llm.InputItem,
) []llm.InputItem {
    // Log the failure
    log.Printf("Attempt %d failed: %s", attempt, err.Message)

    // Stop after 3 attempts
    if attempt >= 3 {
        return nil
    }

    // Return messages to append
    return []llm.InputItem{
        llm.NewUserText(fmt.Sprintf(
            "Response invalid: %s. Please provide valid JSON.",
            err.Message,
        )),
    }
}

result, err := sdk.Structured[Person](ctx, client.Responses, req, opts,
    sdk.StructuredOptions{
        MaxRetries:   5,
        RetryHandler: MyRetryHandler{},
    })

use modelrelay::{RetryHandler, StructuredErrorKind, InputItem};

struct MyRetryHandler;

impl RetryHandler for MyRetryHandler {
    fn on_validation_error(
        &self,
        attempt: u32,
        raw_json: &str,
        error: &StructuredErrorKind,
        messages: &[InputItem],
    ) -> Option<Vec<InputItem>> {
        // Log the failure
        println!("Attempt {} failed: {:?}", attempt, error);

        // Stop after 3 attempts
        if attempt >= 3 {
            return None;
        }

        // Return messages to append
        Some(vec![InputItem::user_text(format!(
            "Response invalid: {:?}. Please provide valid JSON.",
            error
        ))])
    }
}

let result = ResponseBuilder::new()
    .model("claude-sonnet-4-5")
    .user("Extract person...")
    .structured::<Person>()
    .max_retries(5)
    .retry_handler(MyRetryHandler)
    .send(&client.responses())
    .await?;

Error Handling

Structured output can fail in two ways:

Error Type	Cause	Retryable
Decode	Invalid JSON (syntax error)	Yes
Validation	Valid JSON but doesn’t match schema	Yes

import {
  StructuredDecodeError,
  StructuredExhaustedError,
} from "@modelrelay/sdk";

try {
  const result = await mr.responses.structured(PersonSchema, req, {
    maxRetries: 2,
  });
  console.log(result.value);
} catch (error) {
  if (error instanceof StructuredExhaustedError) {
    // All retries failed
    console.error("Failed after all attempts:");
    for (const attempt of error.allAttempts) {
      console.error(`  Attempt ${attempt.attempt}:`, attempt.error);
    }
    console.error("Last raw JSON:", error.lastRawJSON);
  } else if (error instanceof StructuredDecodeError) {
    // JSON parse failed (shouldn't happen with retries enabled)
    console.error("JSON decode error:", error.message);
  } else {
    throw error;
  }
}

import "errors"

result, err := sdk.Structured[Person](ctx, client.Responses, req, opts,
    sdk.StructuredOptions{MaxRetries: 2})
if err != nil {
    var exhaustedErr sdk.StructuredExhaustedError
    var decodeErr sdk.StructuredDecodeError

    if errors.As(err, &exhaustedErr) {
        // All retries failed
        log.Println("Failed after all attempts:")
        for _, attempt := range exhaustedErr.AllAttempts {
            log.Printf("  Attempt %d: %s", attempt.Attempt, attempt.Error.Message)
        }
        log.Println("Last raw JSON:", exhaustedErr.LastRawJSON)
    } else if errors.As(err, &decodeErr) {
        // JSON parse failed
        log.Printf("JSON decode error: %s", decodeErr.Message)
    } else {
        log.Fatal(err)
    }
    return
}

fmt.Printf("%+v\n", result.Value)

use modelrelay::{StructuredError, StructuredExhaustedError};

match result {
    Ok(result) => println!("{:?}", result.value),
    Err(StructuredError::Exhausted(e)) => {
        // All retries failed
        eprintln!("Failed after all attempts:");
        for attempt in &e.all_attempts {
            eprintln!("  Attempt {}: {:?}", attempt.attempt, attempt.error);
        }
        eprintln!("Last raw JSON: {}", e.last_raw_json);
    }
    Err(StructuredError::Decode(e)) => {
        // JSON parse failed
        eprintln!("JSON decode error: {}", e.message);
    }
    Err(e) => return Err(e.into()),
}

Complex Schemas

Structured output works with nested types, enums, and arrays:

const AnalysisSchema = z.object({
  sentiment: z.enum(["positive", "negative", "neutral"]),
  confidence: z.number().min(0).max(1),
  topics: z.array(
    z.object({
      name: z.string(),
      relevance: z.number(),
    })
  ),
  summary: z.string().optional(),
});

const result = await mr.responses.structured(
  AnalysisSchema,
  mr.responses
    .new()
    .model("claude-sonnet-4-5")
    .user("Analyze: The product launch was a huge success!")
    .build()
);

console.log(result.value.sentiment);  // "positive"
console.log(result.value.topics);     // [{name: "product launch", relevance: 0.9}]

type Topic struct {
    Name      string  `json:"name"`
    Relevance float64 `json:"relevance"`
}

type Analysis struct {
    Sentiment  string  `json:"sentiment" enum:"positive,negative,neutral"`
    Confidence float64 `json:"confidence" minimum:"0" maximum:"1"`
    Topics     []Topic `json:"topics"`
    Summary    string  `json:"summary,omitempty"`
}

result, _ := sdk.Structured[Analysis](ctx, client.Responses, req, opts,
    sdk.StructuredOptions{})

fmt.Println(result.Value.Sentiment)  // "positive"
fmt.Println(result.Value.Topics)     // [{Name: "product launch", Relevance: 0.9}]

#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "lowercase")]
enum Sentiment {
    Positive,
    Negative,
    Neutral,
}

#[derive(Debug, Serialize, Deserialize, JsonSchema)]
struct Topic {
    name: String,
    relevance: f64,
}

#[derive(Debug, Serialize, Deserialize, JsonSchema)]
struct Analysis {
    sentiment: Sentiment,
    confidence: f64,
    topics: Vec<Topic>,
    summary: Option<String>,
}

let result = ResponseBuilder::new()
    .model("claude-sonnet-4-5")
    .user("Analyze: The product launch was a huge success!")
    .structured::<Analysis>()
    .send(&client.responses())
    .await?;

println!("{:?}", result.value.sentiment);  // Positive
println!("{:?}", result.value.topics);     // [Topic { name: "product launch", relevance: 0.9 }]

Best Practices

Use descriptive field names — The model uses field names as hints for what to generate
Add descriptions — Field descriptions help the model understand expected values
Keep schemas focused — Extract one concept at a time rather than requesting everything in one schema
Enable retries for reliability — Set maxRetries: 1-2 for production to handle occasional formatting issues
Stream for large responses — Use streaming structured output when generating large objects to reduce time-to-first-byte
Use completeFields for UX — Show fields as they complete instead of waiting for the full response
Handle both error types — Decode errors (bad JSON) and validation errors (schema mismatch) require different handling

Streaming vs Non-Streaming

Feature	Non-Streaming	Streaming
Retry support	Yes	No
Progressive display	No	Yes (via `completeFields`)
Time to first data	After full generation	Immediate
Use case	Reliability-critical	UX-critical

Next Steps

Streaming — Text streaming basics
Tool Use — Let models call functions
Error Handling — Handle errors gracefully