Skip to content

Claude Sonnet 4.5 Returns Incorrectly Wrapped Structured Response #4

@MHindermann

Description

@MHindermann

Claude Sonnet 4.5 Returns Incorrectly Wrapped Structured Response

See RISE-UNIBAS/humanities_data_benchmark#84

Claude Sonnet 4.5 (claude-sonnet-4-5-20250929) is wrapping structured output responses in a parameter object instead of returning the Pydantic model directly, causing validation failures.

Expected Behavior

{
    "metadata": {...},
    "entries": [...]
}

Actual Behavior

{
    "parameter": {
        "metadata": {...},
        "entries": [...]
    }
}

See request_T0225_page_10.json.

Error

2026-01-24 13:52:32,459 INFO:root:Running T0225
2026-01-24 13:52:32,460 INFO:root:Loaded bibliographic_data from ../benchmarks\bibliographic_data\benchmark.py
2026-01-24 13:52:32,666 INFO:root:Running bibliographic_data (anthropic/claude-sonnet-4-5-20250929)...
2026-01-24 13:52:32,667 INFO:root:Found 5 objects to process.
2026-01-24 13:52:32,667 INFO:root:Processing T0225, page_10...
2026-01-24 13:52:32,705 INFO:ai_client.utils:Resized image page_10.jpeg from (1825, 2896) to (1291, 2048)
2026-01-24 13:53:02,806 INFO:httpx:HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
2026-01-24 13:53:02,807 WARNING:ai_client.claude_client:Pydantic validation failed: 2 validation errors for Document
metadata
 Field required [type=missing, input_value={'parameter': {'metadata'... 'incomplete': False}]}}, input_type=dict]
   For further information visit https://errors.pydantic.dev/2.10/v/missing
entries
 Field required [type=missing, input_value={'parameter': {'metadata'... 'incomplete': False}]}}, input_type=dict]
   For further information visit https://errors.pydantic.dev/2.10/v/missing
2026-01-24 13:53:02,827 INFO:root:Saved answer to ..\results\2026-01-24\T0225\request_T0225_page_10.json
2026-01-24 13:53:02,827 INFO:root:Finished page_10 with score: {'fuzzy': 0.0}

Model

  • Provider: anthropic
  • Model: claude-sonnet-4-5-20250929
  • Test: T0225 (bibliographic_data benchmark)

Context

  • Using Anthropic's native tool calling/structured output
  • Pydantic model: Document with fields metadata and entries
  • Other Claude models may not have this issue (needs verification)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions