-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Claude Sonnet 4.5 Returns Incorrectly Wrapped Structured Response
See RISE-UNIBAS/humanities_data_benchmark#84
Claude Sonnet 4.5 (claude-sonnet-4-5-20250929) is wrapping structured output responses in a parameter object instead of returning the Pydantic model directly, causing validation failures.
Expected Behavior
{
"metadata": {...},
"entries": [...]
}Actual Behavior
{
"parameter": {
"metadata": {...},
"entries": [...]
}
}See request_T0225_page_10.json.
Error
2026-01-24 13:52:32,459 INFO:root:Running T0225
2026-01-24 13:52:32,460 INFO:root:Loaded bibliographic_data from ../benchmarks\bibliographic_data\benchmark.py
2026-01-24 13:52:32,666 INFO:root:Running bibliographic_data (anthropic/claude-sonnet-4-5-20250929)...
2026-01-24 13:52:32,667 INFO:root:Found 5 objects to process.
2026-01-24 13:52:32,667 INFO:root:Processing T0225, page_10...
2026-01-24 13:52:32,705 INFO:ai_client.utils:Resized image page_10.jpeg from (1825, 2896) to (1291, 2048)
2026-01-24 13:53:02,806 INFO:httpx:HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
2026-01-24 13:53:02,807 WARNING:ai_client.claude_client:Pydantic validation failed: 2 validation errors for Document
metadata
Field required [type=missing, input_value={'parameter': {'metadata'... 'incomplete': False}]}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.10/v/missing
entries
Field required [type=missing, input_value={'parameter': {'metadata'... 'incomplete': False}]}}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.10/v/missing
2026-01-24 13:53:02,827 INFO:root:Saved answer to ..\results\2026-01-24\T0225\request_T0225_page_10.json
2026-01-24 13:53:02,827 INFO:root:Finished page_10 with score: {'fuzzy': 0.0}
Model
- Provider: anthropic
- Model: claude-sonnet-4-5-20250929
- Test: T0225 (bibliographic_data benchmark)
Context
- Using Anthropic's native tool calling/structured output
- Pydantic model:
Documentwith fieldsmetadataandentries - Other Claude models may not have this issue (needs verification)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working