-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Newer OpenAI models do not support temperature=0.0
From RISE-UNIBAS/humanities_data_benchmark#84
Description
GPT-5 and GPT-5-mini models do not support temperature=0.0 and only accept the default temperature value of 1. This causes repeated API errors and retry attempts, wasting time and API credits.
Hotfix
In humanities_data_benchmark\scripts\benchmark_base.py line 41-42 added:
if self.model in ["gpt-5", "gpt-5-mini"]:
self.temperature = 1
Please remove this hotfix after the issue is resolved.
Error Details
Error code: 400 - {'error': {'message': "Unsupported value: 'temperature' does not support 0.0 with this model. Only the default (1) value is supported.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'unsupported_value'}}
Affected Models
gpt-5gpt-5-minigpt-5-nano- probably 5.1 and 5.2 as well
o3
Some affected Tests
- T0108, T0109, T0110 (business_letters with gpt-5)
- T0111, T0112, T0113, T0114, T0115, T0116 (business_letters with gpt-5-mini)
- T133 (o3)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working