Skip to content

OpenAIClient: Newer models do not support temperature=0.0 #2

@MHindermann

Description

@MHindermann

Newer OpenAI models do not support temperature=0.0

From RISE-UNIBAS/humanities_data_benchmark#84

Description

GPT-5 and GPT-5-mini models do not support temperature=0.0 and only accept the default temperature value of 1. This causes repeated API errors and retry attempts, wasting time and API credits.

Hotfix

In humanities_data_benchmark\scripts\benchmark_base.py line 41-42 added:

if self.model in ["gpt-5", "gpt-5-mini"]:
    self.temperature = 1

Please remove this hotfix after the issue is resolved.

Error Details

Error code: 400 - {'error': {'message': "Unsupported value: 'temperature' does not support 0.0 with this model. Only the default (1) value is supported.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'unsupported_value'}}

Affected Models

  • gpt-5
  • gpt-5-mini
  • gpt-5-nano
  • probably 5.1 and 5.2 as well
  • o3

Some affected Tests

  • T0108, T0109, T0110 (business_letters with gpt-5)
  • T0111, T0112, T0113, T0114, T0115, T0116 (business_letters with gpt-5-mini)
  • T133 (o3)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions