requests with cipher(DHE-RSA-AES256-GCM-SHA384) fail with internal error

[Sheshagiri]

Our team recently moved our nginx image from CBL-Mariner to AzureLinux. After the move we started noticing some of the requests failing with the following error(from nginx logs) during TLS handshake.

SSL: error:0A0C0103:SSL routines::internal error

Upon further investigation we identified that AzureLinux uses SymCrypt under the hood as default crypto library.

When we made openssl the default library, the failures are gone and the request with cipher(DHE-RSA-AES256-GCM-SHA384) work as expected.

Change:

Test with openssl as default provider :

Test with SymCrypt as default provider:

Mariner Nginx Image: mcr.microsoft.com/cbl-mariner/base/nginx:1.22 (cipher mentioned here is working fine in this image)
Azure Linux Nginx Image: mcr.microsoft.com/azurelinux/base/nginx:1.25

Verified with curl from inside the container.
curl -k --tlsv1.2 --tls-max 1.2 --ciphers DHE-RSA-AES256-GCM-SHA384 -v https://localhost/status

OS Details:

root [ / ]# cat /etc/os-release
NAME="Microsoft Azure Linux"
VERSION="3.0.20250910"
ID=azurelinux
VERSION_ID="3.0"
PRETTY_NAME="Microsoft Azure Linux 3.0"
ANSI_COLOR="1;34"
HOME_URL="https://aka.ms/azurelinux"
BUG_REPORT_URL="https://aka.ms/azurelinux"
SUPPORT_URL="https://aka.ms/azurelinux"

RPM Details:

root [ / ]# tdnf list installed | grep SymCrypt
SymCrypt.x86_64                              103.8.0-1.azl3            @System
SymCrypt-OpenSSL.x86_64                      1.9.1-1.azl3              @System
root [ / ]# tdnf list installed | grep openssl
openssl.x86_64                               3.3.5-1.azl3              @System
openssl-libs.x86_64                          3.3.5-1.azl3              @System
root [ / ]#

[mamckee]

The requests only fail when custom DH parameters are used. The SymCrypt provider currently does not support DH key exchange with custom DH parameters. The recommendation is to use a named group if you must use this cipher suite.

We will investigate whether the SymCrypt provider should add support for custom DH groups or continue only allowing named groups.