Skip to content

Conversation

@harveylv
Copy link

Problem

The fopen call in demo.c line 150 uses text mode ("r+") when opening key files for updates. This causes issues on Windows due to automatic line ending translation:

  • When reading: \r\n (CRLF) is translated to \n (LF)
  • When writing: \n (LF) is expanded to \r\n (CRLF)

This breaks the LMS algorithm when updating the signature counter in key files. For example, when the counter reaches 0x0A, it gets written as 0x0D0A instead of just 0x0A, corrupting the key format and causing signature failures.

Solution

Changed the file mode from "r+" (text mode) to "rb+" (binary mode) to prevent automatic line ending translation and ensure byte-for-byte accuracy.

Changes

  • File: demo.c
  • Line: 150
  • Change: fopen(filename, "r+")fopen(filename, "rb+")

Testing

  • Compiled successfully on Windows using: make CC=C:\msys64\ucrt64\bin\gcc.exe AR=C:\msys64\ucrt64\bin\ar.exe CFLAGS="-Wall -O3 -static"
  • All existing functionality preserved
  • Binary mode ensures consistent behavior across platforms

Reproduction Steps (Before Fix)

  1. Generate an LMS key pair on Windows
  2. Sign multiple messages to increment the counter
  3. When counter reaches 0x0A (or any byte containing \n), the key file gets corrupted
  4. Subsequent signatures fail due to malformed key data

Impact

This fix ensures the LMS implementation works correctly on Windows by maintaining binary file integrity for key updates.

Change fopen mode from 'r+' to 'rb+' in demo.c to prevent
automatic line ending translation on Windows that corrupts
LMS key files during counter updates.
@onaki174
Copy link

onaki174 commented Feb 9, 2026

Pls fix this issue on 192 branch also.
We also faced this issue, on 192 branch, but the next signatures weren't fail, but the <>.sig file is corrupted.
I.E. our reproduction steps are:

  1. With demo windows tool, generate a key, by running the command: demo.exe genkey <key_name> SHA192,15/4
  2. Sign a binary file with this key 11 times, by running the command: demo.exe sign <key_name> <file_to_sign>
  3. In the 11th signing time, the signing is success, but the <>.sig file get longer length, and the field sig_lms_type (in offset 1260, 1 DW) is set to 0xE instead of 0xC that expected for this kind of key.
    Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants