Fix V (overflow) flag calculation in ADC and SBC

[garry-jeromson]

Summary

The V flag should indicate signed overflow - when the result of a signed operation doesn't fit in the destination size. The previous code was checking overflow on the result value, but wasn't properly sign-extending the operands before the arithmetic.

The Bug

For example, in 8-bit mode:

• 0x7F + 0x01 should set V (127 + 1 = 128, overflows signed 8-bit) ✓
• But 0x80 + 0x80 was computed as al = 0x100, then (int16_t)0x100 = 256, which the old code thought was overflow
• The actual signed result is -128 + -128 = -256, which DOES overflow, but the detection was accidental

The old approach worked in many cases but failed when the unsigned result wrapped around.

The Fix

Sign-extend each operand to the wider type BEFORE computing, giving the mathematically correct signed result to check against the valid range:

// Old (incorrect for some cases):
al = (cpu->C & 0xff) + val + cpu->P.C;
cpu->P.V = ((int16_t)al < -128 || (int16_t)al > 127) ? 1 : 0;

// New (correct):
int16_t signed_al = (int8_t)(cpu->C & 0xff) + (int8_t)val + cpu->P.C;
cpu->P.V = (signed_al < -128 || signed_al > 127) ? 1 : 0;

Affected Instructions

• ADC in 8-bit binary mode
• ADC in 16-bit binary mode
• SBC in 8-bit mode (binary and decimal)
• SBC in 16-bit mode (binary and decimal)

Test plan

• Verified with W65816 LLVM backend test suite (signed comparison tests)

🤖 Generated with Claude Code