Skip to content

Precision loss for 64-bit Wasm integers #45

@abelstuker

Description

@abelstuker

Currently, the framework uses JavaScript Number to represent numbers.
This Number type uses the IEEE 754 double precision 64 bit floating point format.
Therefore, this representation results in a loss of precision for 64 bit integers.
More concretely, problems arise when the framework handles i64 (Wasm) values the fall outside the [Number.MIN_SAFE_INTEGER, Number.MAX_SAFE_INTEGER] interval.

The following example demonstrates the problem.
The i64_get_max function returns the largest possible 64-bit integer value.
The i64_eq_max function checks whether the provided argument equals the largest possible 64-bit signed integer value (9223372036854775807).
The i64_eq_max_wrong function checks whether the provided argument equals 9223372036854776000 (the JS Number representation for 9223372036854775807).

(func (export "i64_get_max")        (result i64)  (i64.const 9223372036854775807))
(func (export "i64_eq_max")         (param i64)   (result i32) (i64.eq (local.get 0) (i64.const 9223372036854775807)))
(func (export "i64_eq_max_wrong")   (param i64)   (result i32) (i64.eq (local.get 0) (i64.const 9223372036854776000)))

When executing latch tests, we obtain:

A    FAIL  i64_get_max      | args: []                    | assert_eq: 9223372036854775807n
        failure: Expected 9223372036854775807 got 9223372036854776000

B    PASS  i64_get_max      | args: []                    | assert_eq: 9223372036854775807

C    PASS  i64_get_max      | args: []                    | assert_eq: 9223372036854775500

D    FAIL  i64_eq_max       | args: [9223372036854775807] | assert_eq: 1
        failure: Expected 1 got 0

These results demonstrate the imprecise integer representations of i64 values once present in the framework.
A: The expected return value is correctly represented (as BigInt), but the obtained return value is incorrectly represented.
B: The expected return value (not a BigInt) is incorrectly represented. Also the obtained return value is incorrectly represented (as in A). Hence, the test succeeds (although with incorrect representations).
C: This test shows that mutiple large integers can be rounded (i.e. incorrectly represented) as a single representable JS Number. It shows a second example of a value that equals the obtained result.
D: This test lets the Wasm function check whether the supplied argument equals the largest possible 64-bit signed integer. This check fails.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions