Floating point precision bugs?

[patrickmeehan]

Any interest in glitch cases due to floating point precision? I'm opening this issue in case you're looking for repo steps.

The code below shows the error. We'd expect a result with 8 verts, but instead get 7. Rounding to 4 decimal places gives the expected result.

I vaguely recall a way to configure fp rounding in an earlier version, but can't seem to find it in the history. I added rounding to my calling code, so not a big deal. But it would be awesome to see a more robust way to handle these cases or a note in the docs about why they happen and how to avoid them.

Thanks for making a great library! :)

// clamping these to 4 decimal places gives the expected result
float contour0 [] = {
    40.000000, 36.770508,
    -33.541019, 0.000000,
    40.000000, -36.770508,
};

float contour1 [] = {
    -65.000000, 65.000000,
    -65.000000, -65.000000,
    65.000000, -65.000000,
    65.000000, -15.729490,
    10.000000, 11.770510,
    10.000000, -10.000000,
    -10.000000, -10.000000,
    -10.000000, 10.000000,
    10.000000, 10.000000,
    10.000006, -11.770506,
    65.000008, 15.729493,
    65.000000, 65.000000,
};

float contour2 [] = {
    115.000000, 74.270508,
    35.000004, 34.270512,
    35.000000, -34.270508,
    115.000000, -74.270508,
};

TESStesselator* tess = tessNewTess ( 0 );

tessAddContour ( tess, 2, contour0, 8, 3 );
tessAddContour ( tess, 2, contour1, 8, 12 );
tessAddContour ( tess, 2, contour2, 8, 4 );

float normal [] = { 0, 0, 1 };

// should give us a shape with 8 vertices
tessTesselate ( tess, TESS_WINDING_NONZERO, TESS_BOUNDARY_CONTOURS, 0, 0, normal );

const float* verts = tessGetVertices ( tess );
const int* elems = tessGetElements ( tess );
int nelems = tessGetElementCount ( tess );

for ( int i = 0; i < nelems; ++i ) {
    int b = elems [( i * 2 )];
    int n = elems [( i * 2 ) + 1 ];
    
    for ( int j = 0; j < n; ++j ) {
    
        size_t idx = ( b + j ) * 2;
        float x = verts [ idx ];
        float y = verts [ idx + 1 ];
    
        printf ( "%d %d: %f %f\n", i, j, x, y );
    }
}

tessDeleteTess ( tess );

[sfreilich]

@patrickmeehan Probably not going to poke at this one myself, but I want to point out that I added tests to this library recently, and it would be pretty easy to slot the above example into a unit test if you want to poke at it.