Skip to content

Feature Request: Floating-Point Display Precision Control #408

@byrnHDF

Description

@byrnHDF

Feature Request: Floating-Point Display Precision Control

Summary

Add user-configurable precision control for floating-point number display in HDFView, allowing users to control how many decimal places are shown.

Motivation

Currently, HDFView displays floating-point values with full precision, which can lead to:

  • Visual clutter with values like 2.0999999999999996 instead of 2.1
  • Difficulty comparing values visually
  • Confusion about actual data vs. display artifacts

Many scientific applications allow users to control display precision separate from the underlying data precision.

Proposed Solution

1. Add Precision Setting to User Options

Location: User Preferences → HDF Options (UserOptionsHDFPage.java)

New Setting:

  • Name: "Floating-point display precision"
  • Type: Integer spinner (range: 1-16, default: 6)
  • Property key: ViewProperties.DATA_DISPLAY_PRECISION

2. Display Formatting

Apply precision when formatting numbers for display:

// Example implementation in DataDisplayConverterFactory
String format = String.format("%%.%df", precision);
displayValue = String.format(format, doubleValue);

3. User Interface

Add to HDF Options dialog:

┌─ Display Options ────────────────────┐
│                                      │
│  [✓] Show scientific notation        │
│                                      │
│  Floating-point precision: [6  ▼]   │
│  (Number of decimal places: 1-16)    │
│                                      │
└──────────────────────────────────────┘

Benefits

  1. Better readability: Users can hide insignificant digits
  2. Visual comparison: Easier to spot differences in data
  3. Publication-ready: Match journal/paper formatting requirements
  4. Performance: Potentially faster rendering with shorter strings
  5. User control: Power users can still see full precision when needed

Implementation Details

Files to Modify

  1. ViewProperties.java

    • Add DATA_DISPLAY_PRECISION constant
    • Add getter/setter methods
  2. UserOptionsHDFPage.java

    • Add precision spinner control
    • Load/save precision setting
  3. DataDisplayConverterFactory.java

    • Apply precision when formatting float/double values
    • Respect scientific notation setting
  4. Complex Number Formatting

    • Apply to both real and imaginary parts
    • Format: 1.23+4.56i with configurable precision

Default Behavior

  • Default precision: 6 decimal places (matches scientific convention)
  • Preserve full precision: Still store/save full double precision
  • Scientific notation: Respect existing show/hide setting
  • Complex numbers: Apply to both real and imaginary parts

Edge Cases

  1. Very small/large numbers: Use scientific notation automatically when appropriate
  2. Integer values: Display without decimal point (1 not 1.000000)
  3. Special values: NaN, Inf, -Inf display as-is
  4. Complex zero: 0.0+0.0i simplified appropriately

Testing Considerations

  1. Test with various precision settings (1, 3, 6, 12, 16)
  2. Test with values that have floating-point representation issues (0.1, 0.2, 2.1, etc.)
  3. Test with complex numbers
  4. Test with arrays of different datatypes (float32, float64, float16)
  5. Verify setting persists across HDFView sessions

Related Issues

  • Floating-point precision display has been a common user question
  • Affects all numeric datatypes: float16, bfloat16, float32, float64, long double
  • Particularly important for complex datatypes where precision shows in both components

Priority

Medium - Nice-to-have improvement for user experience, not critical functionality

Estimated Effort

  • Small to Medium (few hours)
  • Low complexity - straightforward formatting change
  • Well-defined scope

Metadata

Metadata

Labels

Component - HDFViewImprovements to the visual interface layer

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions