Skip to content

Conversation

@ajpotts
Copy link
Contributor

@ajpotts ajpotts commented Jan 20, 2026

Add stateless, index-based RNG via SplitMix64

Summary

This PR introduces a stateless, index-based random number generator to Arkouda using a SplitMix64 mixing function. The new implementation enables reproducible random sequences that are independent of execution order, parallelism, and locale count, making them suitable for sampling, shuffling, sketching, and other distributed algorithms.

The PR adds:

  • A new Chapel module (SplitMix64RNG) implementing SplitMix64-based kernels
  • Python-side generator APIs for stateless uint64 and uniform[0,1) generation
  • In-place and allocating variants
  • Comprehensive correctness and statistical test coverage

Motivation

Existing RNG paths in Arkouda rely on mutable state, which can make results sensitive to chunking and execution order. Many analytics and data-parallel algorithms instead need pure functions of (seed, stream, index).

Design & Implementation

  • Chapel: new SplitMix64RNG module with fillRandU64 and fillUniform01 kernels
  • Python: new internal Generator methods for stateless RNG
  • Deterministic mapping: r[i] = splitmix64((start_idx + i) ^ key(seed, stream))

Testing

Adds extensive reproducibility, statistical, and independence tests, with conservative thresholds to avoid CI flakiness.

Compatibility

No changes to existing RNG behavior; new functionality is opt-in.

Future Work

Potential public API exposure and use in distributed shuffle/sampling paths.

Closes #5316: SplitMix64RNG

@ajpotts ajpotts force-pushed the 5316_SplitMix64RNG branch 12 times, most recently from 7fc8273 to 90d54b8 Compare January 22, 2026 12:45
@codecov
Copy link

codecov bot commented Jan 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@9bedaa2). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #5323   +/-   ##
========================================
  Coverage        ?   100.00%           
========================================
  Files           ?         4           
  Lines           ?        63           
  Branches        ?         0           
========================================
  Hits            ?        63           
  Misses          ?         0           
  Partials        ?         0           
Flag Coverage Δ
python-coverage 100.00% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ajpotts ajpotts marked this pull request as ready for review January 22, 2026 17:37
Copy link
Contributor

@1RyanK 1RyanK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@ajpotts ajpotts force-pushed the 5316_SplitMix64RNG branch from 90d54b8 to 7070782 Compare January 23, 2026 19:25
@ajpotts ajpotts force-pushed the 5316_SplitMix64RNG branch from e47b9f0 to eb72a01 Compare January 23, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SplitMix64RNG

2 participants