ENH: Sampling a contiguous block from a DF

[YM2132]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

I wish I could use Pandas to return a random block of data from a DF.

Like .sample() but the data returned is contiguous, in essence the method randomly samples 1 row idx and then samples the next N from that point.

Feature Description

Add a contiguous param to .sample or create a new .sample_contgiuous

Draw from distribution a single idx

and select N around that as a contiguous block

Alternative Solutions

def random_contiguous_sample(df, n):
max_start = len(df) - n

start_idx = np.random.randint(0, max_start + 1)
return df.iloc[start_idx:start_idx + n]

Additional Context

No response

[gautamvarmadatla]

Hi @YM2132,

This would be really handy. I have wanted “.sample() but as a contiguous window” a few times ( for traaining purposes )

Based off your idea, my thought is that a new method like sample_window or sample_contiguous might be easier to land than adding contiguous= to .sample(). .sample() already has a lot of knobs like weights, replace, frac, and axis, and it is not obvious how they should work for a window. For example, would weights apply to rows or to possible start positions, and would replace=True mean replacing rows or replacing windows.

If we go the new methd route, we could define it as positional and iloc-based. Pick a random start in [0, len(obj) - n] and return obj.iloc[start:start+n]. This keeps it independent of index labels or timestamps. If someone wants contiguous in time, they can sort_index() first.

Also, it would be good to agree on behavior when n > length, when n = zero, when the object is empty, and when n is negative. I can add tests for non-RangeIndex, duplicate labels, and MultiIndex as well to confirm the positional behavior stays consistent.

If this direction sounds good, I am happy to work on a PR with implementation, tests, and docs.

[YM2132]

Hey @gautamvarmadatla,

I agree with your point a new method sounds better, adding it to .sample also sort of breaks what .sample() is about (random independent samples).

n > length you'd want an error to be raised right?
n = zero also an error ? as it shouldn't be possible to sample a window of size 0 etc nothing
It'll be worth reviewing how pandas typically handles such cases, etc what if you sample n > len(df)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[20], line 6
      1 test_df = pd.DataFrame(
      2     np.random.randn(5, 3),
      3     columns=['Column_A', 'Column_B', 'Column_C']
      4 )
----> 6 test_df.sample(10)

File ~/Documents/PyCharm_Projects/gaze_detection/venv313/lib/python3.13/site-packages/pandas/core/generic.py:6140, in NDFrame.sample(self, n, frac, replace, weights, random_state, axis, ignore_index)
   6137 if weights is not None:
   6138     weights = sample.preprocess_weights(self, weights, axis)
-> 6140 sampled_indices = sample.sample(obj_len, size, replace, weights, rs)
   6141 result = self.take(sampled_indices, axis=axis)
   6143 if ignore_index:

File ~/Documents/PyCharm_Projects/gaze_detection/venv313/lib/python3.13/site-packages/pandas/core/sample.py:152, in sample(obj_len, size, replace, weights, random_state)
    149     else:
    150         raise ValueError("Invalid weights: weights sum to zero")
--> 152 return random_state.choice(obj_len, size=size, replace=replace, p=weights).astype(
    153     np.intp, copy=False
    154 )

File numpy/random/mtrand.pyx:1020, in numpy.random.mtrand.RandomState.choice()

ValueError: Cannot take a larger sample than population when 'replace=False'

To your last point, I was hoping to work on this PR myself as it would be my first contribution to pandas. I'd really appreciate your support and I'm sure I'll get stuck, perhaps I give it a crack and let you know how I get on with it?

Finally, what is the typical approach to PRs in pandas? Can we start coding right away or shall we wait for a maintainer to agree on direction?

[gautamvarmadatla]

Hi @YM2132,

sure! Please take a look at this link for contribs. I wanted to be sure with the API design and tests ( including all the edge cases ) with a maintainer before starting.

[YM2132]

@gautamvarmadatla

Sounds good will check it out, however I havent found any reference which says explicitly that we ought to wait for maintainer approval. Suppose it's a slow period too.

Perhaps I can put together an MVP PR and will let you know once I get something up and running, then can start expanding out to edge cases + feedback