Skip to content

Conversation

@wmanley
Copy link

@wmanley wmanley commented Apr 24, 2021

This will allow using DirIter with rayon's par_bridge.

It may not be thread-safe to call readdir concurrently from multiple
threads on a single DIR*, but all Send requires is that we can call
it from different threads non-concurrently - so this is fine.

man readdir says:

It is expected that a future version of POSIX.1 will require that
readdir() be thread-safe when concurrently employed on different
directory streams.

so in the future we may also be able to implement Sync. The std
implements both Send and Sync on ReadDir, but they implement
iteration using the deprecated readdir_r.

To be sure I've also looked at the implementation of readdir on glibc,
musl and freebsd and they all use per struct DIR allocations for
the dirent buffer.

This will allow using `DirIter` with rayon's `par_bridge`.

It may not be thread-safe to call readdir concurrently from multiple
threads on a single `DIR*`, but all `Send` requires is that we can call
it from different threads non-concurrently - so this is fine.

`man readdir` says:

> It is expected that a future version of POSIX.1 will require that
> `readdir()` be thread-safe when concurrently employed on different
> directory streams.

so in the future we may also be able to implement `Sync`.  The `std`
implements both `Send` and `Sync` on [`ReadDir`], but they implement
iteration using the deprecated [`readdir_r`].

To be sure I've also looked at the implementation of readdir on [glibc],
[musl] and [freebsd] and they all use per `struct DIR` allocations for
the dirent buffer.

[`ReadDir`]: https://github.com/rust-lang/rust/blob/7e11f3a8f3c1b2683125e7def0acb68a6d684f92/library/std/src/sys/unix/fs.rs#L478-L511
[`readdir_r`]: https://man7.org/linux/man-pages/man3/readdir_r.3.html
[glibc]: https://code.woboq.org/userspace/glibc/sysdeps/posix/readdir.c.html
[musl]: https://git.musl-libc.org/cgit/musl/tree/src/dirent/readdir.c
[freebsd]: https://github.com/freebsd/freebsd-src/blob/373ffc62c158e52cde86a5b934ab4a51307f9f2e/lib/libc/gen/opendir.c#L337
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant