feat: allow specifying stream buffer size

[francisdb]

With some more benchmarks I found that the buffer size is a serious bottleneck for larger streams >= 1MB

write several medium streams/total/buffer=8192
                        time:   [11.472 ms 11.629 ms 11.718 ms]
                        thrpt:  [4.1671 GiB/s 4.1988 GiB/s 4.2561 GiB/s]
write several medium streams/total/buffer=65536
                        time:   [6.7118 ms 6.7530 ms 6.8050 ms]
                        thrpt:  [7.1753 GiB/s 7.2306 GiB/s 7.2750 GiB/s]

[francisdb]

@mdsteele I'm not so sure about the proliferation of the functions for creating streams but did not want to break backwards compatibility. What are your thoughts? I like to keep the buffer size per stream. Clients can then choose to only set it for big streams where it matters.

I have some further more involving refactors that I will create a PR for once this is merged.

[mdsteele]

Interesting, I'm surprised that a buffer larger than 8192 bytes makes such a large difference. Maybe there is just too much overhead right now with refilling the buffer?

Regardless, I agree it makes sense to give the caller control over the buffer size when needed; BufReader for example has similar functionality.

I do share your concern over the proliferation of stream-creation methods, especially if we ever need to add another parameter in the future. One possibility to consider would be to create a new type akin to std::fs::OpenOptions, which could allow specifying the buffer size in addition to e.g. create-new/allow-existing flags. Then we'd only need a single new open_stream_with_options (or whatever the right name is) method, with the existing methods serving the common cases. And we could add new options in the future without adding more methods.

[mdsteele]

(To clarify, BTW, I'm open to being convinced either way on the API question; just raising another possible option.)

[francisdb]

@mdsteele switched to options

[mdsteele]

Thanks!