Process/collect hangs on large stdin

[Artamus]

I'm trying to run a command that has a very large output and read it into my program, which causes the stdout collector to hang.

How to reproduce

The best example I can give is to try and read in a large file via cat or less. (In reality I am invoking a command that outputs this large data, but cat and less seem to be a reasonable proxy). In my example the file was about ~350 KiB and was a git diff of a binary file change (i.e. git diff --binary <hash>~ <hash>).

open Feather

let foo = process "less" [ "large-git-diff.diff" ] |> collect stdout in
print_endline foo;

I think this is a Feather-related issue, since a combination of Unix.open_process_in and In_channel.input_all seems to work, but I do enjoy Feather's API more than using raw Unix.

Please let me know if you need more information.

[charlesetc]

Weird, yes it seems to start hang between 44 and 88 kilobytes for me. Not ideal!

I'll try to take a look sometime soon, though I can't guarantee I'll get a fix. Thanks for letting me know!

[rand00]

+1 I also just ran into this issue.

There is so much potential in this library for replacing a lot of shell-scripting and introducing people to OCaml - so I think this isssue should get a high priority. Feather is btw. much better as a direct replacement for shell-scripting than the other related libraries in the ecosystem - many people want the elegance of bash oneliners but with the sensible semantics of OCaml when you e.g. need a variable holding the lines collected from stdout of a command - this library hits the spot. If we can't collect an arbitrary amount of lines from stdout into memory, I think much of the point of this library is lost :(

Btw. my situation is that I made a for-fun repository introducing this library to a friend - and we found that he had too many audio-files for the script to work directly :/ (we piped to a temporary file as a fix, but this limitation doesn't sell this as a bash-alternative too well)

I'll have a look at the implementation of Feather later

[rand00]

Oh - I just pinned to --dev and this is fixed - fantastic. Could we have a release soon?