Add a progressr-based progress bar for sampling.#1138
Add a progressr-based progress bar for sampling.#1138josswright wants to merge 16 commits intostan-dev:masterfrom
progressr-based progress bar for sampling.#1138Conversation
This uses the [`progressr`](https://progressr.futureverse.org/index.html) framework to enable a progress bar for sampling operations. By default, this replaces standard iteration messages, but not other informative messages produced during sampling. From a user point of view, this adds two arguments to the `$sample()` method: - `show_progress_bar`: Default = FALSE If TRUE, registers a progress bar via `progressr` and signals an update for every output line that matches "Iteration:". The user is responsible for registering those progress updates with an appropriate handler. - `suppress_iteration_messages`: Defaults to the value of show_progress_bar, but can also be set directly. If TRUE, disables display of output lines matching "Iteration:", while still causing the progress bar to update. This keeps all of Stan's other informative output, but just removes the superfluous iteration messages when using a progress bar. I've tried to keep all additions to the code as non-intrusive as possible. One minor decision I made was to pass the value of `refresh` from the `$sample()` method through to the `CmdStanProcs` class so that the progress bar can report and update the number of steps in the bar to the same 'scale' as the number of iterations. (Calling `$sample()` with `iter_sampling=8000` and `refresh=100` will result in 80 'ticks' of the progress bar, each increasing the progress by 100.) By default, `progressr` doesn't register a handler to display the progress bar, as this is the responsibility of the user. Multiple packages can be used to display the progress bar. A default progress bar can be registered using the [`cli`](https://cli.r-lib.org/) library in this way: ```r library(progressr) library(cli) handlers( global=TRUE ) handlers("cli") options( cli.spinner = "moon", cli.progress_show_after = 0, cli.progress_clear = FALSE ) handlers( handler_cli( format = "{cli::pb_spin} Progress: |{cli::pb_bar}| {cli::pb_current}/{cli::pb_total} | {cli::pb_percent} | ETA: {cli::pb_eta}", clear = FALSE )) ``` A variety of alternative progress bar handlers are available, including audible and notification-based handlers, and ones that interact with RStudio's jobs pane: <https://progressr.futureverse.org/articles/progressr-11-handlers.html>
In addition to updating the progress bar on stdout lines that match "Iteration:", the CmdStanProcs object(s) now pass the current stdout line to the progress bar as a message for _every_ line, even if the number of completed iterations hasn't increased.
Added a `register_default_progress_handler()` function to `utils.R` that creates a default progress bar for sampling operations and registers it as global handler for progressr. Requires `cli` and `progressr`.
Was accidentally passing `refresh` to each informational progress bar update, rather than the calculated `progress_amount`. This caused each message line to update the progress bar, even if it didn't match an iteration message.
Moved code to close the progress bar to after the `check_finished()` function.
The nature of CmdStan's output makes pulling appropriate refresh and update values for the progress bar slightly tricky. In the original verison of the code, this led to the bar occasionally updating past the point where it was full, or terminating early due to an incorrectly calculated number of steps. This commit reworks the calculations of updates and refresh values for the progress bar, as well as adding some extra conditions to avoid the bar potentially crashing in odd scenarios.
New arguments for `iter_warmup` and `iter_sampling` were not being assigned a default value if not specified in sampling statement.
|
Looks like some tests are failing because the lower bound for iter_sampling should be 0 instead of 1. |
Great. Thanks for this. (First pull request for me on a project like this, so I was a little unfamiliar with the process and running the unit tests.) I've fixed the initial errors, and a couple of other warnings. Now running the tests and documentation generation commands locally before I push an update. |
- `iter_{warmup,sampling}` arguments to MCMCProcs are allowed to be 0.
- Documentation for `register_default_progress_handler` added to
repository.
- Ensured that calls to `progressr` functions are appropriately
prefixed.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1138 +/- ##
==========================================
- Coverage 87.49% 86.92% -0.58%
==========================================
Files 14 14
Lines 5983 6087 +104
==========================================
+ Hits 5235 5291 +56
- Misses 748 796 +48 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Added documentation for `show_progress_bar` and `suppress_iteration_messages` in `sample()` function. Changed `require()` calls for `progressr` and `cli` to suggested `requireNamespace()` alternatives.
|
Thanks for the help with the unit tests. In the spirit of peer review, some notes and thoughts:
|
|
Awesome, thank you for the PR! I think I have the flu right now, but I will take a look when I'm feeling better and caught up on work. |
No rush on my side! Hope you feel better soon. |
4 participants
Submission Checklist
Summary
As discussed in #18 , this pull request adds a progress bar to CmdStanR's
sample()function via theprogressrframework.This can be enabled via two new options to
model$sample(...):show_progress_bar- boolean - defaultFALSEEnables the progress bar.
suppress_iteration_messages- boolean - defaultFALSEifshow_progress_barisFALSE, andTRUEifshow_progress_barisTRUE.Causes any output lines reporting an iteration (such as
Chain 1 Iteration 1 / 2000 (Warmup)) to be suppressed.For the progress bar to be displayed, a progress bar handler must be created and registered by the end user. (See: https://progressr.futureverse.org/) For ease of use, a
register_default_progress_handler()function has been added. Callingregister_default_progress_handler()before amodel$sample(...)call will spawn a default progress bar via the cli library.Copyright and Licensing
Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
Joss Wright, University of Oxford
By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses: