More tutorials

[SamDuffield]

Some ideas:

• Train an energy-based model i.e. Boltzmann machine with SGLD
• Demonstrate composability with a Bayesian model in a PPL (e.g. Pyro)
• A reinforcement learning/bandits problem using EKF (paper)

More ideas welcome!!

[simeneide]

HI @SamDuffield
first, thanks for making this library. Ive built various bayesian inference algorithms in pytorch before, but its good to have everything organized into a package!

What is your plan ahead with this repo? I see there hasnt been too much activity the last couple of months.. I think more tutorials and uncertainty visualization would be very good. I played around a bit with it this weekend and created an mnist classifier with sgmcmc, but i did not manage to get any variability of the posterior (it was effectively all point estimates in the posterior predictive). It also required me to build some more things like a trace collector etc.

[SamDuffield]

Hi @simeneide ! Great to see you exploring the package!!

Yes we'd really like to recover some momentum going with posteriors.

My vision for the library is to really establish mini-batch inference as first class for Bayesian statistics, this would include proper benchmarks and adaptation methods, whilst still keeping the close relationship to stochastic gradient optimization. But I'd love to hear your thoughts on what features you'd like to see added! I'd be very happy to work with you to add them.

Regarding the variability in the posterior, mode collapse can happen with SGMCMC, in the posteriors paper we argued that parallel SGMCMC/deep ensemble provides much better diversity.

[simeneide]

Great. Ill play around a bit more. on the last, yes, there are numerous problems like this. thats why it would be great with a simple example showing how you could use the posterior on some demo data :) It was one of the things I would need to see "ah this library makes sense, I can continue develop here"

Ill keep experimenting a bit, the idea here is exactly what I missed in my previous bayes-nn-projects (pyro became a bit too involved and detached from generic modelling)