Hello 👋 As the development of Pyro's normalizing flow API has stopped for quite some time, I have been discussing with @fritzo about recommending an external normalizing flow library that would be compatible with Pyro.

I believe that Zuko would be a great recommendation, but disclaimer I am its maintainer.

Why Zuko?

I have extensively used and developed normalizing flows for my research and I always preferred the approach Pyro takes to define parametrized and conditional transformations over other libraries (nflows, FrEIA, FlowTorch, ...). In particular, I liked that Pyro's distributions/transformations shared the interface of PyTorch distributions/transformations, which is not the case of other libraries.

Hence, I started (2 years ago) coding my own normalizing flows for my research and at some point (1 year ago) decided to publish it as an open-source library: Zuko. The project has grown and now features many flow architectures (MAF, NSF, CNF, NAF, UMNN, GF, ...), automatic tests, a fully documented API and tutorials. The repository does not have as many stars as other libraries (mainly because I never advertised it), but actually has many users.

Most importantly, Zuko is very compatible with Pyro as it relies heavily on PyTorch's distributions interface. In Zuko, a flow is a distribution, with sample, log_prob, even_shape, etc.

Zuko is a reference to a powerful firebender (pyromancer) in the Avatar cartoon.

Roadmap

If the Pyro team agrees with recommending Zuko, I propose to

• Update the current normalizing flow tutorial to mention that the flow API is not actively developed anymore.
• Add a simple normalizing flow tutorial with Zuko. This would probably take inspiration from Zuko's tutorials.
• Add a SVI tutorial with a normalizing flow guide.
• Add a VAE tutorial with a normalizing flow prior.

WDYT?