Question/feature request about amortized inference

[itsdfish]

Hello,

I have been looking for a Julia package to perform amortized Bayesian inference on simulation-based models. In many areas of science, there are models with unknown/intractable likelihood functions. Unfortunately, Bayesian inference with these models have historically been difficult. Somewhat recently there has been progress in this area using a special type of normalizing flow. This method uses a neural network to optimize summary statistics for approximate Bayesian computation. The end result is very accurate amortized inference which can be used for any model, including those without a known likelihood function. Currently, this method is only implemented in a Python package called BayesFlow.

Given Turing's interest in machine learning and Bayesian inference, I was wondering whether there is interest in adding this method to the Turing ecosystem. I think it would add a lot of value to the community.

[Red-Portal]

Amortized inference would be something that is supported by AdvancedVI.jl, but unfortunately, it is not yet possible to do this without pirating the internals. Supporting amortization is on the agenda of AdvancedVI.jl though.

[itsdfish]

Thank you for your reply. Please feel free to close or transfer this issue.

[Red-Portal]

With the current state of things, I think this would be a simple example in the documentation once #38 is worked out.

[itsdfish]

I am not an expert in NNs, but my understanding is that part of the "magic" of BayesFlow is that it uses either a DeepSet or SetTransformer for its summary network. I couldn't find any record of that in AdvancedVI. Does this exist in Julia or is it something you plan to add?

[Red-Portal]

The magic of Julia is that we don't everything to be implemented in-house in order for things to work. If DeepSets is implemented elsewhere, it could definitely be used here. Though using transformers for general Bayes look quite a bit overkill to me.