Implicit Variational Inference for High-Dimensional Posteriors

Anshuk Uppal, Kristoffer Stensbo-Smidt, Wouter Boomsma, Jes Frellsen

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

12 Downloads (Pure)

Abstract

In variational inference, the benefits of Bayesian models rely on accurately capturing the true posterior distribution. We propose using neural samplers that specify implicit distributions, which are well-suited for approximating complex multimodal and correlated posteriors in high-dimensional spaces. Our approach introduces novel bounds for approximate inference using implicit distributions by locally linearising the neural sampler. This is distinct from existing methods that rely on additional discriminator networks and unstable adversarial objectives. Furthermore, we present a new sampler architecture that, for the first time, enables implicit distributions over tens of millions of latent variables, addressing computational concerns by using differentiable numerical approximations. We empirically show that our method is capable of recovering correlations across layers in large Bayesian neural networks, a property that is crucial for a network's performance but notoriously challenging to achieve. To the best of our knowledge, no other method has been shown to accomplish this task for such large models. Through experiments in downstream tasks, we demonstrate that our expressive posteriors outperform state-of-the-art uncertainty quantification methods, validating the effectiveness of our training algorithm and the quality of the learned implicit approximation.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 36 pre-proceedings (NeurIPS 2023)
Number of pages24
PublisherNeurIPS Proceedings
Publication date2023
Publication statusPublished - 2023
Event37th Conference on Neural Information Processing Systems - NeurIPS 2023 - New Orleans., United States
Duration: 10 Dec 202316 Dec 2023

Conference

Conference37th Conference on Neural Information Processing Systems - NeurIPS 2023
Country/TerritoryUnited States
CityNew Orleans.
Period10/12/202316/12/2023
SeriesAdvances in Neural Information Processing Systems
Volume36
ISSN1049-5258

Cite this