Martin Magris, Mostafa Shabani, Alexandros Iosifidis
arXiv 23 May 2022 · Statistics — Machine Learning · 2 citations (OpenAlex)
arXiv:2205.11568 · PDF · DOI · OpenAlex · Extracted main text
We develop an optimization algorithm suitable for Bayesian learning in complex models. Our approach relies on natural gradient updates within a general black-box framework for efficient training with limited model-specific derivations. It applies within the class of exponential-family variational posterior distributions, for which we extensively discuss the Gaussian case for which the updates have a rather simple form. Our Quasi Black-box Variational Inference (QBVI) framework is readily applicable to a wide class of Bayesian inference problems and is of simple implementation as the updates of the variational posterior do not involve gradients with respect to the model parameters, nor the prescription of the Fisher information matrix. We develop QBVI under different hypotheses for the posterior covariance matrix, discuss details about its robust and feasible implementation, and provide a number of real-world applications to demonstrate its effectiveness.
appendix boundary found by appendix_command · 39% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | M. E. Khan and D. Nielsen (2018) Fast yet simple natural-gradient descent for variational inference in complex models | 0.956 | 8 | 4 | 88% |
| 2 | D. P. Kingma and M. Welling (2014) Auto-encoding variational bayes | 0.928 | 5 | 3 | 80% |
| 3 | M. Khan, D. Nielsen, V. Tangkaratt, W. Lin, Y. Gal, and A. Srivastava (2018) Fast and scalable bayesian deep learning by weight-perturbation in adam | 0.874 | 12 | 5 | 67% |
| 4 | M. Khan and W. Lin (2017) Conjugate-computation variational inference: Converting variational inference in non-conjugate models to inferences in conjugate… | 0.874 | 6 | 4 | 67% |
| 5 | J. Paisley, D. M. Blei, and M. I. Jordan (2012) Variational bayesian inference with stochastic search | 0.811 | 4 | 2 | 100% |
| 6 | T. Salimans and D. A. Knowles (2014) On using control variates with stochastic approximation for variational bayes and its connection to stochastic linear regression | 0.811 | 4 | 2 | 100% |
| 7 | R. Ranganath, S. Gerrish, and D. M. Blei (2014) Black box variational inference | 0.794 | 10 | 5 | 50% |
| 8 | M.-N. Tran, T.-N. Nguyen, and V.-H. Dao (2021) A practical tutorial on variational bayes | 0.794 | 8 | 6 | 50% |
| 9 | K. Osawa, S. Swaroop, M. E. Khan, A. Jain, R. Eschenhagen, R. E. Tur… (2019) Practical deep learning with bayesian principles | 0.794 | 6 | 4 | 50% |
| 10 | M.-N. Tran, D. H. Nguyen, and D. Nguyen (2021) Variational bayes on manifolds | 0.773 | 13 | 7 | 46% |
Showing the top 10 of 56 scored citations.