Rubén Loaiza-Maya, Michael Stanley Smith, David J. Nott, Peter J. Danaher
arXiv 15 May 2020 · Statistics — Methodology · publishedJournal of Econometrics (2021) · 5 citations (OpenAlex)
arXiv:2005.07430 · PDF · DOI · OpenAlex · Extracted main text
Models with a large number of latent variables are often used to fully utilize the information in big or complex data. However, they can be difficult to estimate using standard approaches, and variational inference methods are a popular alternative. Key to the success of these is the selection of an approximation to the target density that is accurate, tractable and fast to calibrate using optimization methods. Most existing choices can be inaccurate or slow to calibrate when there are many latent variables. Here, we propose a family of tractable variational approximations that are more accurate and faster to calibrate for this case. It combines a parsimonious parametric approximation for the parameter posterior, with the exact conditional posterior of the latent variables. We derive a simplified expression for the re-parameterization gradient of the variational lower bound, which is the main ingredient of efficient optimization algorithms used to implement variational estimation. To do so only requires the ability to generate exactly or approximately from the conditional posterior of the latent variables, rather than to compute its density. We illustrate using two complex contemporary econometric examples. The first is a nonlinear multivariate state space model for U.S. macroeconomic variables. The second is a random coefficients tobit model applied to two million sales by 20,000 individuals in a large consumer panel from a marketing study. In both cases, we show that our approximating family is considerably more accurate than mean field or structured Gaussian approximations, and faster than Markov chain Monte Carlo. Last, we show how to implement data sub-sampling in variational inference for our approximation, which can lead to a further reduction in computation time. MATLAB code implementing the method for our examples is included in supplementary material.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Danaher, P. J., Danaher, T. S., Smith, M. S., and Loaiza-Maya, R (2020) Advertising effectiveness for multiple retailer-brands in a multimedia and multichannel environment self | 1.000 | 12 | 3 | 100% |
| 2 | Huber, F., Koop, G., and Onorante, L (2020) Inducing sparsity and shrinkage in time-varying parameter models | 1.000 | 9 | 3 | 100% |
| 3 | Ong, V. M.-H., Nott, D. J., and Smith, M. S (2018) Gaussian variational approximation with a factor covariance structure self | 1.000 | 6 | 4 | 100% |
| 4 | Gunawan, D., Tran, M.-N., and Kohn, R (2017) Fast inference for intractable likelihood problems using variational Bayes | 0.874 | 5 | 2 | 100% |
| 5 | Archer, E., Park, I. M., Buesing, L., Cunningham, J., and Paninski, L (2015) Black box variational inference for state space models | 0.843 | 3 | 3 | 100% |
| 6 | Smith, M. S., Loaiza-Maya, R., and Nott, D. J (2020) High-dimensional copula variational approximation through transformation self | 0.811 | 4 | 2 | 100% |
| 7 | Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J (2013) Stochastic variational inference | 0.811 | 4 | 2 | 100% |
| 8 | Kingma, D. P. and Welling, M (2014) Auto-encoding variational Bayes | 0.644 | 2 | 2 | 100% |
| 9 | Allenby, G. M. and Rossi, P. E (1998) Marketing models of consumer heterogeneity | 0.644 | 2 | 2 | 100% |
| 10 | Bottou, L (2010) Large-scale machine learning with stochastic gradient descent | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 59 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.