David T. Frazier, Ryan Covey, Gael M. Martin, Donald Poskitt
arXiv 10 Aug 2023 · Econometrics · 1 citations (OpenAlex)
arXiv:2308.05263 · PDF · DOI · OpenAlex · Extracted main text
We demonstrate that the forecasting combination puzzle is a consequence of the methodology commonly used to produce forecast combinations. By the combination puzzle, we refer to the empirical finding that predictions formed by combining multiple forecasts in ways that seek to optimize forecast performance often do not out-perform more naive, e.g. equally-weighted, approaches. In particular, we demonstrate that, due to the manner in which such forecasts are typically produced, tests that aim to discriminate between the predictive accuracy of competing combination strategies can have low power, and can lack size control, leading to an outcome that favours the naive approach. We show that this poor performance is due to the behavior of the corresponding test statistic, which has a non-standard asymptotic distribution under the null hypothesis of no inferior predictive accuracy, rather than the {standard normal distribution that is} {typically adopted}. In addition, we demonstrate that the low power of such predictive accuracy tests in the forecast combination setting can be completely avoided if more efficient estimation strategies are used in the production of the combinations, when feasible. We illustrate these findings both in the context of forecasting a functional of interest and in terms of predictive densities. A short empirical example {using daily financial returns} exemplifies how researchers can avoid the puzzle in practical settings.
appendix boundary found by appendix_command · 58% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Geweke, J. and Amisano, G (2011) Optimal prediction pools | 1.000 | 8 | 4 | 100% |
| 2 | Smith, J. and Wallis, K. F (2009) A simple explanation of the forecast combination puzzle | 0.969 | 22 | 6 | 91% |
| 3 | West, K. D (1996) Asymptotic inference about predictive ability | 0.928 | 4 | 3 | 100% |
| 4 | Gneiting, T. and Ranjan, R (2013) Combining predictive distributions | 0.811 | 4 | 2 | 100% |
| 5 | White, H (2000) A reality check for data snooping | 0.737 | 3 | 2 | 100% |
| 6 | Stock, J. H. and Watson, M. W (2004) Combination forecasts of output growth in a seven-country data set | 0.644 | 4 | 1 | 100% |
| 7 | Hall, S. G. and Mitchell, J (2007) Combining density forecasts | 0.644 | 2 | 2 | 100% |
| 8 | Hansen, P. R (2005) A test for superior predictive ability | 0.644 | 2 | 2 | 100% |
| 9 | Martin, G. M., Loaiza-Maya, R., Maneesoonthorn, W., Frazier, D. T.,… (2021) Optimal probabilistic forecasts: When do they work? self | 0.644 | 2 | 2 | 100% |
| 10 | Zischke, R., Martin, G. M., Frazier, D. T., and Poskitt, D. S (2022) On measuring the sampling variability of estimated combinations of distributional forecasts self | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 48 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Flexible global forecast combinations | 0.405 | 1 | 1 |