arXiv 16 Oct 2019 · Statistics — Methodology · 14 citations (OpenAlex)
arXiv:1910.07325 · PDF · DOI · OpenAlex · Extracted main text
In recent years, probabilistic forecasting is an emerging topic, which is why there is a growing need of suitable methods for the evaluation of multivariate predictions. We analyze the sensitivity of the most common scoring rules, especially regarding quality of the forecasted dependency structures. Additionally, we propose scoring rules based on the copula, which uniquely describes the dependency structure for every probability distribution with continuous marginal distributions. Efficient estimation of the considered scoring rules and evaluation methods such as the Diebold-Mariano test are discussed. In detailed simulation studies, we compare the performance of the renowned scoring rules and the ones we propose. Besides extended synthetic studies based on recently published results we also consider a real data example. We find that the energy score, which is probably the most widely used multivariate scoring rule, performs comparably well in detecting forecast errors, also regarding dependencies. This contradicts other studies. The results also show that a proposed copula score provides very strong distinction between models with correct and incorrect dependency structure. We close with a comprehensive discussion on the proposed methodology.
appendix boundary found by appendix_titled_section at “Appendix” · 99% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Pinson, P. and Tastu, J (2013) Discrimination ability of the energy score | 1.000 | 14 | 6 | 100% |
| 2 | Scheuerer, M. and Hamill, T. M (2015) Variogram-based proper scoring rules for probabilistic forecasts of multivariate quantities | 1.000 | 6 | 3 | 100% |
| 3 | Székely, G. J. and Rizzo, M. L (2013) Energy statistics: A class of statistics based on distances | 0.928 | 4 | 3 | 100% |
| 4 | Gneiting, T. and Raftery, A. E (2007) Strictly proper scoring rules, prediction, and estimation | 0.874 | 9 | 2 | 100% |
| 5 | Hong, T., Pinson, P., Fan, S., Zareipour, H., Troccoli, A., and Hynd… (2016) Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond | 0.843 | 3 | 3 | 100% |
| 6 | Möller, A., Lenkoski, A., and Thorarinsdottir, T. L (2013) Multivariate probabilistic forecasting using ensemble bayesian model averaging and copulas | 0.843 | 3 | 3 | 100% |
| 7 | Lerch, S., Thorarinsdottir, T. L., Ravazzolo, F., Gneiting, T., et al (2017) Forecaster’s dilemma: Extreme events and forecast evaluation | 0.811 | 4 | 2 | 100% |
| 8 | Diebold, F. X (2015) Comparing predictive accuracy, twenty years later: A personal perspective on the use and abuse of diebold–mariano tests | 0.737 | 3 | 2 | 100% |
| 9 | Gneiting, T., Stanberry, L. I., Grimit, E. P., Held, L., and Johnson… (2008) Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds | 0.644 | 2 | 2 | 100% |
| 10 | Junk, C., von Bremen, L., Kühn, M., Späth, S., and Heinemann, D (2014) Comparison of postprocessing methods for the calibration of 100-m wind ensemble forecasts at off-and onshore sites | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 36 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.