Ali Habibnia, Esfandiar Maasoumi
arXiv 25 Apr 2019 · Econometrics · publishedJournal of Quantitative Economics (2021) · 1 citations (OpenAlex)
arXiv:1904.11145 · PDF · DOI · OpenAlex · Extracted main text
This paper considers improved forecasting in possibly nonlinear dynamic settings, with high-dimension predictors ("big data" environments). To overcome the curse of dimensionality and manage data and model complexity, we examine shrinkage estimation of a back-propagation algorithm of a deep neural net with skip-layer connections. We expressly include both linear and nonlinear components. This is a high-dimensional learning approach including both sparsity L1 and smoothness L2 penalties, allowing high-dimensionality and nonlinearity to be accommodated in one step. This approach selects significant predictors as well as the topology of the neural network. We estimate optimal values of shrinkage hyperparameters by incorporating a gradient-based optimization technique resulting in robust predictions with improved reproducibility. The latter has been an issue in some approaches. This is statistically interpretable and unravels some network structure, commonly left to a black box. An additional advantage is that the nonlinear part tends to get pruned if the underlying process is linear. In an application to forecasting equity returns, the proposed approach captures nonlinear dynamics between equities to enhance forecast performance. It offers an appreciable improvement over current univariate and multivariate models by RMSE and actual portfolio performance.
appendix boundary found by appendix_titled_section at “Appendix: Automatic Differentiation” · 88% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Larsen, J., Hansen, L., Svarer, C., and Ohlsson, M (1996) Design and regularization of neural networks: The optimal use of a validation set | 0.511 | 2 | 1 | 100% |
| 2 | Ng, A. Y (2004) Feature selection, l1 vs. l2 regularization, and rotational invariance | 0.511 | 2 | 1 | 100% |
| 3 | Goodfellow, I., Bengio, Y., and Courville, A (2016) Deep Learning | 0.405 | 1 | 1 | 100% |
| 4 | Andersen, L. N., Larsen, J., Hansen, L. K., and Hintz-Madsen, M (1997) Adaptive regularization of neural classifiers | 0.405 | 1 | 1 | 100% |
| 5 | Arlot, S. and Celisse, A (2010) A survey of cross-validation procedures for model selection | 0.405 | 1 | 1 | 100% |
| 6 | Armstrong, J. and Collopy, F (1992) Error measures for generalizing about forecasting methods: Empirical comparisons | 0.405 | 1 | 1 | 100% |
| 7 | Baydin, A. G., Pearlmutter, B. A., Radul, A. A., and Siskind, J. M (2017) Automatic differentiation in machine learning: a survey | 0.405 | 1 | 1 | 100% |
| 8 | Bengio, Y (2000) Gradient-based optimization of hyperparameters | 0.405 | 1 | 1 | 100% |
| 9 | Bergstra, J., Bardenet, R., Bengio, Y., and Kégl, B (2011) Algorithms for hyper-parameter optimization | 0.405 | 1 | 1 | 100% |
| 10 | Bergstra, J. S., Bardenet, R., Bengio, Y., and Kégl, B (2011) Algorithms for hyper-parameter optimization | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 56 scored citations.