Jialuo Chen, Zhaoxing Gao, Ruey S. Tsay
arXiv 7 Jul 2025 · Statistics — Methodology
arXiv:2507.04668 · PDF · DOI · OpenAlex · Extracted main text
We investigate forward variable selection for ultra-high dimensional linear regression using a Gram-Schmidt orthogonalization procedure. Unlike the commonly used Forward Regression (FR) method, which computes regression residuals using an increasing number of selected features, or the Orthogonal Greedy Algorithm (OGA), which selects variables based on their marginal correlations with the residuals, our proposed Gram-Schmidt Forward Regression (GSFR) simplifies the selection process by evaluating marginal correlations between the residuals and the orthogonalized new variables. Moreover, we introduce a new model size selection criterion that determines the number of selected variables by detecting the most significant change in their unique contributions, effectively filtering out redundant predictors along the selection path. While GSFR is theoretically equivalent to FR except for the stopping rule, our refinement and the newly proposed stopping rule significantly improve computational efficiency. In ultra-high dimensional settings, where the dimensionality far exceeds the sample size and predictors exhibit strong correlations, we establish that GSFR achieves a convergence rate comparable to OGA and ensures variable selection consistency under mild conditions. We demonstrate the proposed method {using} simulations and real data examples. Extensive numerical studies show that GSFR outperforms commonly used methods in ultra-high dimensional variable selection.
appendix boundary found by appendix_titled_section at “Supplementary Material” · 99% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Wang, Hansheng (2009) Forward regression for ultra-high dimensional variable screening | 1.000 | 6 | 4 | 100% |
| 2 | Ing, Ching-Kang and Lai, Tze Leung (2011) A Stepwise Regression Method and Consistent Model Selection for High-Dimensional Sparse Linear Models | 1.000 | 6 | 3 | 100% |
| 3 | Borodin, P. A. and Konyagin, S. V (2021) Projection Greedy Algorithm | 0.644 | 2 | 2 | 100% |
| 4 | Ing, Ching-Kang (2020) Model selection for high-dimensional linear regression with dependent observations | 0.644 | 2 | 2 | 100% |
| 5 | Gao, Zhaoxing and Tsay, Ruey S (2025) Supervised dynamic pca: Linear dynamic forecasting with many predictors self | 0.511 | 2 | 1 | 100% |
| 6 | McCracken, Michael W and Ng, Serena (2016) FRED-MD: A monthly database for macroeconomic research | 0.511 | 2 | 1 | 100% |
| 7 | Stock, James H and Watson, Mark W (2002) Macroeconomic forecasting using diffusion indexes | 0.511 | 2 | 1 | 100% |
| 8 | Peter J. Bickel and Elizaveta Levina (2008) Regularized estimation of large covariance matrices | 0.405 | 1 | 1 | 100% |
| 9 | Shuo-Chieh Huang and Ruey S. Tsay (2024) Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Data self | 0.405 | 1 | 1 | 100% |
| 10 | Peter Bühlmann (2006) Boosting for high-dimensional linear models | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 23 scored citations.