arXiv 13 Sep 2023 · Econometrics · 1 citations (OpenAlex)
arXiv:2309.06693 · PDF · DOI · OpenAlex · Extracted main text
I study the estimation of semiparametric monotone index models in the scenario where the number of observation points $n$ is extremely large and conventional approaches fail to work due to heavy computational burdens. Motivated by the mini-batch gradient descent algorithm (MBGD) that is widely used as a stochastic optimization tool in the machine learning field, I proposes a novel subsample- and iteration-based estimation procedure. In particular, starting from any initial guess of the true parameter, I progressively update the parameter using a sequence of subsamples randomly drawn from the data set whose sample size is much smaller than $n$. The update is based on the gradient of some well-chosen loss function, where the nonparametric component is replaced with its Nadaraya-Watson kernel estimator based on subsamples. My proposed algorithm essentially generalizes MBGD algorithm to the semiparametric setup. Compared with full-sample-based method, the new method reduces the computational time by roughly $n$ times if the subsample size and the kernel function are chosen properly, so can be easily applied when the sample size $n$ is large. Moreover, I show that if I further conduct averages across the estimators produced during iterations, the difference between the average estimator and full-sample-based estimator will be $1/\sqrt{n}$-trivial. Consequently, the average estimator is $1/\sqrt{n}$-consistent and asymptotically normally distributed. In other words, the new estimator substantially improves the computational speed, while at the same time maintains the estimation accuracy.
appendix boundary found by appendix_titled_section at “Appendix” · 64% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Elhanan Helpman, Marc Melitz, and Yona Rubinstein (2008) Estimating trade flows: Trading partners and trading volumes | 1.000 | 7 | 3 | 100% |
| 2 | Jean-Jacques Forneron (2022) Estimation and inference by stochastic optimization | 1.000 | 5 | 4 | 100% |
| 3 | Hidehiko Ichimura (1993) Semiparametric least squares (sls) and weighted sls estimation of single-index models | 1.000 | 5 | 3 | 100% |
| 4 | Shakeeb Khan, Xiaoying Lan, and Elie Tamer (2023) Estimating high dimensional monotone index models by iterative convex optimization1 | 0.817 | 11 | 5 | 55% |
| 5 | Roger W Klein and Richard H Spady (1993) An efficient semiparametric estimator for binary response models | 0.737 | 3 | 2 | 100% |
| 6 | Boris T Polyak and Anatoli B Juditsky (1992) Acceleration of stochastic approximation by averaging | 0.737 | 3 | 2 | 100% |
| 7 | Léon Bottou, Frank E Curtis, and Jorge Nocedal (2018) Optimization methods for large-scale machine learning | 0.644 | 2 | 2 | 100% |
| 8 | Sebastian Ruder (2016) An overview of gradient descent optimization algorithms | 0.644 | 2 | 2 | 100% |
| 9 | Alekh Agarwal, Sham Kakade, Nikos Karampatziakis, Le Song, and Grego… (2014) Least squares revisited: Scalable approaches for multi-class prediction | 0.405 | 1 | 1 | 100% |
| 10 | Hyungtaik Ahn, Hidehiko Ichimura, James L Powell, and Paul A Ruud (2018) Simple estimators for invertible index models | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 28 scored citations.