Jikai Jin, Yiping Lu, Jose Blanchet, Lexing Ying
arXiv 28 Sep 2022 · Machine Learning · 1 citations (OpenAlex)
arXiv:2209.14430 · PDF · DOI · OpenAlex · Extracted main text
Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional data analysis, causal inference, and multi-agent reinforcement learning. In this paper, we study the statistical limit of learning a Hilbert-Schmidt operator between two infinite-dimensional Sobolev reproducing kernel Hilbert spaces. We establish the information-theoretic lower bound in terms of the Sobolev Hilbert-Schmidt norm and show that a regularization that learns the spectral components below the bias contour and ignores the ones that are above the variance contour can achieve the optimal learning rate. At the same time, the spectral components between the bias and variance contours give us flexibility in designing computationally feasible machine learning algorithms. Based on this observation, we develop a multilevel kernel operator learning algorithm that is optimal when learning linear operators between infinite-dimensional function spaces.
appendix boundary found by appendix_command · 40% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Li, Zhu, Meunier, Dimitri, Mollenhauer, Mattes, Gretton, Arthur (2022) Optimal Rates for Regularized Conditional Mean Embedding Learning | 1.000 | 14 | 4 | 100% |
| 2 | Hoop, Maarten V, Kovachki, Nikola B, Nelsen, Nicholas H, Stuart, And… (2021) Convergence rates for learning linear operators from noisy data | 1.000 | 11 | 3 | 100% |
| 3 | Lu, Yiping, Blanchet, Jose, Ying, Lexing (2022) Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent self | 1.000 | 5 | 4 | 100% |
| 4 | Talwai, Prem, Shameli, Ali, Simchi-Levi, David (2022) Sobolev Norm Learning Rates for Conditional Mean Embeddings | 0.976 | 14 | 5 | 93% |
| 5 | Lu, Lu, Jin, Pengzhan, Karniadakis, George Em (2019) Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of op… | 0.928 | 4 | 3 | 100% |
| 6 | Fischer, Simon, Steinwart, Ingo (2020) Sobolev Norm Learning Rates for Regularized Least-Squares Algorithms. | 0.880 | 22 | 7 | 68% |
| 7 | Li, Zhihan, Fan, Yuwei, Ying, Lexing (2021) Multilevel fine-tuning: Closing generalization gaps in approximation of solution maps under a limited budget for training data self | 0.874 | 5 | 2 | 100% |
| 8 | Lye, Kjetil O, Mishra, Siddhartha, Molinaro, Roberto (2021) A multi-level procedure for enhancing accuracy of machine learning algorithms | 0.874 | 5 | 2 | 100% |
| 9 | Boullé, Nicolas, Kim, Seick, Shi, Tianyi, Townsend, Alex (2022) Learning Green’s functions associated with time-dependent partial differential equations | 0.811 | 4 | 2 | 100% |
| 10 | Giles, Michael B (2008) Multilevel monte carlo path simulation | 0.737 | 3 | 2 | 100% |
Showing the top 10 of 73 scored citations.