Simultaneous Inference for Local Structural Parameters with Random Forests

titlepage\begin{spacing}{1} \begin{abstract} \smalltonormalsize{We construct simultaneous confidence intervals for solutions to conditional moment equations. The intervals are built around a class of nonparametric regression algorithms based on subsampled kernels. This class encompasses various forms of subsampled random forest regression, including Generalized Random Forests athey2019generalized. Although simultaneous validity is often desirable in practice---for example, for fine-grained characterization of treatment effect heterogeneity---only confidence intervals that confer pointwise guarantees were previously available. Our work closes this gap. As a by-product, we obtain several new order-explicit results on the concentration and normal approximation of high-dimensional $U$-statistics.} \\ \\ Keywords: Random Forest Regression, Half-Sample Bootstrap, $U$-statistics \\ JEL: C01, C14, C12 \end{abstract} \end{spacing}

spacing{1.3} \section{Introduction} Consider an independent and identically distributed sample $\mathbf{D}_n = (D_i)_{i=1}^n$. The scalar parameter $\theta_0(x)$ is defined as the unique solution to the conditional moment equation \begin{equation} M(x; \theta, g_0) = \mathbb{E}\left[m(D_i;\theta,g_0) \mid X_i = x\right] = 0 \end{equation} in $\theta$. Here, $X_i$ is a sub-vector of the observation $D_i$, $g_0$ is an unknown nuisance parameter, identified via an auxiliary statistical problem, and $m(\cdot;\theta,g)$ is a known moment function. We propose a method for constructing simultaneous confidence intervals for the parameter vector $\theta_0(\bm{x}^{(d)})=(\theta_0(x^{(j)}))_{j=1}^d$, where $\bm{x}^{(d)}=(x^{(j)})_{j=1}^d$ is a specified $d$-vector in the domain of $X_i$.\footnote{The leading examples for choices of the query-vector $\bm{x}^{(d)}$ are cases where $\bm{x}^{(d)}$ is taken to be the observed values of the covariates $X_1,\ldots,X_n$ or where $\bm{x}^{(d)}$ gives a fine grid over the domain of $X_i$.} Many problems encountered in applied economics and causal inference can be formulated as instances of this setting, including nonparametric regression, quantile regression, and estimation of conditional average treatment effects. To fix concepts, consider banerjee2015multifaceted, who study the effects of a poverty alleviation program implemented in Ghana.\footnote{banerjee2015multifaceted study data collected from several similar graduation programs. We focus on the data from their evaluation of the program implemented in Ghana. (ref) gives further details.} For each individual in their sample, they observe the data $D_i = (Y_i, W_i, Z_i)$, where $Y_i$ is a measurement of total assets taken two years after the implementation of the program, $W_i$ is an indicator denoting assignment to the program, and $Z_i$ is a vector of covariates. A broad aim of the study is to determine the conditions under which recipients of aid experience lasting improvements in welfare. One quantity that can inform this determination is the conditional average treatment effect (CATE) \begin{equation} \theta_0(x) = \mathbb{E}_P \left[ Y_i(1) - Y_i(0) \mid X_i = x\right] , \end{equation} where $Y_i(1)$ and $Y_i(0)$ are the potential outcomes generated by the intervention $W_i$ and $X_i$ is some chosen subvector of $Z_i$. Many modern approaches to estimating CATEs are premised on the observation that $\theta_0(x)$ is the solution to the conditional moment equation \begin{equation} M(x; \theta, g_0) = \mathbb{E}\left[(\mu_1(Z_i) - \mu_0(Z_i)) + \beta(W_i, Z_i) (Y_i - \mu_{W_i}(Z_i)) - \theta \mid X_i = x\right]=0 , \end{equation} where the nuisance parameter $g_0$ collects the conditional outcome regression and Horvitz-Thompson weight \begin{equation} \mu_{w}(z) = \mathbb{E}_P\left[Y_i \mid W_i = w, Z_i = z\right]\quadand\quad \beta(w, z) = \frac{w}{\pi(z)} - \frac{1- w}{1-\pi(z)} , \end{equation} for the propensity score $\pi(z) = P\{W_i = 1 \mid Z_i = z\}$. See semenova2021debiased, foster2023orthogonal, and kennedy2023towards, for further discussion. Often, estimates of solutions to conditional moment equations of the form (ref) or (ref) are obtained by solving the empirical conditional moment equation \begin{equation} M_{n}(x; \theta,\hat{g}_n, \mathbf{D}_n) = \sum_{i=1}^n K(x, X_i) m(D_i; \theta, \hat{g}_n) = 0 \end{equation} in $\theta$, where $\hat{g}_n$ is some first-stage estimator of the nuisance parameter $g_0$ and $K(x,x^\prime)$ is some, potentially random and data-dependent, kernel function measuring the distance between $x$ and $x^\prime$. Popular examples of data-dependent kernel estimators include subsampled nearest-neighbor regression fix1989discriminatory,khosravi2019non,demirkaya2024optimal and random forest regression breiman2001random,wager2018estimation. Solutions to conditional moment equations (ref) constructed with subsampled random forest regression are referred to as Orthogonal Random Forests (ORF) oprescu2019orthogonal. ORFs generalize Generalized Random Forests (GRF) athey2019generalized to incorporate nuisance parameters estimated via auxiliary statistical problems. (ref) displays CATE estimates for the experiment studied in banerjee2015multifaceted, where the chosen conditioning covariates $X_i$ are pretreatment measurements of monthly consumption and total assets. The nuisance parameter estimate $\hat{g}_n$ and the kernel $K(x,x^\prime)$ are constructed with the implementation of random forest regression made available through the “GRF” R package athey2019generalized. The graduation program appears to be most effective for individuals with a high level of baseline consumption and a low level of baseline assets.\footnote{The quartiles of baseline log consumption are 3.33, 3.76, and 4.20. The quartiles of baseline assets are -0.45, -0.71, and 0.03. Panel A of (ref) displays a scatter plot of baseline log consumption and assets.} These results are suggestive of a poverty trap: individuals with an opportunity to increase their assets are able to do so only if they have a high level of baseline consumption balboni2022people,kraay2014poverty. \begin{figure}[t] \begin{centering} \caption{CATE Estimates} \begin{tabular}{c} \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays a heat map giving CATE estimates for the intervention studied in banerjee2015multifaceted on post-treatment assets. The color of each rectangle indicates the estimate of the CATE queried at the rectangle's central point. The horizontal and vertical axes display the baseline monthly consumption, normalized to dollars and measured in logs base 10, and the baseline value of an index for total assets. CATE estimates are obtained by solving the empirical moment equation (ref) for each value $x$ on an evenly spaced grid on both axes. See (ref) for further details.} \end{figure} We contribute a method for assessing the statistical significance of estimates typified by (ref). In particular, we propose a computationally simple procedure for constructing simultaneous upper and lower confidence bounds for solutions to conditional moment equations (ref) centered around estimators of the form (ref). Formally, we construct a family of random intervals \begin{equation} \hat{\mathcal{C}}(\bm{x}^{(d)}) = \left\{\hat{\mathcal{C}}(x^{(j)}) = [\hat{c}_{L}(x^{(j)}), \hat{c}_{U}(x^{(j)})] \right\}_{j=1}^d , \end{equation} on the basis of the observed data, such that \begin{equation} \sup_{P\in\mathbf{P}} \big\vert P\left\{ \theta_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}) \right\} - (1-\alpha) \big\vert \leq r_{n,d} \end{equation} for some sequence $r_{n,d}$, where $\mathbf{P}$ is some statistical family that contains the distribution $P$ of the data $D_i$. We say that a region (ref) satisfying (ref) is uniformly asymptotically valid at the rate $r_{n,d}$. Here, uniformity operates over both the $d$-dimensional query-vector $\bm{x}^{(d)}$ and the statistical family $\mathbf{P}$. The main theoretical contribution of this paper is the construction of confidence regions whose error rate $r_{n,d}$ converges to zero in asymptotic regimes where the number of points $d$ in the query-vector $\bm{x}^{(d)}$ may increase much more quickly than the sample size $n$. Many leading applications of conditional moment regression in applied economics necessitate simultaneous coverage guarantees. For example, researchers are often interested in characterizing statistically significant dimensions of treatment effect heterogeneity or in other forms of post-hoc exploratory statistical analysis. By contrast, only confidence intervals with pointwise guarantees, i.e., coverage guarantees that are valid at a single pre-specified point $x$, are available. See, for example, methods developed by sexton2009standard,wager2014confidence,mentch2016quantifying,wager2018estimation,athey2019generalized,oprescu2019orthogonal,khosravi2019non, and demirkaya2024optimal. Moreover, prior approaches for simultaneous confidence intervals, that have the potential to be relevant to random forest methods song2019approximating, are not applicable, as they require assumptions that do not hold in hyperparameter regimes that ensure consistency of these methods. Our work bridges this gap between theory and practice and enables the use of random forest approaches, when simultaneous coverage guarantees are required. We begin, in (ref), by defining the proposed confidence region and illustrating its application to the banerjee2015multifaceted experiment. Our construction can be seen as an instance of subsampling politis2012subsampling,politis1994large, although our formal analysis is more directly connected to the exchangeably weighted bootstrap praestgaard1993exchangeably,chernozhuokov2022improved, and, in particular, the half-sample bootstrap chung2013exact,yadlowsky2023evaluating. In (ref), we bound the accuracy of the proposed confidence region, under two substantive restrictions. First, we require that the estimator (ref) be based on a kernel constructed with subsampling. Second, we require that the moment function (ref) satisfy a Neyman orthogonality condition, in the sense of chernozhukov2018double. This restriction mitigates the error induced by estimation of nuisance parameters. Our analysis builds on oprescu2019orthogonal, who study pointwise inference for subsampled random forest estimators of solutions to Neyman orthogonal conditional moment equations. See also khosravi2019non for a pointwise analysis of generic subsampled kernel regression estimators. We conclude (ref) by documenting that the proposed confidence region is accurate and informative at empirically relevant sample sizes with a simulation calibrated to the banerjee2015multifaceted data. As a by-product of this analysis, we give several new results on the concentration and normal approximation of large order, high-dimensional, $U$-statistics. In particular, we give a concentration inequality and central limit theorem for high-dimensional $U$-statistics with explicit order-dependence. These bounds are applicable to non-degenerate $U$-statistics whose order $b$ satisfies $b = o(n)$, up to a dimension dependent logarithmic factor. This generality represents a substantial improvement over existing results song2019approximating,minsker2023u, that apply to the regime $b = o(n^{1/3})$, and is essential for our application.\footnote{In particular, the subsampled kernel regression estimators considered in this paper require $n^{1 - c/p} \lesssim b$ for consistency, where $p$ is the dimension of $Z_i$.} Our results hinge on a new concentration inequality for the difference between a $U$-statistic and its H\'{a}jek projection hajek1968asymptotic, enabled by a symmetrization inequality due to sherman1994maximal. We collect these results in (ref). (ref) concludes. \subsection{Related Literature} There is an extensive literature on estimation of solutions to conditional moment equations. See, for example, Newey1993efficient, ai2003efficient, chen2012estimation, and chernozhukov2023constrained. chen2018optimal and christensen2024adaptive propose related approaches for constructing uniform confidence bands for parameters identified by conditional moments, emphasizing achieving minimax rates in H\"{o}lder classes by building confidence regions around carefully constructed sieve estimators with Lepski's method chernozhukov2014anti. By contrast, our aim is to provide a simple procedure for constructing simultaneous confidence intervals around estimators whose precise structure may be unknown to the user. We contribute to a large literature on the role of Neyman orthogonality in estimation of solutions to moment equations with nuisance parameters. chernozhukov2018double, chernozhukov2022locally, and ichimura2022influence provide extensive discussion and guidance on the derivation of orthogonal moments. semenova2021debiased, foster2023orthogonal, and kennedy2023towards apply aspects of this analysis to conditional moment estimation. Our results are comparable to the generic bounds on the accuracy of Gaussian multiplier bootstrap confidence regions for nonparametric regression and $Z$-estimation given in chernozhukov2014anti and belloni2018uniformly, respectively. We generalize these results, in the sense that we treat inference for conditional $Z$-estimators whose score function is potentially unknown to the researcher. Our formal analysis builds on a groundbreaking sequence of papers on central limit theorems for maxima of sums initiated by chernozhukov2013gaussian. Extensions and refinements of these results are given in chernozhukov2017central and chernozhuokov2022improved. The asymptotic analysis of $U$-statistics has a long and involved history. We provide a more detailed literature review in (ref). Finally, we contribute to a large literature on the statistical analysis of random forest regression. A wide variety of consistency results are given in, e.g., buhlmann2002analyzing, lin2006random, biau2008consistency, mentch2014ensemble, scornet2015consistency, and cattaneo2024pointwise. High dimensional consistency results are given in syrgkanis2020estimation, chi2022asymptotic, and huo2023adaptation. \subsection{Notation}The quantities $c$ and $C$ denote universal positive constants, whose values are allowed to depend only on the family of distributions $\mathbf{P}$. For two real-valued functions $f$ and $g$ on a domain $\mathcal{X}$, we say $g(x) \lesssim f(x)$ if $g(x)\leq C f(x)$ for each $x$ in $\mathcal{X}$. For a functional $F$ on $\mathcal{F}$, we use the notation \[ \partial_f F(f)[h] = \frac{\text{d}}{\text{d}t}F(f+th)\big\vert_{t=0} \quad\text{and}\quad \partial_{f,f} F(f)[h]=\frac{\text{d}^2}{\text{d}t^2}F(f+th)\big\vert_{t=0} \] to denote first and second order directional derivatives, respectively. For any function $f(x)$ and vector $\bm{x}^{(d)}=(x^{(j)})_{j=1}^d$, we let $f(\bm{x}^{(d)})$ denote the vector $(f(x^{(1)}), \cdots, f(x^{(d)}))$. The set $\mathcal{S}_{n,b}$ collects all of the subsets of $[n]=\{1,\ldots,n\}$ of size $b$ and the quantity $D_{\mathsf{s}}$ denotes the subset of the observed data $\mathbf{D}_n$ with indices in the set $\mathsf{s}$. \section{Simultaneous Confidence Intervals} We build simultaneous confidence intervals around solutions to the empirical moment equation \begin{equation} M_{n}(x; \theta,\hat{g}_n, \mathbf{D}_n) = \sum_{i=1}^n K(x, X_i) m(D_i; \theta, \hat{g}_n) = 0 \end{equation} in the variable $\theta$, evaluated at each $x^{(j)}$ in $\bm{x}^{(d)}$.\footnote{For the sake of simplicity, we consider only the case that the moment $m(\cdot ;\theta,g_0)$ and the parameter $\theta$ are scalar-valued. The vector-valued case is relevant, e.g., when estimating CATEs with multiple treatment variables. Our results generalize to the vector-valued case at the cost of additional notation.} Let $\hat{\theta}_n(\bm{x}^{(d)})$ denote the vector of solutions to (ref) evaluated at each $x^{(j)}$ in $\bm{x}^{(d)}$. Let $\mathsf{h}$ denote a random element of $\mathcal{S}_{n,n/2}$, i.e., a random half-sample of $[n]$ and let $\hat{\theta}_\mathsf{h}(\bm{x}^{(d)})$ denote the vector of the solutions to (ref), with the data $\mathbf{D}_n$ replaced by the half-sample $D_\mathsf{h}$. Our proposal is premised on approximating the distribution of the root \begin{equation} R_n(\bm{x}^{(d)}) = \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \end{equation} with the conditional distribution of the half-sample bootstrap root \begin{equation} R^*_n(\bm{x}^{(d)}) = \hat{\theta}_\mathsf{h}(\bm{x}^{(d)}) - \hat{\theta}_n(\bm{x}^{(d)}) , \end{equation} The nuisance parameter estimator $\hat{g}_n$ does not need to be re-estimated when computing (ref). A version of the half-sample bootstrap is implemented by default in the GRF R package athey2019generalized.\footnote{The version of the half-sample bootstrap considered in athey2019generalized is based on combining estimates of the variances of components of a linearization of the moment $M(x;\theta,g)$ with a Delta method type argument. By contrast, the bootstrap root (ref) is agnostic to the structure of the conditional moment under consideration.} The half-sample bootstrap is an instance of subsampling politis1994large,politis2012subsampling. Let $\hat{\lambda}^2_{n,j}$ denote the variance of $\sqrt{n}R^*_{n}(x^{(j)})$, conditioned on the data $\mathbf{D}_n$, and let $\hat{\mathsf{cv}}_{n}(\alpha)$ denote the $1-\alpha$ quantile of the distribution of the studentized statistic \begin{equation} \hat{S}^*_{n}(\bm{x}^{(d)}) = \sqrt{n} \left\| \hat{\Lambda}_n^{-1/2}R^*_n(\bm{x}^{(d)}) \right\|_\infty , \end{equation} again conditioned on the data $\mathbf{D}_n$. Here, $\hat{\Lambda}_n$ denotes the diagonal matrix with elements $\hat{\lambda}^2_{n,j}$.\footnote{The quantities $\hat{\lambda}^2_{n,j}$ and $\mathsf{cv}_{n}(\alpha)$ are easily approximated by resampling the bootstrap root (ref). To simplify exposition, we omit explicit consideration of residual randomness induced by this approximation.} We consider confidence intervals with the following structure. \begin{defn} Define the intervals \begin{equation} \hat{\mathcal{C}}(x^{(j)}) = \hat{\theta}_n(x^{(j)}) \pm n^{-1/2}\hat{\lambda}_{n,j} \hat{\mathsf{cv}}_{n}(\alpha)\quadfor each\quad j in [d] . \end{equation} The level-$\alpha$ half-sample confidence region for $\theta_0(\bm{x}^{(d)})$ is given by $\hat{\mathcal{C}}(\bm{x}^{(d)})$. \end{defn} Confidence intervals with the same structure, based on different choices of bootstrap root, are studied in, e.g., chernozhukov2014anti and belloni2018uniformly. The essential feature of the bootstrap root (ref) is that it can be computed without knowing anything about the structure of the estimator $\hat{\theta}_n(\bm{x}^{(d)})$. In particular, approaches based on the Rademacher or Gaussian multiplier bootstrap rely on knowledge of a linear approximation to $\hat{\theta}_n(\bm{x}^{(d)})$. To gain intuition, suppose that the estimator $\hat{\theta}_n(\bm{x}^{(d)})$ satisfies a linear representation \begin{equation} \hat{\theta}_n(\bm{x}^{(d)}) = \frac{1}{n} \sum_{i=1}^n \bar{u}(\bm{x}^{(d)}, D_i) \end{equation} for some function $\bar{u}(\cdot,\cdot)$. Let $V_1,\ldots,V_n$ denote a collection of random variables, where $V_i$ takes the value $1$ if the index $i$ is an element of the random set $\mathsf{h}$ used to define the half-sample bootstrap root (ref), and takes the value $-1$ otherwise. Observe that \begin{flalign} R^*_n(\bm{x}^{(d)}) = \hat{\theta}_\mathsf{h}(\bm{x}^{(d)}) - \hat{\theta}_n(\bm{x}^{(d)}) &= \frac{2}{n} \sum_{i=1}^n \mathbb{I}\{ i \in \mathsf{h}\} \bar{u}(\bm{x}^{(d)}, D_i) - \frac{1}{n} \sum_{i=1}^n \bar{u}(\bm{x}^{(d)}, D_i)\nonumber\\ &= \frac{1}{n}\sum_{i=1}^n V_i \left(\bar{u}(\bm{x}^{(d)}, D_i) - \theta_0(\bm{x}^{(d)})\right) . \end{flalign} The weights $V_i$ are exchangeable Rademacher random variables, i.e., they are uniformly distributed on $\{1,-1\}$. If the weights were fully independent, the representation (ref) reduces to the more familiar Rademacher bootstrap.\footnote{The representation (ref) is due to yadlowsky2023evaluating, who draw on a similar observation made in the context of two-sample testing in chung2013exact. In (ref), we consider a variant of the bootstrap root (ref) based on re-estimating and re-scaling $\hat{\theta}_n(\bm{x}^{(d)})$ on a subsample of a random size $\mathsf{Bin}(n,1/2)$. For this construction, the equivalent objects to the weights $V_i$ are fully independent. That is, this bootstrap root is equivalent to the Rademacher bootstrap root for linear statistics. We show that the resulting confidence regions obtain the same error rates on coverage accuracy.} In practice, many widely applied estimators are not perfectly linearly decomposable. Often, however, estimators do satisfy an approximate linear decomposition, in the sense that the equality (ref) holds with a remainder term of order, say, $o_p(n^{-\gamma})$ for some positive constant $\gamma$. As we will see, estimators constructed with subsampled kernels are approximately linear around some unknown function $\bar{u}(\cdot,\cdot)$. If an estimator $\hat{\theta}_n(\bm{x}^{(d)})$ is approximately linear, then the subsampled estimate $\hat{\theta}_{\mathsf{h}}(\bm{x}^{(d)})$ is immediately also approximately linear.\footnote{Approximate linearity does not immediately imply a representation analogous to (ref) for a root constructed with an empirical bootstrap, as approximate linearity would not necessarily hold under the empirical distribution.} Thus, for approximately linear statistics, the representation (ref) continues to hold, now with a remainder term of order $o_p(n^{-\gamma})$. As a consequence, the validity of the confidence region formulated in (ref), for approximately linear estimators, follows by deriving central limit theorems for the statistics (ref) and (ref) and applying an appropriate generalization of Slutsky's Theorem. \subsection{Application} We now return to the data studied in banerjee2015multifaceted. (ref) displays upper and lower half-sample confidence bounds for the CATE (ref) on post-treatment assets. The null hypothesis that the CATE is equal to zero is only rejected for individuals with low baseline assets. The lower bounds are meaningfully larger than zero only for individuals with low baseline assets and high baseline consumption. That is, the graduation program has a positive impact on individuals who do not have many assets to begin with, but who do have access to a stable source of consumption. On the other hand, the confidence regions contain zero for individuals who have either low baseline consumption or high baseline assets. \begin{figure}[t] \begin{centering} \caption{Half-Sample Confidence Region} \begin{tabular}{c} Panel A: Upper Bound\tabularnewline \tabularnewline Panel B: Lower Bound\tabularnewline \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays heat maps giving half-sample upper and lower confidence bounds for the CATE of the intervention studied in banerjee2015multifaceted on post-treatment total assets. The confidence bounds are constructed at level $\alpha = 0.1$. The upper and lower bounds are displayed with different color palettes to emphasize the use of different scales. A contour line has been superimposed over the lower bound to demarcate where the bound crosses zero. The axes and estimator are the same as in (ref).} \end{figure} It is illustrative to contrast the estimates and confidence bounds displayed in (ref) with a more frequently encountered method for assessing treatment effect heterogeneity---interacted linear regression. (ref) reports estimates and standard errors associated with several linear regression specifications, constructed with the same data. The first column reports the coefficient from a regression of post-treatment assets on a treatment indicator. Consistent with results reported in banerjee2015multifaceted, the average effect of the program is positive and statistically significant. The second and third specifications interact the treatment indicator with pre-treatment assets and consumption, respectively. In both cases, the estimate of the coefficient on the interaction is statistically insignificant. The fourth specification interacts both pre-treatment assets and consumption with a treatment indicator. Here, all coefficients lose statistical significance. The implicit view of much of applied economics appears to be that the flexibility afforded by nonparametric methods is not worth sacrificing the statistical precision of more parsimonious, linear, alternatives.\footnote{We conduct a small survey of papers published in the \textit{American Economic Review} in the first six months of 2023. Of 38 empirical papers, 30 assess treatment effect heterogeneity in some way. As best as we can tell, only two papers display nonparametric estimates of effect heterogeneity. By contrast, 10 display the results of an interacted linear regression typified by (ref). The rest are either structural papers, or only display coefficients on interactions of binary covariates. See (ref) for further details.} The exercise here suggests otherwise. The linearity imposed by interacted linear regression masks the structure in the effect heterogeneity recovered by random forest regression. The resulting bias is so substantial that statistical significance is lost. The half-sample confidence regions developed in this paper enable the recovery of statistically significant measurements of effect heterogeneity. The remainder of the paper is devoted to developing theoretical guarantees on the accuracy of confidence bounds typified by (ref). \begin{table}[t] \begin{centering} \caption{Interacted Linear Regression} { \begin{tabular}{lcccc} \toprule & \multicolumn{4}{c}{Dependent Variable: Post-Treatment Assets}\tabularnewline \midrule & (1) & (2) & (3) & (4)\tabularnewline \midrule \midrule \multirow{1}{*}{Treatment} & 0.218 (0.031) & 0.194 (0.032) & 0.102 (0.172) & 0.111 (0.201)\tabularnewline \multirow{1}{*}{Assets} & & 0.770 (0.051) & & 1.229 (0.288)\tabularnewline \multirow{1}{*}{Consumption} & & & 0.06 (0.022) & -0.026 (0.026)\tabularnewline \multirow{1}{*}{Assets $\times$ Consumption} & & & & -0.119 (0.076)\tabularnewline \multirow{1}{*}{Treatment $\times$ Assets} & & 0.047 (0.093) & & -0.065 (0.552)\tabularnewline \multirow{1}{*}{Treatment $\times$ Consumption} & & & 0.031 (0.045) & 0.021 (0.050)\tabularnewline \multirow{1}{*}{Treatment $\times$ Assets $\times$ Consumption} & & & & 0.027 (0.141)\tabularnewline \midrule \midrule Observations: & 2,438 & 2,438 & 2,438 & 2,438\tabularnewline \bottomrule \end{tabular}} \end{centering} \justifying {Notes: (ref) reports estimates of the coefficients of four linear regression specifications, constructed with the banerjee2015multifaceted data. Robust standard errors are displayed in parentheses.} \end{table} \section{Coverage Accuracy} We establish a bound on the accuracy of the nominal coverage probability for the confidence region introduced in (ref). We begin in (ref) by discussing subsampled kernel regression and introducing several quantities that take a prominent role in our analysis. Our results apply to conditional moments that satisfy a Neyman orthogonality condition, in addition to several simple regularity conditions. We overview these restrictions in (ref). Our main result is stated in (ref). The results of a simulation calibrated to the banerjee2015multifaceted data are reported in (ref). Throughout, we assume that the data $Z_i$ and $X_i$ take values in the spaces $\mathcal{Z}$ and $\mathcal{X}$, respectively. \subsection{Subsampled Kernel Regression} Subsampled kernel regression is a broad class of algorithms for solving regression problems of the form (ref), based on constructing a data-driven kernel function $K(x,x^\prime)$ with subsampling. Formally, fix some positive integer $r$ and let $(\mathsf{s}_q)_{q=1}^r$ collect a sequence of subsets of $[n]$ drawn independently and uniformly from $\mathcal{S}_{n,b}$. Let $\xi$ denote some auxiliary source of randomness and let $(\xi_{\mathsf{s}_q})_{q=1}^r$ collect a set of independent random variables with the same distribution as $\xi$. We study conditional empirical moment estimators of the form (ref), where the kernel function $K(x,x^\prime)$ admits the decomposition \begin{equation} K(x, X_i) = \sum_{q=1}^r \mathbb{I}\{i \in \mathsf{s}_q\} \kappa(x, X_i, D_{\mathsf{s}_q}, \xi_{\mathsf{s}_q}) \end{equation} for some known kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}}, \xi_{\mathsf{s}})$. Several widely applied instances of subsampled kernels are as follows.\footnote{The half-sample bootstrap is particularly computationally efficient for estimators constructed with subsampling, as the estimator and the half-sampled estimator can be constructed using the same collection of subsamples. See Section 4.1 of athey2019generalized for further discussion.} \begin{ex} Subsampled nearest-neighbors regression is a simple example of a kernel with the structure (ref) fix1989discriminatory. Here, the kernel $\kappa(x,X_{i^\prime},D_{\mathsf{s}}, \xi_{\mathsf{s}})$ is non-zero if and only if $X_{i^\prime}$ is one of the $k$ closest points to $x$ among the points in the subsample $D_{\mathsf{s}}$. \end{ex} \begin{ex} Random forest regression, introduced by breiman2001random, is another example of a kernel with the structure (ref). In this case, each pair $(D_{\mathsf{s}}, \xi_{\mathsf{s}})$ generates some partition of $\mathcal{X}$. The kernel $\kappa(x,x^\prime,D_{\mathsf{s}}, \xi_{\mathsf{s}})$ is non-zero if and only if $x$ and $x^\prime$ are in the same element of the partition generated by $(D_{\mathsf{s}}, \xi_{\mathsf{s}})$. Often, such partitions are constructed with recursive algorithms, e.g., the “CART” algorithm of breiman2017classification. \end{ex} We impose the following restrictions on the kernels under consideration. \begin{assumption}[Honesty and Positive Symmetry]$\text{ }$\\ (i) The kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}},\xi)$ is Honest in the sense that \begin{equation} \kappa(x,X_i,D_{\mathsf{s}},\xi_{\mathsf{s}}) \ci m(D_i; \theta, g) \mid X_i, D_{\mathsf{s}_{-i}} , \end{equation} where $\ci$ denotes conditional independence and $\mathsf{s}_{-i}$ denotes the set $\mathsf{s}\setminus\{i\}$. (ii) The kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}},\xi)$ is positive and satisfies the restriction $\sum_{i\in{s}}\kappa(\cdot, X_i, D_{\mathsf{s}},\xi_{\mathsf{s}}) = 1$ almost surely. Moreover, the conditional expectation $\mathbb{E}\left[\kappa(\cdot, X_i, D_{\mathsf{s}},\xi_{\mathsf{s}})\mid D_{\mathsf{s}}\right]$ is invariant to permutations of the data $D_{\mathsf{s}}$. \end{assumption} The “Honesty” condition stipulated in Part (i) of (ref) imposes the restriction that any part of the data $D_i$ that can affect the value of the moment $m(D_i; \theta, g)$ cannot affect the value of the kernel $\kappa(x,X_i,D_{\mathsf{s}},\xi_{\mathsf{s}})$. This condition was introduced in athey2016recursive. Honesty is often achieved through kernel construction schemes based on sample-splitting; see wager2018estimation and athey2019generalized for further discussion. Part (ii) of (ref) imposes several weak regularity conditions. The following two quantities restrict the “size” and “variability” of the chosen kernel. Define the generic norm $\|\cdot\|_\infty$ on $\mathcal{X}$. \begin{defn}[Shrinkage and Incrementality] Let $\mathsf{s}$ be an arbitrary element of $\mathcal{S}_{n,b}$ and let $l$ be any element of $\mathsf{s}$.\\ (i) We say that the kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}},\xi_{\mathsf{s}})$ has a uniform shrinkage rate $\varepsilon_{b}$ if \begin{equation} \sup_{P \in \mathbf{P}} \sup_{j \in [d]} \mathbb{E}\left[ \max\left\{ \|X_i - x^{(j)} \|_\infty : \kappa(x^{(j)},X_i,D_{\mathsf{s}},\xi_{\mathsf{s}}) > 0 \right\} \right] \leq \varepsilon_{b} . \end{equation} (ii) We say that the kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}},\xi_{\mathsf{s}})$ is uniformly incremental if \begin{equation} \inf_{P \in \mathbf{P}} \sup_{j \in [d]} \Var\left( \mathbb{E}\left[ \sum_{i\in\mathsf{s}} \kappa(x^{(j)}, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g) \mid D_l = D \right] \right) \gtrsim b^{-1} \end{equation} where $D$ is an independent random variable with distribution $P$. \end{defn} The shrinkage rate of a kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}},\xi_{\mathsf{s}})$ is analogous to the bandwidth of a classical, deterministic, kernel. The incrementality restriction ensures that the chosen kernel is not overly dependent on a single data point. Both notions were introduced by wager2018estimation and have been characterized explicitly for various widely applied subsampled kernel estimators.\footnote{The terminology “shrinkage” was introduced in oprescu2019orthogonal, and is not intended to connote (explicit) regularization.} \begin{ex}[continues=ex: knn] In the case of honest subsampled $k$-NN regression, khosravi2019non show that $\varepsilon_{b} \lesssim b^{-1/p}$, where $p$ is the “intrinsic dimension" of the measure of the covariates $X_i$. Roughly speaking, a distribution has an intrinsic dimension of $p$ if it is (locally) well approximated by a measure supported on a subspace of $\mathcal{X}$ of dimension $p$. See e.g., kpotufe2011k for further discussion. In turn, khosravi2019non and peng2022rates show that the kernels associated with both honest and non-honest variants of $k$-NN regression are incremental, up to logarithmic factors that depend on the dimension of the covariates. \end{ex} \begin{ex}[continues=ex: random forest] Analogously, for honest random forest regression, under suitable regularity conditions, wager2018estimation establish that $\varepsilon_{b} \lesssim b^{-c/p}$, where $p$ is the dimension of the domain of $X_i$. See e.g., wager2018estimation and oprescu2019orthogonal for further discussion. Bounds adaptive to the intrinsic dimension of the measure of $X_i$ are given in huo2023adaptation under further restrictions. wager2018estimation and peng2022rates give simple conditions under which the kernel associated with subsampled, honest, random forest regression is uniformly incremental, again up to dimension dependent logarithmic factors. \end{ex} \subsection{Moment Restrictions} The uniform confidence region introduced in (ref) makes no explicit correction for bias. In other words, the confidence regions that we consider are reliant on the use of estimators $\hat{\theta}_n(\bm{x}^{(d)})$ whose bias is of a smaller stochastic order than the sampling variance. To this end, we emphasize the use of conditional moments $M(\cdot;\theta_0, g_0)$, that satisfy a local Neyman orthogonality condition chernozhukov2018double. To introduce Neyman orthogonality, we impose further structure on the nuisance parameter $g_0$. The nuisance parameter $g_0$ is a collection of $h$ real-valued functions $g_0 = (g_0^{(k)})_{k=1}^h$, each having domain $\mathcal{Z}$. Define the norm \begin{equation} \| g - g_0 \|_{2,\infty} = \sup_{k\in [h]} \sup_{j\in [d]} \left(\mathbb{E} \left[ (g^{(k)}(Z_i) - g_0^{(k)}(Z_i))^2 \mid X_i = x^{(j)} \right] \right)^{1/2} \end{equation} for any $g = (g^{(k)})_{k=1}^h$. The nuisance parameter $g_0$ takes values in the space $\mathcal{G}$. \begin{defn}[Local Neyman Orthogonality] We say that a conditional moment $M(\cdot;\theta_0, g_0)$ is uniformly locally Neyman orthogonal if \begin{equation} \partial_g M(x^{(j)}; \theta_0, g_0)[g-g_0] = 0 \end{equation} for all $P$ in $\mathbf{P}$ and $x^{(j)}$ in $\bm{x}^{(d)}$. \end{defn} The use of Neyman orthogonal moments ensures that the bias induced by the estimation of the nuisance parameter $g_0$ with $\hat{g}_n$ is small chernozhukov2018double. In addition to Neyman orthogonality, we require several smoothness restrictions on the function $m(\cdot;\theta, g)$. In the main text, to ease exposition, we impose the following linearity and boundedness restriction. \begin{assumption}[Moment Linearity and Boundedness] The moment function $m(\cdot;\theta, g)$ satisfies the linear representation \begin{equation} m(D_i;\theta, g) = m^{(1)}(D_i;\theta, g)\cdot \theta + m^{(2)}(D_i; g) \end{equation} for some known functions $m^{(1)}(\cdot;\theta, g)$ and $m^{(2)}(\cdot; g)$. Moreover, the absolute value of the function $m(\cdot;\theta, g)$ is bounded by the constant $(\theta+1)\phi$ for some $\phi \geq 1$ almost surely. \end{assumption} The linearity restriction entailed in (ref) is inessential and is imposed for the sake of simplicity.\footnote{In (ref), we show that moment linearity can be replaced by the high-level assumption that $\hat{\theta}_n(\bm{x}^{(d)})$ is consistent for $\theta_0(\bm{x}^{(d)})$. This state of affairs is standard in $M$-estimation problems newey1994large. As our running examples use linear moments, and sufficient conditions for the consistency of $\hat{\theta}_n(\bm{x}^{(d)})$ have been established oprescu2019orthogonal, we omit a detailed consideration of nonlinear moments.} The boundedness restriction can be easily weakened to a slightly more involved assumption stated in terms of sub-exponential norms. Again, we impose boundedness to ease exposition. We maintain the following additional mild smoothness restrictions. \begin{assumption}[Moment Smoothness]$\text{ }$\\ \textbf{(i)} The moment $M(\cdot; \theta, g_0)$ is uniformly second order smooth, in the sense that \begin{equation} \sup_{P\in\mathbf{P}} \sup_{j\in[d]} \big\vert \partial_{g,g} M(x^{(j)}; \theta_0, g_0)[g-g_0]\big\vert \lesssim \|g - g_0\|^2_{2,\infty} \end{equation} for each $g$ in $\mathcal{G}$. \textbf{(ii)} The variogram \begin{equation*} V(x ; g) = \mathbb{E}\left[ (m(D_i ;\theta_0(x), g) - m(D_i ;\theta_0(x), g_0))^2 \mid X = x\right] \end{equation*} is uniformly Lipschitz in its first component, in the sense that \begin{equation} \sup_{P\in\mathbf{P}} \sup_{g\in\mathcal{G}} \big\vert V(x ; g) - V(x^\prime ; g) \big\vert \lesssim \|x - x^\prime\|_\infty \end{equation} holds for all $x$ and $x^\prime$ in $\mathcal{X}$. Moreover, the variogram satisfies the mean-squared continuity condition \begin{equation} \sup_{P\in\mathbf{P}} \sup_{j\in[d]} \vert V(x^{(j)}, g) - V(x^{(j)}, g^\prime)\vert \lesssim \|g - g^\prime\|^2_{2,\infty} \end{equation} for each $g$ and $g^\prime$ in $\mathcal{G}$. \textbf{(iii)} Define the moments \begin{align} M^{(1)}(x;\theta,g) &= \mathbb{E}\left[ m^{(1)}(D_i;\theta,g) \mid X_i = x\right] \quad\text{and}\\ M^{(2)}(x;g) &= \mathbb{E}\left[ m^{(2)}(D_i;g) \mid X_i = x\right] , \end{align} associated with the functions $m^{(1)}(\cdot;\theta, g)$ and $m^{(2)}(\cdot; g)$ introduced in (ref). Both moments are uniformly Lipschitz in their first component. That is, it holds that \begin{equation} \sup_{P\in\mathbf{P}} \sup_{g\in\mathcal{G}} \big\vert M^{(1)}(x ;\theta, g) - M^{(1)}(x^\prime ;\theta, g) \big\vert \lesssim \|x - x^\prime\|_\infty \end{equation} for all $x$, $x^\prime$, and $\theta$, and analogously for $M^{(2)}(x;g)$. Moreover, the first moment is uniformly Lipschitz in its second component and bounded from below in the sense that \begin{align} \sup_{P\in\mathbf{P}} \sup_{j\in[d]} \big\vert M^{(1)}(x^{(j)}; \theta_0,g) - M^{(1)}(x^{(j)}; \theta_0,g_0)\big\vert & \lesssim \|g - g_0\|_{2,\infty}\quad\text{and}\\ \inf_{P\in\mathbf{P}} \inf_{j\in[d]} \big\vert M^{(1)}(x^{(j)};\theta_0,g)\big\vert & \geq c \end{align} for each $g$ and $\theta$ and some positive constant $c$. \end{assumption} Neyman orthogonal moments satisfying the smoothness restrictions specified in (ref) and (ref) are available for many widely considered statistical problems. For example, the moment (ref) is the unique Neyman orthogonal identifying moment for the parameter (ref) hahn1998role, chernozhukov2018double.\footnote{For estimation of CATEs, Part (i) of (ref) is implied by the more refined bound \begin{equation} \sup_{P\in\mathbf{P}} \sup_{j\in[d]} \big\vert \partial_{g,g} M(x^{(j)}; \theta_0, g_0)[g-g_0]\big\vert \lesssim \|\mu - \mu_0\|_{2,\infty} \|\beta - \beta_0\|_{2,\infty} , \end{equation} where $\mu$ and $\beta$ are defined in (ref). This structure yields the celebrated “Double Robustness” result for estimation of average treatment effects and conditional average treatment effects; see chernozhukov2018double and kennedy2023towards for further discussion. We impose the more general condition (ref), as this bound exhibits many of the same features and will hold for a wider variety of problems.} Additional examples of problems where smooth Neyman orthogonal identifying moments are available include estimation of partially linear regression and partially linear instrumental variable regression chernozhukov2018double, local average treatment effects tan2006regression,frolich2007nonparametric, dynamic treatment effects lewis2021double, and long-term treatment effects identified by surrogate outcomes athey2020estimating,chen2023semiparametric. \subsection{Coverage Accuracy} The following theorem gives a bound on the error in the nominal coverage probability of the confidence regions introduced in (ref). We emphasize that the result is applicable to asymptotic regimes where the dimension of the query-vector $\bm{x}^{(d)}$ can be exponentially larger than the sample size $n$. \begin{theorem} Suppose that the kernel $\kappa(\cdot,\cdot,D_{\mathsf{s}},\xi_{\mathsf{s}})$ satisfies (ref), has uniform shrinkage rate $\varepsilon_{b}$, and is uniformly incremental. Additionally, suppose that the Neyman orthogonal moment function $M(\cdot;\theta_0, g_0)$ satisfies (ref) and (ref). Moreover, suppose that quantity $\|\theta_0(\bm{x}^{(d)})\|_\infty$ is uniformly bounded as $P$ varies over $\mathbf{P}$ and that $r$ has been chosen to satisfy $n \leq b \sqrt{r}$. If the nuisance parameter estimator $\hat{g}_n$ is statistically independent of the data $\mathbf{D}_n$ and satisfies the probability bound \begin{align} \sup_{P\in\mathbf{P}} P\left\{ \|\hat{g}_n - g_0 \|^2_{2,\infty} \geq \sqrt{\frac{b}{n}} \delta^2_{n,g} \right\} \lesssim \frac{1}{n} \end{align} for some sequence $\delta_{n,g}$, then the confidence region formulated in (ref) satisfies the bound \begin{align} & \sup_{P\in{\mathbf{P}}}\big\vert P\left\{ \theta_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}) \right\} - (1 - \alpha) \big\vert \nonumber \\ & \quad \quad \lesssim \left(\frac{b \phi^4 \log^5(dn)}{n}\right)^{1/4} + \left(\delta^2_{n,g} + \sqrt{\frac{n}{b}}\varepsilon_b\right)\sqrt{\log(d)} , \end{align} for all $b$ larger than $C\log(n)$.\footnote{The statistical family $\mathbf{P}$ is defined implicitly by the omitted constants in the uniform bounds stated in (ref) and (ref), in addition to the restriction that $\|\theta_0(\bm{x}^{(d)})\|_\infty$ is bounded as $P$ varies over $\mathbf{P}$. } \end{theorem} \begin{remark} To expedite exposition, in stating (ref), we have assumed that the nuisance parameter estimator $\hat{g}_n$ is computed on a separate sample, independent of the data $\mathbf{D}_n$. This can be achieved by randomly splitting the available data into two subsamples. The first subsample can be used to construct the nuisance parameter estimate $\hat{g}_n$. The second subsample can then be used to construct the confidence region $\hat{\mathcal{C}}(\bm{x}^{(d)})$. There are practical issues with this strategy. First, the region $\hat{\mathcal{C}}(\bm{x}^{(d)})$ might be sensitive to the choice of the split of the data ritzwoller2023reproducible. Second, by splitting the data, researchers incur a potentially meaningful loss in statistical precision. In (ref) we give additional conditions under which the nuisance parameter estimator $\hat{g}_n$ can be computed using the same data used to construct the region $\hat{\mathcal{C}}(\bm{x}^{(d)})$.\footnote{Our analysis builds on a similar result given in chen2022debiased, who give a pointwise analysis of unconditional moment estimators.} If these conditions are not plausible for a given application, in (ref), we detail a modified version of the confidence region given in (ref) that is constructed with cross-splitting, and will be less sensitive to residual randomness or subject to losses in statistical power. \end{remark} \begin{remark} (ref) follows from an application of a more general result stated in (ref). This result applies to conditional moment estimators that are not necessarily constructed with subsampled kernels. Rather, the result holds under the high-level assumption that the estimator $\hat{\theta}_n(\bm{x}^{(d)})$ is approximately linear, with a sufficiently small remainder term, and has sufficiently small bias and stochastic equicontinuity. An outline of the argument used to verify these conditions for subsampled kernel regression is given in (ref). The detailed proof is given in (ref). Some aspects of this argument are standard. Others are new. First, through a standard series of expansions chernozhukov2018double, oprescu2019orthogonal, we show that the root $R_n(\bm{x}^{(d)})$ can be approximated by the statistic \begin{align} \bar{U}_{n,b}(\bm{x}^{(d)}) & = \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \mathbb{E}\left[u(\bm{x}^{(d)}; D_{\mathsf{s}}, \xi_{\mathsf{s}}, \theta_0, g_0)\mid D_{\mathsf{s}}\right] ,\quad\text{where}\\ u(x; D_\mathsf{s}, \xi_{\mathsf{s}}, \theta, g) &= - M^{(1)}(x;g_0)^{-1}\sum_{i\in\mathsf{s}} \big( \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g) \\ & \quad\quad\quad\quad\quad\quad\quad\quad\quad- \mathbb{E}\big[ \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g)\big]\big) ,\nonumber \end{align} with a remainder term given by the second term in (ref). The quantity (ref) can be recognized as a complete, deterministic, $U$-statistic of order $b$.\footnote{The incrementality condition specified in (ref) ensures that this $U$-statistic is non-degenerate.} Second, we apply a new result that demonstrates that $U$-statistics of order $b$ are approximately linear with a remainder term of order approximately $(b/n)^{b/2}$, up to dimension dependent logarithmic factors. This is a dramatic improvement over analogous results given in song2019approximating and minsker2023u, whose remainder terms exhibit polynomial decay as $b$ and $n$ grow and only apply to regime $b = o(n^{1/3})$. This result has other applications and is presented in detail in (ref). Written differently, we establish that the root $R_n(\bm{x}^{(d)})$ satisfies a linear representation of the form (ref), up to a small remainder term. It immediately follows that the bootstrap root $R^*_n(\bm{x}^{(d)})$ satisfies the linear representation (ref), up to a small remainder term. We conclude by applying suitable central limit theorems chernozhuokov2022improved to the linear terms (ref) and (ref). \end{remark} \begin{remark} The bound (ref) can be interpreted as a bias-variance decomposition. Or, perhaps more literally, a bias-normal-approximation decomposition. The first term in (ref) results from a Berry-Esseen type bound on the accuracy of a normal approximation to (the maximum of) (ref). The term involving the kernel shrinkage $\varepsilon_b$ is a remnant of a bound on the supremum of the bias of the estimator $\hat{\theta}_n(\bm{x}^{(d)})$. \end{remark} (ref) states a bound on coverage error in terms of two generic sequences: $\delta_{n,g}$, expressing the rate of convergence of the nuisance parameter estimator $\hat{g}_n$, and $\varepsilon_b$, measuring the effective bandwidth of the kernel. The bound (ref) exhibits an interesting tradeoff between these objects and the choice of subsample size $b$. To see this, suppose that \begin{equation} b = n^{\gamma_b},\quad \varepsilon_b \lesssim b^{-\gamma_\varepsilon},\quad\text{and}\quad \|\hat{g}_n - g_0\|_{2,\infty} \lesssim n^{-\gamma_g} , \end{equation} with probability greater than $1-1/n$, for some constants $\gamma_b$, $\gamma_\varepsilon$, and $\gamma_g$ between $0$ and $1$. In this case, ignoring logarithmic factors and other constants, the bound (ref) can be re-expressed as \begin{equation} n^{\frac{\gamma_b - 1}{4}} + n^{\frac{1-\gamma_b-4\gamma_g}{2}} + n^{\frac{1-\gamma_b(1+2\gamma_\varepsilon)}{2} } . \end{equation} In other words, the confidence region defined in (ref) is consistent if \begin{equation} 1 \leq \gamma_b+ 4\gamma_g \quad\text{and}\quad 1 \leq \gamma_b(1+2\gamma_\varepsilon) , \end{equation} respectively. That is, we are able to accommodate larger values of the shrinkage rate $\varepsilon_b$ and nuisance parameter estimation error $\delta_{n,g}$ if the subsample size $b$ is larger, relative to the sample size $n$. However, as the subsample size $b$ increases, the normal approximation error (i.e., the first term in (ref)) increases. Recall from (ref) that, for many popular, honest, subsampled kernel estimators, the shrinkage rate $\varepsilon_b$ satisfies a bound $\varepsilon_b \lesssim b^{-c/p}$ for some small constant $c$ and some integer $p$ measuring the (potentially, intrinsic) dimension of the covariates $X_i$. Thus, in order to ensure that the second inequality in consistency condition (ref) is satisfied, it is essential to accommodate subsample sizes $\gamma_b$ close to one, i.e., the regime $b = o(n)$. This is enabled by the general results on the asymptotic linearity of $U$-statistics given in (ref). When the subsample size scaling factor $\gamma_b$ is close to one, the restriction imposed by the consistency condition (ref) on the rate of convergence of the nuisance parameter estimator $\hat{g}_n$ is less stringent. That is, if $\gamma_b = 1- c/p$, then we need that $\gamma_g \geq c/4p$. On the other hand, if $\gamma_b$ is close to zero, i.e., if $b$ is very small, then the first condition in (ref) reduces to the familiar condition that nuisance parameters can be estimated at the rate $n^{-1/4}$ in root mean squared error chernozhukov2018double. If the nuisance parameter estimator $\hat{g}_n$ is itself estimated with random forest regression, syrgkanis2020estimation, chi2022asymptotic, and huo2023adaptation, among others, give conditions under which sufficient rates of convergence can be achieved. \subsection{Performance} We now measure the performance of the confidence region formulated in (ref). We apply a method for simulation design proposed by athey2021using. In particular, we calibrate a simulation to the banerjee2015multifaceted data using a Generative Adversarial Network (GAN) goodfellow2014generative. Further details on this calibration are given in (ref). In effect, we construct a data generating process that approximates the banerjee2015multifaceted data, where we know the true value of the CATE $\theta_0(x)$ queried at each value $x$ used to construct the grids displayed in (ref). In this section, we consider the setting where the nuisance parameter estimator $\hat{g}_n$ is computed using the same data used to construct the region $\hat{\mathcal{C}}(\bm{x}^{(d)})$.\footnote{Recall that a theoretical justification for this approach is given in (ref).} Analogous results for an estimator constructed with cross-splitting are given in (ref). The results reported there are very similar. Measurements of performance are taken as two parameters vary. First, we consider several values of the sample size $n$. In particular, we consider settings with $n = h\cdot n_0$, for $h$ in $\{1, 2.5, 5, 7.5\}$, where $n_0$ is the sample size of the banerjee2015multifaceted data. Second, we vary the proportion $b/n$. We consider three regimes: $b/n = 0.05$, $b/n = (2/(h+1))0.05$, and $b/n=(1/h)0.05$. Observe that $b$ increases in proportion to $n$ in the first regime and that $b$ is constant as $n$ varies in the third regime. The second regime resides between these two extremes. \begin{figure}[t] \begin{centering} \caption{Performance} \begin{tabular}{c} \end{tabular} \end{centering} \justifying {Notes: (ref) displays several measurements of the performance of the confidence intervals formulated in (ref) in a simulation calibrated to the banerjee2015multifaceted data. The nominal level is $\alpha = 0.1$; a horizontal dotted line is displayed at the nominal coverage $1-\alpha$ in the first panel. The confidence bounds considered are constructed analogously to the confidence bounds displayed in (ref). The $x$-axis of each panel is the sample multiplier $h$. The color of each measurement varies with the choice of $b/n$. Further details on the design and implementation of this simulation are given in (ref).} \end{figure} (ref) displays measurements of the coverage and width of the confidence region formulated in (ref), in addition to measurements of the bias of the estimator $\hat{\theta}_n(\bm{x}^{(d)})$. The first row displays measurements of the coverage of the confidence region, the coverage of the lower bound (i.e., Panel B of (ref)), and the coverage of the upper bound (i.e., Panel A of (ref)). The nominal level is $\alpha = 0.1$. Throughout, the confidence region is somewhat conservative. The second row of (ref) illustrates a bias-variance trade-off with the subsample size $b$. The first panel displays measurements of the average width of the confidence region. Here, the average is taken over both simulation draws and the query-vector $\mathbf{x}^{(d)}$. The width of the confidence region is increasing in the proportion $b/n$ and is essentially constant if $b/n$ is constant as $n$ increases. By contrast, the second two panels display the maximum and average bias of the estimator $\hat{\theta}_n(\bm{x}^{(d)})$, again taken over the query-vector $\mathbf{x}^{(d)}$. The bias is decreasing in the proportion $b/n$ and is essentially constant if $b$ is constant as $n$ varies. \section{General Results for High-Dimensional $U$-Statistics} An essential step in the proof of (ref) follows from a new order-explicit bound on the remainder in a linear approximation to a high-dimensional $U$-statistic. In this section, we present this result and state several corollaries. In particular, we give new order-explicit results on the concentration and normal approximation of high-dimensional $U$-statistics. That is, we consider the asymptotic behavior of the $b$ order $U$-statistic \begin{equation} \bar{U}_{n,b}(\bm{x}^{(d)}) = \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} u(\bm{x}^{(d)}; D_\mathsf{s}) , \end{equation} where the vector $ u(\bm{x}^{(d)}; \cdot) $ collects the deterministic, symmetric, real-valued kernel function $ u(x; \cdot), $ evaluated at the $d$-vector of points $\bm{x}^{(d)} = (x^{(j)})_{j=1}^d$ in the space $\mathcal{X}$. We assume that each component of the kernel function $u(\bm{x}^{(d)}; D_\mathsf{s})$ has mean zero. \subsection{Context} The asymptotic analysis of $U$-statistics was initiated by hoeffding1948class, who established a central limit theorem in the regime where the order $b$ is fixed and the sample size $n$ is increasing. The Hoeffding central limit theorem has been extended only recently to the regime where the order $b$ increases with the sample size $n$. diciccio2022clt give a result with this flavor in the regime where $b = o(n^{1/2})$. wager2018estimation, peng2022rates, and minsker2023u strengthen this result to the regime where $b=o(n)$. We state and prove this more general result for the sake of completeness, and because its main ideas will serve as useful touch points in the more involved analysis to follow. Define the kernel variance $\nu^2_j = \Var(u(x^{(j)}; D_{[b]}))$, the H\'{a}jek projection \begin{equation} u^{(1)}(x^{(j)}; D) = \mathbb{E}\left[ u(x^{(j)}; D_{[b]}) \mid D_1 = D\right] , \end{equation} and the H\'{a}jek projection variance $\sigma^2_{b,j} = \Var(u^{(1)}(x^{(j)}; D_i))$. \begin{theorem} For any sequence of kernel orders $b = b_n$, where $\nu^2_j/n\sigma^2_{b,j} \to 0$ as $n\to \infty$, we have that \begin{equation} \sqrt{\frac{n}{\sigma^2_{b,j} b^2}} \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} u(x^{(j)}; D_\mathsf{s}) \overset{d}{\to} \mathsf{N}(0,1) , \end{equation} as $n\to\infty$, where $\overset{d}{\to}$ denotes convergence in distribution. \end{theorem} The proof is given in (ref). As a by-product of the proof, we show that $b \sigma^2_{b,j} \leq \nu^2_j$. Thus, the normalization $\nu^2_j/n\sigma^2_{b,j} \to 0$ implies that $b = o(n)$. Large deviation bounds for high-dimensional $U$-statistics, i.e., $U$-statistics with vector-valued kernels, were not given until hoeffding1963probability. This result is now more standard; a modern version is stated as follows. A proof is given in song2019approximating. The norm $\|\cdot\|_{\psi_1}$ denotes the $\psi_1$-Orlicz norm.\footnote{Random variables are sub-exponential if and only if they have a finite $\psi_1$-Orlicz norm vershynin2018high.} \begin{lemma}[Lemma A.5, song2019approximating]If the bound $\|u(x^{(j)};D_{\mathsf{s}})\|_{\psi_1} \leq \phi$ is satisfied for each $j$ in $[d]$, then \begin{equation} \|\bar{U}_{n,b}(\bm{x}^{(d)}) \|_\infty \lesssim \sqrt{\frac{b\bar{\nu}^2 \log(dn)}{n}} + \frac{b\phi \log^2(dn)}{n} \end{equation} with probability greater than $1 - C/n$, where $\bar{\nu}^2 = \max_{j\in[d]} \nu^2_j$. \end{lemma} Again, (ref) demonstrates that $\bar{U}_{n,b}(\bm{x}^{(d)})$ concentrates in the regime that $b = o(n)$, up to a logarithmic factor that depends on the dimension $d$. Here, however, concentration is expressed in terms of the quantity $n^{-1} b\bar{\nu}^2$, rather than the more appropriate, and potentially substantively smaller, normalizing quantity $n^{-1} b^2\sigma^2_{b,j}$ used in (ref). In part motivated by this incongruity, arcones1993limit, arcones1995bernstein, gine2000exponential, establish a series of refined large deviation bounds for high-dimensional $U$-statistics that use the appropriate normalizing factor (among many other related results).\footnote{peel2010empirical give empirical Bernstein versions of the arcones1993limit deviation bounds. Again, here, the constants used to express these bounds depend exponentially on the order $b$.} See de2012decoupling for a textbook treatment. However, the constants used to express these bounds depend implicitly on the order $b$, and so are not applicable to asymptotic regimes where $b$ may be growing with the sample size $n$. More recently, chen2018gaussian, chen2019randomized, and song2019approximating have studied central limit theorems for high-dimensional $U$-statistics. Of these papers, only song2019approximating gives results with explicit dependence on the order $b$. Their results are only applicable to the regime where $b = o(n^{1/3})$. minsker2023u gives a large deviation bound with the correct normalizing factor and explicit order dependence, but this result is again only applicable to the regime $b = o(n^{1/3})$. Our main application to inference for subsampled kernel regression requires $n^{1 - c/p} \lesssim b$, where $p$ is the dimension of the covariate vector $Z_i$. Thus, results applicable only to the regime $b = o(n^{1/3})$ are insufficient. \subsection{Concentration of the H\'{a}jek Residual} We obtain a large deviation bound on the difference \begin{equation} \bar{U}_{n,b}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n u^{(1)}(\bm{x}^{(d)}; D_i) . \end{equation} We refer to the quantity (ref) as the H\'{a}jek residual hajek1968asymptotic. This bound is used in the proof of (ref) and implies a new large deviation bound and central limit theorem for high-dimensional $U$-statistics, stated in the following subsection. The proof is given in (ref). \begin{theorem} If the kernel $u(x^{(j)}; D_{\mathsf{s}})$ satisfies the bound $\|u(x^{(j)};D_{\mathsf{s}})\|_{\psi_1} \leq \phi$ for each $j$ in $[d]$, then \begin{align} & \Big\| \bar{U}_{n,b}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n u^{(1)}(\bm{x}^{(d)}; D_i) \Big\|_{\infty} \lesssim \xi_{n,b} , \quad\text{where}\\ &\quad\quad\quad\xi_{n,b} =\left(\frac{C b\log(dn)}{n}\right)^{b/2}\left(\bar{\nu} + b \sqrt{\frac{b}{n}} \phi\log^2(dn)\right)\nonumber , \end{align} with probability greater than $1-C/n$, where $\bar{\nu}^2 = \max_{j\in[d]} \nu^2_j$. \end{theorem} \begin{remark} Roughly speaking, the bound (ref) follows by first demonstrating that the H\'{a}jek residual can be expressed as a degenerate $U$-statistic of order $b$. This allows us to derive Hoffman-J\orgensen type bounds on higher moments of (ref) with a symmetrization argument. Here, we make essential use of a symmetrization inequality for completely degenerate kernels, with explicit dependence on the order, due to sherman1994maximal. This symmetrization inequality was also used in song2019approximating and minsker2023u. These papers apply the sherman1994maximal symmetrization inequality separately to terms in a Hoeffding expansion of the $U$-statistic $\bar{U}_{n,b}(\bm{x}^{(d)})$. Aggregating the resulting bounds necessitates the restriction to the regime $b = o(n^{1/3})$. By contrast, \emph{we avoid this restriction by applying the symmetrization inequality once, to the H\'{a}jek residual itself.} \end{remark} The bound (ref) indicates that the H\'{a}jek residual is of stochastic order $b (b\log(dn)/n)^{b/2}$. That is, roughly speaking, the H\'{a}jek residual concentrates like $(b/n)^{b/2}$ for sufficiently large $b$. Recall that ${n \choose b}$ is proportional to $(b/n)^b$. So, (ref) can be interpreted as showing that complete $U$-statistics, i.e., sums of approximately $(b/n)^b$ dependent random variables, concentrate like sums of $(b/n)^b$ independent random variables around their H\'{a}jek projections. Substantively, if $b = n^{1-\delta}$ for some $\delta >0$, then the bound (ref) shrinks \emph{exponentially} as $n$ increases. Exponential convergence, of this form, is a dramatic improvement over existing bounds song2019approximating,minsker2023u, which decay polynomially as $n$ increases, and only in the regime $b = o(n^{1/3})$. In our view, this result hints at an explanation for the widespread success of subsampling as a primitive operation in machine learning workflows and in statistical inference. Subsampled statistics, of a large order, are \emph{essentially} linear. Belaboring the point, aggregation of a potentially highly non-linear statistic $u(x; D_{[b]})$---a decision tree, say---through subsampling is a linearizing operation. Once linearized, the regular behavior---concentration, normality---of independent sums holds. \subsection{Concentration and Normal Approximation} We now state a large deviation bound and central limit theorem for the high-dimensional $U$-statistic (ref). Both results are corollaries of (ref), apply to the regime $b=o(n)$, and depend on the correct normalizing factor. Proofs are given in (ref). \begin{cor} Suppose that the conditions of (ref) continue to hold. Let $Z$ denote a centered Gaussian random vector with the same covariance matrix as the observation $u^{(1)}(\bm{x}^{(d)}, D_i)$ and let $\mathcal{R}$ denote the set of hyper-rectangles in $\mathbb{R}^d$. There exist some $C_1<1$ such that if $b\log(dn) / n < C_1$ and $b^{-C_2} \leq \underline{\sigma}^2_b$ for some $C_2 > 0$, then: \textbf{(i)} The inequality \begin{equation} \sqrt{\frac{n}{b^2}} \| \Sigma^{-1/2} \bar{U}_{n,b}(\bm{x}^{(d)}) \|_\infty \lesssim \log^{1/2}(dn) + \frac{\phi\log^2(dn)}{\underline{\sigma}_b n^{1/2}} \end{equation} holds with probability greater than $1-Cn^{-1}$ for all $b \gtrsim C_2\log(n)$. \textbf{(ii)} The inequality \begin{align} \sup_{\mathsf{R} \in \mathcal{R}} \bigg\vert P\left\{\sqrt{\frac{n}{b^2}}\Sigma^{-1/2} \bar{U}_{n,b}(\bm{x}^{(d)}) \in \mathsf{R}\right\} - P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}\right\}\bigg\vert \lesssim \left(\frac{ \phi^2 \log^5(dn)}{\underline{\sigma}^2_b n}\right)^{1/4} \end{align} holds for all $b \gtrsim C_2 \log(n)$. \end{cor} \begin{remark} The restriction that $b \gtrsim C \log(n)$ can be dropped along any sequence with $b/n\to0$. That is, analogous results will hold for sufficiently large $n$ and any $b=b_n$ such that $b_n / n \to 0$ as $n\to\infty$. \end{remark} \begin{remark} de2012decoupling state a result analogous to Part (i) of (ref), in the sense that the $U$-statistic $\bar{U}_{n,b}(\bm{x}^{(d)})$ is normalized by the correct quantity $n^{1/2}b^{-1}\Sigma^{-1/2}$. The constants used in their bound depend implicitly on $b$. Part (i) of (ref) improves substantially on (ref) in contexts where the H\'{a}jek projection variances $\sigma_{b,j}^2$ are smaller than $b^{-1}$.\footnote{Section 4 of song2019approximating gives several examples of statistics where the H\'{a}jek projection variance $\sigma_{b,j}^2$ is smaller than $b^{-1}$. Under the restriction $\sigma_{b,j}^2\gtrsim b^{-1}$, imposed in the application to subsampled kernel regression considered in (ref), Part (i) of (ref) only improves on (ref) by a constant factor.} Part (ii) of (ref) gives a high-dimensional generalization of (ref). An analogous half-sample bootstrap central limit theorem follows from arguments very similar to parts of the proof of (ref). \end{remark} \section{Conclusion} We build simultaneous confidence intervals for solutions to conditional moment equations. The confidence intervals are built around an estimator based on subsampled kernel regression. As a running example, we consider the construction of confidence regions for conditional average treatment effects around an estimator based on random forest regression. Empirically, we document that the proposed confidence region is able to recover treatment effect heterogeneity undetected by interacted linear regression. Theoretically, we establish a bound on coverage accuracy that illustrates a bias-variance tradeoff in the user-chosen subsample size. In order to do this, we obtain several new, best-available results on the asymptotics of high-dimensional $U$-statistics. We give conditions sufficient for the asymptotic validity of the proposed confidence intervals. However, the confidence intervals are not necessarily optimal, in any particular sense, under the maintained assumptions. It is likely to be the case that an optimal confidence region would need to incorporate a bias estimate and procedure for choosing tuning parameters to balance bias and variance (see e.g., chernozhukov2014anti and christensen2024adaptive for instantiations of these ideas). Adapting this approach to subsampled moment regression is an interesting direction for further research.

spacing{1.2}

appendix\begin{center} {\it Supplemental Appendix to:} \vskip0.2cm \begin{spacing}{1} {Simultaneous Inference for\\ Local Structural Parameters with Random Forests\daggerfootnote{Date: \today}}\\ \begin{tabular}[t]{c@{\extracolsep{4em}}c} {David M. Ritzwoller} & {Vasilis Syrgkanis}\\ {Stanford University} & {Stanford University} \\ \end{tabular} \end{spacing} \end{center} \begin{spacing}{1.13} \DoToC \end{spacing} \thispagestyle{empty} \setcounter{page}{0} \setcounter{figure}{0} \begin{spacing}{1.4} \section{An Abstract Bound on Coverage Error} In this appendix, we give an abstract bound on the accuracy of the nominal coverage probability for the confidence region introduced in (ref). We make no use of the kernel structure expressed in (ref). The proof is given in (ref). We require several mild smoothness restrictions on the moment function $M(\cdot;\theta, g)$. In contrast to the set of assumptions specified in (ref), we do not require moment linearity. Instead, we impose the following generalization of Part (iii) of (ref). \begin{assumption}[Moment Restrictions] The moment function $M(x;\theta_0, g_0)$ is twice continuously differentiable in its second argument. Let \begin{equation} M^{(1)}(x;\theta,g) = \frac{\partial}{\partial \theta^\prime} M(x;\theta^\prime, g_0) \vert_{\theta^\prime = \theta} \quadand\quad H(x;\theta,g) = \frac{\partial^2}{\partial^2 \theta^\prime} M(x;\theta^\prime, g_0) \vert_{\theta^\prime = \theta} \end{equation} denote the Jacobian and Hessian of $M(\cdot;\theta_0, g_0)$ in $\theta$, respectively. The Jacobian $M^{(1)}(x;\theta,g)$ is uniformly Lipschitz in its second argument and bounded from below in the sense that \begin{align} \sup_{P\in\mathbf{P}} \sup_{j\in[d]} \big\vert M^{(1)}(x^{(j)};\theta,g) - M^{(1)}(x^{(j)};\theta,g_0)\big\vert & \lesssim \|g - g_0\|_{2,\infty}\quadand\\ \inf_{P\in\mathbf{P}} \inf_{j\in[d]} \big\vert M^{(1)}(x^{(j)};\theta,g)\big\vert & \geq c \end{align} for each $g$ and $\theta$ and some positive constant $c$. The Hessian $H(x;\theta,g)$ is uniformly bounded as $x$, $\theta$, and $g$ vary over their respective domains. \end{assumption} Moreover, we impose an analogous restriction on the centered empirical moment \begin{equation} \bar{M}_n(x;\theta,g) = M_n(x;\theta,g,\mathbf{D}_n) - \mathbb{E}\left[ M_n(x;\theta,g,\mathbf{D}_n) \right] , \end{equation} where we have made the dependence on $\mathbf{D}_n$ implicit to ease notation. \begin{assumption}[Empirical Smoothness] The centered empirical moment (ref) is twice continuously differentiable in its second argument. Let \begin{align} \bar{H}_n(x;\theta, g) &= \frac{\partial^2}{\partial^2 \theta^\prime} \bar{M}_n(x;\theta^\prime, g_0) \vert_{\theta^\prime = \theta} \end{align} denote the Hessian of $\bar{M}_n(\cdot;\theta, g)$ in $\theta$, respectively. The Hessian $H_n(x;\theta,g)$ is uniformly bounded almost surely as $x$, $\theta$, and $g$ vary over their respective domains. \end{assumption} Next, we impose a set of high-level restrictions on the structure of the empirical conditional moment (ref). At times we refer to the normalized statistic \begin{equation} U_{n}(x) = -(M^{(1)}(x;g_0))^{-1}\bar{M}_{n}(x;\theta_0,g_0) . \end{equation} First, we impose a condition that ensures that (ref) is approximately linear. \begin{assumption}[Approximate Linearity] There exists a function $\bar{u}(\cdot,\cdot)$, a constant $\varphi\geq 1$, and real-valued sequences $\delta_{n,u}$ and $\rho_{n,u}$ such that $\mathbb{E}\left[\bar{u}(x^{(j)},D_i)\right] = 0$, \begin{equation} \|\bar{u}(x^{(j)},D_i)\|_{\psi_1} \leq \varphi , \quadand\quad \mathbb{E}\left[\bar{u}^4(x^{(j)},D_i)\right] \leq \Var(\bar{u}(x^{(j)},D_i)) \varphi^2 \end{equation} hold for all $j$ in $[d]$ and $P$ in $\mathbf{P}$. Moreover, if $\lambda^2_{j}$ denotes $\Var(\bar{u}(x^{(j)},D_i))$ and $\underline{\lambda}^2 = \min_{j\in[d]} \lambda^2_{j}$, then \begin{equation} \sup_{P\in\mathbf{P}} P\left\{ \sqrt{\frac{n}{\lambda^2}} \big\| U_n(\bm{x}^{(d)}) - \frac{1}{n}\sum_{i=1}^n \bar{u}(\bm{x}^{(d)}, D_i) \big\|_\infty \geq \delta_{n,u} \right\} \leq \rho_{n,u} . \end{equation} \end{assumption} Second, we impose several restrictions relating to the estimators $\hat{\theta}_n(\bm{x}^{(d)})$ and $\hat{g}_n$. We measure the error in the estimator of $\hat{\theta}_n(\bm{x}^{(d)})$ in terms of the norm \begin{equation} \|\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty = \sup_{j\in[d]} | \hat{\theta}_n(x^{(j)}) - \theta_0(x^{(j)}) | . \end{equation} A closely related collection of conditions is stated in chernozhukov2018double. \begin{assumption}[Bias, Consistency, and Stochastic Equicontinuity]Recall the definition of the object $\underline{\lambda}^2$ introduced in (ref). Let \begin{align} \bar{M}^{(1)}_n(x;\theta, g) & = \frac{\partial}{\partial \theta^\prime} \bar{M}_n(x;\theta^\prime, g_0) \vert_{\theta^\prime = \theta} \end{align} denote the Jacobian of the centered empirical moment $\bar{M}_n(x; \theta, g)$.\\ \textbf{(i)} Define the quantity \begin{equation} \mathsf{Bias}_n(x;\theta, g) = M(x;\theta,g)- \mathbb{E}\left[M_{n}(x;\theta,g,\bm{D}_n)\right] . \end{equation} There exists a sequence $\delta_{n,B}$ such that \begin{equation} \sup_{P\in\mathbf{P}} \sup_{g\in\mathcal{G}} \sqrt{\frac{n}{\underline{\lambda}^2}} \| \mathsf{Bias}_n(\bm{x}^{(d)};\theta(\bm{x}^{(d)}), g) \|_\infty \lesssim (1 + \|\theta(\bm{x}^{(d)})\|_\infty) \delta_{n,B} \end{equation} uniformly over any vector $\theta(\bm{x}^{(d)}) = \{\theta(x^{(j)})\}_{j=1}^d$. \textbf{(ii)} There exist sequences $\delta_{n,m}$, $\delta_{n,g}$, $\delta_{n,\theta}$, $\rho_{n,m}$, $\rho_{n,g}$, and $\rho_{n,\theta}$ such that \begin{align} \sup_{P\in\mathbf{P}} P\left\{ \sqrt{\frac{n}{\underline{\lambda}^2}} \big\| \bar{M}^{(1)}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), g_0) \big\|^2_\infty \geq \delta_{n,m} \right\} & \leq \rho_{n,m} ,\\ \sup_{P\in\mathbf{P}} P\left\{ \sqrt{\frac{n}{\underline{\lambda}^2}} \|\hat{g}_n - g_0 \|_{2,\infty}^2 \geq \delta^2_{n,g} \right\} \leq \rho_{n,g} , \quad\text{and}\\ \sup_{P\in\mathbf{P}} P\left\{ \sqrt{\frac{n}{\underline{\lambda}^2}} \|\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty^2 \geq \delta^2_{n,\theta} \right\} \leq \rho_{n,\theta} . \end{align} \textbf{(iii)} There exist sequences $\delta_{n,S}$, $\delta_{n,J}$, $\rho_{n,S}$, and $\rho_{n,J}$ such that, uniformly over $P$ in $\mathbf{P}$, \begin{align} \sqrt{\frac{n}{\underline{\lambda}^2}} \big\| \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), \hat{g}_n) - \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), g_0) \big\|_\infty \geq \delta_{n,S} \end{align} with probability less than $\rho_{n,S}$ and \begin{align} \sqrt{\frac{n}{\underline{\lambda}^2}} \big\| \bar{M}^{(1)}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), \hat{g}_n) - \bar{M}^{(1)}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), g_0) \big\|_\infty \geq \delta_{n,J} , \end{align} with probability less than $\rho_{n,J}$, respectively. \end{assumption} The following theorem gives a non-asymptotic bound on the error in the nominal coverage probability of the confidence regions introduced in (ref). \begin{theorem} Collect the error sequences \begin{align*} \delta_n &= \delta^2_{n,g} + \delta^2_{n,\theta} + \delta^2_{n,m} + \delta_{n,B} + \delta_{n,S} + \delta_{n,u} + \underline{\lambda}^{1/2}n^{-1/4} \delta_{n,\theta}\left(\delta_{n,B} + \delta_{n,J}\right) \end{align*} and $\rho_n= \rho_{n,m} + \rho_{n,g} + \rho_{n,\theta} + \rho_{n,S} + \rho_{n,J} + \rho_{n,u}$ and assume that $\delta_{\varepsilon n} \leq C_\varepsilon \delta_{n}$ and $\rho_{\varepsilon n} \leq C_\varepsilon \rho_{n}$ for any $0<\varepsilon<1$. Suppose that the Neyman orthogonal moment function $M(x;\theta_0, g_0)$ satisfies (ref) and Part (i) of (ref) and that the centered empirical moment function $\bar{M}_n(x;\theta_0, g_0)$ satisfies (ref). If (ref) and (ref) hold, then the confidence region defined in (ref) satisfies \begin{equation} \sup_{P\in\mathbf{P}} \big\vert P\left\{ \theta_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}) \right\} - (1 - \alpha)\big\vert \lesssim \left( \frac{\varphi^2\log^{5}\left(dn\right)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{d}}+ \rho_n . \end{equation} \end{theorem} \section{Outline of Proof of (ref)} The result follows from an application of (ref). We outline the proof in this Appendix. The detailed proof is given in (ref). First observe that the simplifying (ref) implies that (ref) holds with \begin{equation} \bar{M}_n^{(1)}(x; \theta,g) = \sum_{i=1}^n \left( K(x,X_i)m^{(1)}(D_i; \theta, g) - \mathbb{E}\left[K(x,X_i) m^{(1)}(D_i; \theta, g)\right]\right) \end{equation} and $\bar{H}_n(x; \theta,g) = 0$, respectively. The main challenge in proving (ref) is to quantify the various generic sequences defined in (ref) and (ref). To quantify the generic sequences $\delta_n^{u}$ and $\rho_n^{u}$ defined in (ref), we consider the quantity \begin{align} U_{n}(x) & = - \frac{1}{r} \sum_{q=1}^r u(x; D_{\mathsf{s}_q}, \xi_{\mathsf{s}}, \theta_0, g_0) ,\quad\text{where}\\ u(x; D_\mathsf{s}, \xi_{\mathsf{s}}, \theta, g) &= M^{(1)}(x;g_0)^{-1}\sum_{i\in\mathsf{s}} \big( \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g) \nonumber\\ &\quad\quad\quad\quad\quad\quad\quad\quad- \mathbb{E}\big[ \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g)\big]\big) .\nonumber \end{align} In particular, we apply (ref) to show that the, suitably scaled, difference between (ref) and its H\'{a}jek projection is small with high probability. To quantify the various sequences introduced in (ref), we apply arguments similar to those used in wager2018estimation and oprescu2019orthogonal. The novelty is that each bound is uniform over the query-vector $\bm{x}^{(d)}$. \section{Proof of (ref)} We are interested in studying the quantity \begin{equation} \bar{U}_{n,b}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) . \end{equation} Observe that \begin{align} \frac{b}{n} \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) & = {n-1 \choose b-1} {n \choose b}^{-1} \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \nonumber \\ & = \frac{1}{N_b} \sum_{s\in\mathcal{S}_{n,b}} \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \mathbb{I}\{ i \in \mathsf{s}\} \end{align} and that consequently the difference (ref) can be written \begin{align} \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} \left( \tilde{u}(\bm{x}^{(d)}; D_{\mathsf{s}}) - \sum_{i\in\mathsf{s}} \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i)\right) . \end{align} In other words, the difference (ref) can be re-expressed as a scaled, complete, $U$-statistic of order $b$ with the kernel function \begin{equation} h(\bm{x}^{(d)}; D_{\mathsf{s}}) = \tilde{u}(\bm{x}^{(d)}; D_{\mathsf{s}}) - \sum_{i\in\mathsf{s}} \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) . \end{equation} Moreover, the kernel function (ref) is completely degenerate, in the standard sense that \begin{equation} \mathbb{E}\left[h(\bm{x}^{(d)}; D_{\mathsf{s}}) \mid i \in \mathsf{s}, D_i\right] = 0 \end{equation} almost surely. To give a high-probability bound on the difference (ref), we construct a bound on the higher-order moment \begin{equation*} \mathbb{E}\left[ \bigg\vert \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} h(x; D_{\mathsf{s}}) \bigg\vert^q \right] \end{equation*} for an arbitrary $q\geq2$, where $x$ is an arbitrary element of the query-vector $\bm{x}^{(d)}$. We express this problem more tractably through a symmetrization argument. In particular, we apply the following symmetrization inequality for completely degenerate functions, due to sherman1994maximal. See Theorem 5.2 of song2019approximating for an expedited proof. \begin{lemma}[sherman1994maximal] Let $Z_1,\ldots,Z_n$ denote a collection of independent and identically distributed real-valued random variables. Consider a real-valued symmetric kernel function $f$ of order $b$ that satisfies \begin{equation} \mathbb{E}\left[ f(Z_1,\ldots,Z_b) \mid Z_1 \right] = 0 \end{equation} almost surely. Let $V_1,\ldots,V_n$ denote an independent collection of Rademacher random variables. If $\Phi(\cdot)$ is any convex function on $[0,\infty)$, then the symmetrization inequality \begin{equation} \mathbb{E} \left[ \Phi\left( \bigg\vert \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} f(Z_{\mathsf{s}}) \bigg\vert \right) \right] \leq \mathbb{E} \left[ \Phi\left( \bigg\vert 2^b \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} V_{\mathsf{s}} f(Z_{\mathsf{s}}) \bigg\vert \right) \right] \end{equation} holds, where $V_{\mathsf{s}} = \Pi_{i\in\mathsf{s}} V_i$ for each subset $\mathsf{s}$ in $[n]$. \end{lemma} We note that (ref) was also used in song2019approximating and minsker2023u. We differ in the object that we are applying (ref) to. Both song2019approximating and minsker2023u, decompose the $U$-statistic $\bar{U}_{n,b}(\bm{x}^{(d)})$ into a collection of degenerate $U$-statistics with a Hoeffding expansion, and apply (ref) to each component of this decomposition. Our main insight is that (ref) can be applied to the $U$-statistic (ref) directly, as the kernel function (ref) is completely degenerate. As in song2019approximating and minsker2023u, the application of (ref) is facilitated by the following moment bound for higher moments of Rademacher chaos, often referred to as the Bonami inequality. \begin{lemma}[Theorem 3.2.2, de2012decoupling] Fix a collection of real-valued quantities $\{ z_\mathsf{s} : \mathsf{s} \in \mathcal{S}_{n,b} \}$ and let $V_1,\ldots,V_n$ denote an independent collection of Rademacher random variables. Consider the homogeneous Rademacher chaos of order $b$, given by \begin{equation} Z_{b} = \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} V_\mathsf{s} z_{\mathsf{s}} , \end{equation} where $V_{\mathsf{s}} = \Pi_{i\in\mathsf{s}} V_i$ for each subset $\mathsf{s}$ in $[n]$. The moment inequality \begin{equation} \mathbb{E} \left[ \vert Z_b \vert^q \right] \leq q^{bq/2} (\Delta_b)^{q/2} , \quad\text{where}\quad \Delta_b = \sum_{s\in\mathcal{S}_{n,b}} (z_{\mathsf{s}})^2 , \end{equation} holds for every $q>2$. \end{lemma} (ref) implies that \begin{align} \mathbb{E}\left[ \bigg\vert \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} V_\mathsf{s} h(x; D_{\mathsf{s}}) \bigg\vert^q \mid \mathbf{D}_n\right] \leq q^{bq/2} \left( \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} {n\choose b}^{-1} \left(h(x; D_{\mathsf{s}})\right)^2 \right)^{q/2} . \end{align} Consequently, (ref) implies that \begin{align} \mathbb{E}\left[ \bigg\vert \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} h(x; D_{\mathsf{s}}) \bigg\vert^q \right] & \leq 2^{bq} \mathbb{E} \left[ \mathbb{E}\left[ \bigg\vert \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} V_\mathsf{s} h(x; D_{\mathsf{s}}) \bigg\vert^q \mid \mathsf{D}_n \right] \right] \nonumber\\ & \leq 2^{bq} q^{bq/2} \mathbb{E} \left[ \left( \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} {n\choose b}^{-1} \left(h(x; D_{\mathsf{s}})\right)^2 \right)^{q/2} \right] . \end{align} To simplify the expression (ref), we apply the following representation of complete $U$-statistics, due to hoeffding1948class. To express this result, we require some additional notation. Let $\mathcal{P}_n$ denote the set of permutations of $[n]$, treating each permutation $\pi$ in $\mathcal{P}_n$ as a bijection from $[n]$ to $[n]$. For each permutation $\pi$, define the set \begin{equation} \mathsf{s}_{\pi,l} = \{ \pi((l-1)b),\ldots, \pi(lb) \} . \end{equation} Observe that if $n$ is divisible by $b$, the collection $\mathsf{s}_{\pi,1},\ldots,\mathsf{s}_{\pi,n/b}$ is a mutually exclusive partition of the set $[n]$ for each permutation $\pi$. \begin{lemma}[hoeffding1948class] The complete $U$-statistic of order $b$ with kernel function $u(\cdot)$ admits the alternative representations \begin{equation*} \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} u(D_\mathsf{s}) = \frac{1}{n!} \sum_{\pi \in \mathcal{P}} \bigg\lfloor \frac{b}{n} \bigg\rfloor \sum_{l = 1}^{\lfloor n/b \rfloor} u(D_{\mathsf{s}_{\pi,l}}) , \end{equation*} where $\lfloor x \rfloor$ denotes the largest integer smaller than or equal to $x$. \end{lemma} In particular, by (ref), Jensen's inequality, and the bound \[ \left(\frac{n}{b}\right)^b \leq {n \choose b} \] we have that \begin{align} & 2^{bq} q^{bq/2} \mathbb{E} \left[ \left( \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} {n\choose b}^{-1} \left(h(x; D_{\mathsf{s}})\right)^2 \right)^{q/2} \right] \nonumber \\ & \quad = 2^{bq} q^{bq/2} \mathbb{E} \left[ \left( \frac{1}{n!} \sum_{\pi \in \mathcal{P}_n} \bigg\lfloor \frac{n}{b}\bigg\rfloor^{-1} \sum_{l=1}^{\lfloor n/b\rfloor} {n\choose b}^{-1} \left(h(x; D_{\mathsf{s_{\pi,l}}})\right)^2 \right)^{q/2} \right] \nonumber \\ & \quad \leq 2^{bq} q^{bq/2} \mathbb{E} \left[ \left( \bigg\lfloor \frac{n}{b}\bigg\rfloor^{-1} \sum_{l=1}^{\lfloor n/b \rfloor} \left(\frac{b}{n}\right)^{b} \left(h(x; D_{\mathsf{s_{\pi,l}}})\right)^2 \right)^{q/2} \right] , \end{align} where $\pi$ is an arbitrary element of $\mathcal{P}_n$. We note that the summands in (ref) are now independent and identically distributed. To bound the expectation (ref), we apply the following version of Rosenthal's inequality for non-negative random variables. \begin{lemma}[Theorem 15.10, boucheron2013concentration] Let $Z_1,\ldots,Z_n$ denote a collection of independent real-valued and non-negative random variables. For all $q\geq 1$, the moment inequality \[ \mathbb{E}\left[ \big\vert \sum_{i=1}^n Z_i\big\vert^q \right]^{1/q} \lesssim \mathbb{E}\left[ \sum_{i=1}^n Z_i \right] + q \mathbb{E}\left[ \max_{i \in [n]} \vert Z_i\vert^q \right]^{1/q} \] holds. \end{lemma} In particular, we find that \begin{align} & 2^{bq} q^{bq/2} \mathbb{E} \left[ \left( \bigg\lfloor \frac{n}{b}\bigg\rfloor^{-1} \sum_{l=1}^{\lfloor n/b \rfloor} \left(\frac{b}{n}\right)^{b} \left(h(x; D_{\mathsf{s_{\pi,l}}})\right)^2 \right)^{q/2} \right] \nonumber \\ & \quad \leq C^{bq} q^{bq/2} \Bigg( \mathbb{E} \left[ \bigg\lfloor \frac{n}{b}\bigg\rfloor^{-1} \sum_{l=1}^{\lfloor n/b \rfloor} \left(\frac{b}{n}\right)^{b} \left(h(x; D_{\mathsf{s_{\pi,l}}})\right)^2 \right] \nonumber\\ &\quad \quad\quad \quad\quad \quad\quad +\left(\frac{q}{2}\right) \mathbb{E} \left[ \max_{l \in [n/b]} \left( \left(\frac{b}{n}\right)^{b+1} \left(h(x; D_{\mathsf{s_{\pi,l}}})\right)^2 \right)^{q/2} \right]^{2/q} \Bigg)^{q/2} \nonumber \\ & \quad \lesssim C^{bq} q^{bq/2} \left(\frac{b}{n}\right)^{bq/2} \Bigg( \mathbb{E} \left[ \left(h(x; D_{\mathsf{s}})\right)^2 \right]^{q/2}\nonumber\\ &\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad +q^{q/2} \left(\frac{b}{n}\right)^{q/2} \mathbb{E} \left[ \max_{l \in [n/b]} \left( h(x; D_{\mathsf{s_{\pi,l}}}) \right)^q \right] \Bigg) . \end{align} where the first inequality follows from (ref) and the second inequality follows from the inequality $(x+y)^n \leq 2^n(x^n + y^ n)$. It remains to bound the two moments in (ref). To bound the variance term, we re-express the symmetric statistic $\tilde{u}(x; D_{\mathsf{s_{\pi,l}}})$ in terms of its Hoeffding expansion. That is, we apply the following result from efron1981jackknife). \begin{lemma} Let $Z_1,\ldots,Z_b$ be a collection of $b$ independent and identically distributed real-valued random variables and $f:\mathbb{R}^b \to \mathbb{R}$ denote some symmetric function satisfying $\Var(f(Z_{[b]}))<\infty$. There exist functions $f_1,\ldots,f_b$ such that \begin{equation} f(Z_1,\ldots,Z_b) = \mathbb{E}[f(Z_1,\ldots,Z_b)] + \sum_{l = 1}^b \sum_{\mathsf{s}\in\mathcal{S}_{b,l}} f_l(Z_{\mathsf{s}}) , \end{equation} and all $2^{b} - 1$ random terms on the right-hand side of (ref) are mean-zero and uncorrelated. Moreover, the function $f_1(\cdot)$ is given by the H\'{a}jek projection $f_1(z) = \mathbb{E}[f(Z_1,\ldots,Z_n)\mid Z_1 = z]$. \end{lemma} In particular, (ref) implies that there exist functions $\tilde{u}^{(1)},\ldots,\tilde{u}^{(b)}$ such that \begin{equation} \tilde{u}(x; D_{\mathsf{s_{[b]}}}) = \sum_{l=1}^b \sum_{s \in \mathsf{s}_{b,l}} \tilde{u}^{(l)}(x; D_\mathsf{s}) , \end{equation} where all $2^{b} - 1$ random terms on the right-hand side of (ref) are mean-zero and uncorrelated. Consequently, we find that \begin{align} \Var\left( h(x^{(j)}; D_{\mathsf{s_{[b]}}}) \right) & = \Var\left( \tilde{u}(x; D_{\mathsf{s_{[b]}}}) - \sum_{i=1}^b \tilde{u}^{(1)}(x; D_{\mathsf{s_{[b]}}})\right) \nonumber \\ &= \Var\left( \sum_{l=2}^b \sum_{s \in \mathsf{s}_{b,l}} \tilde{u}^{(l)}(x; D_\mathsf{s}) \right) \nonumber\\ &\leq \Var\left( \sum_{l=1}^b \sum_{s \in \mathsf{s}_{b,l}} \tilde{u}^{(l)}(x; D_\mathsf{s}) \right) = \Var(\tilde{u}(x; D_{\mathsf{s_{[b]}}})) = \bar{\nu}^2 \end{align} To bound the higher order moment in (ref), we apply the following standard maximal inequality, which follows from a simple application of e.g., Lemma 5.5, song2019approximating. \begin{lemma} Let $Z_1, \ldots, Z_k$ denote a collection of centered real-valued random variables. If $\|Z_j\|_{\psi_1} \leq \varphi$ for all $j$ in $[k]$, then \begin{equation} \mathbb{E}\left[ \max_{j \in [k]} \vert Z_j \vert^q \right] \lesssim (2q\varphi \log(2k))^q . \end{equation} \end{lemma} The bound $\| h(x; D_{\mathsf{s_{[b]}}}) \|_{\psi_1} \leq (b+1)\phi$, follows from the triangle inequality and the fact that $\|\tilde{u}^{(1)}(x; D_i))\|_{\psi_1} \leq \phi$ by Jensen's inequality. Thus, (ref) implies that \begin{equation} \mathbb{E} \left[ \max_{l \in [n/b]} \left( h(x; D_{\mathsf{s_{\pi,l}}}) \right)^q \right] \lesssim \left(4 q b \phi \log(2n)\right)^q . \end{equation} Putting the pieces together, the bounds (ref) and (ref) imply that \begin{align} &\mathbb{E}\left[ \bigg\vert \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} h(x; D_{\mathsf{s}}) \bigg\vert^q \right] \nonumber \\ & \quad\leq C^{bq} q^{bq/2} \left(\frac{b}{n}\right)^{bq/2} \Bigg( \mathbb{E} \left[ \left(h(x; D_{\mathsf{s}})\right)^2 \right]^{q/2} +q^{q/2} \left(\frac{b}{n}\right)^{q/2} \mathbb{E} \left[ \max_{l \in [n/b]} \left( h(x; D_{\mathsf{s_{\pi,l}}}) \right)^q \right] \Bigg) \nonumber \\ & \quad \leq C^{bq} q^{bq/2} \left(\frac{b}{n}\right)^{bq/2} \Bigg(\bar{\nu}^{q} +q^{3q/2} b^q \left(\frac{b}{n}\right)^{q/2} \phi^q\log^q(n) \Bigg) \nonumber \\ & \quad = \left(C^{b} q^{b/2} \left(\frac{b}{n}\right)^{b/2} \left(\bar{\nu} +q^{3/2} b \left(\frac{b}{n}\right)^{1/2} \phi\log(n) \right)\right)^q , \end{align} Hence, an application of Markov's inequality and a union bound implies that \begin{align} & P\bigg\{ \bigg\| \left(\bar{U}_{n}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i)\right) \bigg\|_\infty \nonumber\\ & \quad\quad \geq \frac{1}{e} \left(C q\frac{b}{n}\right)^{b/2} \left( \bar{\nu} +q^{3/2} b \left(\frac{b}{n}\right)^{1/2} \phi\log(n) \right) \bigg\} \nonumber\\ & \quad \quad \leq \frac{d \exp(-q) \mathbb{E}\left[ \bigg\vert \frac{1}{N_b} \sum_{\mathsf{s} \in \mathcal{S}_{n,b}} h(x^{(j)}; D_{\mathsf{s}}) \bigg\vert^q \right]} {C^{bq} q^{qb/2} \left(\frac{b}{n}\right)^{qb/2} \left(\bar{\nu} +q^{3/2} b \left(\frac{b}{n}\right)^{1/2} \phi\log(n) \right)^q} \leq d \exp(-q) . \end{align} Through the choice $q = \log(dn)$, we find that \begin{align} & \bigg\| \left(\bar{U}_{n}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i)\right) \bigg\|_\infty \nonumber \\ & \quad \quad \leq \left(\frac{C b\log(dn)}{n}\right)^{b/2} \left(\bar{\nu} + b \left(\frac{b}{n}\right)^{1/2} \phi\log^2(dn)\right) \end{align} with probability greater than $1 - 1/n$, as required.\qed \section{Proof of (ref)} Proofs for the Lemmas stated in this appendix are given in (ref). Throughout, without loss, we will assume that there exists some constant $c$ such that \begin{equation} \frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n} + \delta_n \sqrt{\log(dn)} < c < 1 , \end{equation} as otherwise the desired bound (ref) is vacuous. It will suffice to show that \begin{flalign} & \sup_{z\in\mathbb{R}} \big\vert P\left\{\sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty < z \right\} - P\left\{\sqrt{n} \| \hat{\Lambda}_n^{-1/2}R^*_{n}(\bm{x}^{(d)}) \|_\infty < z \mid \mathbf{D}_n \right\} \big\vert \nonumber\\ & \quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{dn}} + \rho_n \end{flalign} with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$. To see this, let $\mathsf{cv}(\gamma)$ denote the $1-\gamma$ quantile of $\sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty$ for each $\gamma$ in $(0,1)$ and fix \begin{equation} \beta_{n,d} = \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{dn}} + \rho_n . \end{equation} Observe that (ref) implies that \begin{align} & P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R^*_{n}(\bm{x}^{(d)}) \|_\infty < \mathsf{cv}(\alpha-\beta_{n,d}) \mid \mathbf{D}_n \right\} \nonumber \\ & \geq P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty < \mathsf{cv}(\alpha-\beta_{n,d}) \right\} - \beta_{n,d} \geq 1-\alpha \end{align} and \begin{align} & P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R^*_{n}(\bm{x}^{(d)}) \|_\infty < \mathsf{cv}(\alpha+\beta_{n,d}) \mid \mathbf{D}_n \right\} \nonumber \\ & \leq P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty < \mathsf{cv}(\alpha+\beta_{n,d}) \right\} + \beta_{n,d} \leq 1-\alpha \end{align} each with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$. Thus, recalling that $\hat{\mathsf{cv}}(\alpha)$ denotes the $1-\alpha$ quantile of $\sqrt{n} \| \hat{\Lambda}_n^{-1/2}R^*_{n}(\bm{x}^{(d)}) \|_\infty$ conditioned on the data $\mathbf{D}_n$, the inequalities (ref) and (ref) imply that \begin{align} & P\left\{\mathsf{cv}(\alpha+\beta_{n,d}) < \hat{\mathsf{cv}}(\alpha) < \mathsf{cv}(\alpha-\beta_{n,d})\right\} \gtrsim 1- \left(\frac{\varphi^2\log^{3}(dn)}{\underline{\lambda}^2n}\right)^{1/2} - \rho_n \geq 1- \beta_{n,d} , \end{align} where the second inequality follows from the fact that the normalization (ref) implies that \begin{equation} \left(\frac{\varphi^2\log^{3}(dn)}{\underline{\lambda}^2n}\right)^{1/2} = \left(\frac{\varphi^2\log(dn)}{\underline{\lambda}^2n}\right)^{1/4} \left(\frac{\varphi^2\log^{5}(dn)}{\underline{\lambda}^2n}\right)^{1/4} \lesssim \left(\frac{\varphi^2\log^{5}(dn)}{\underline{\lambda}^2n}\right)^{1/4} . \end{equation} Consequently, as we can write \begin{align} P\left\{ \bm{\theta}_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}, \mathbf{D}_n) \right\} = P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty \leq \hat{\mathsf{cv}}(\alpha) \right\} \end{align} by definition, the inequality (ref) implies that \begin{align} & P\left\{ \bm{\theta}_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}, \mathbf{D}_n) \right\} \nonumber \\ & \leq P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty \leq \mathsf{cv}(\alpha-\beta_{n,d}) \right\} + \beta_{n,d} \lesssim 1- \alpha + \beta_{n,d} \end{align} and \begin{align} & P\left\{ \bm{\theta}_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}, \mathbf{D}_n) \right\} \nonumber \\ & \geq P\left\{ \sqrt{n} \| \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)}) \|_\infty \geq \mathsf{cv}(\alpha-\beta_{n,d}) \right\} -\beta_{n,d} \gtrsim 1- \alpha - \beta_{n,d} \end{align} respectively, as required. Hence, the remainder of the proof is devoted to establishing the probability bound (ref). To do this, we require some additional notation. Let $\mathcal{R}$ denote the set of hyper-rectangles in $\mathbb{R}^d$. Let $Z$ denote a centered Gaussian random vector with covariance matrix $\Var(\bar{u}(\bm{x}^{(d)}, D_i))$. Let $\Lambda$ be the diagonal matrix with components $\lambda_{j}^2 = \Var(\bar{u}(x^{(j)}, D_i))$. To verify (ref) it will suffice to establish that \begin{flalign} &\sup_{\mathsf{R} \in \mathcal{R}} \big\vert P\left\{\sqrt{n} \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \big\vert \nonumber \\ &\quad\quad \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{dn}} + \rho_n \end{flalign} and that \begin{flalign} &\sup_{\mathsf{R} \in \mathcal{R}} \big\vert P\left\{\sqrt{n} \hat{\Lambda}_n^{-1/2}R^*_{n}(\bm{x}^{(d)})\in\mathsf{R} \mid \mathbf{D}_n \right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \big\vert \nonumber \\ &\quad\quad \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{dn}} \end{flalign} with probability greater than $1-Cn^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) - \rho_n$. That is, the probability bound (ref) follows from (ref) and (ref) by considering hyper-rectangles of the form $\mathsf{R} = [-\infty \bm{1}_{d}, z \bm{1}_{d} ]$ and applying the triangle inequality. For the sake of exposition, we provide the details for the proofs of the upper bounds encoded in the absolute inequalities (ref) and (ref), respectively. In particular, fix a rectangle $\mathsf{R}=[a_{l},a_{u}]$ in $\mathcal{R}$, where $a_{l}$ and $a_{u}$ are vectors in $\mathbb{R}^{d}$ with $a_{l}\leq a_{u}$, interpreted componentwise, and define the enlarged rectangle $\mathsf{R}_{t}=[a_{l}-t\bm{1}_{d},a_{u}+t\bm{1}_{d}]$ for each $t>0$. We obtain upper bounds \begin{flalign} & P\left\{\sqrt{n} \hat{\Lambda}_n^{-1/2}R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \nonumber\\ & \quad\quad\leq \big\vert P\left\{ \sqrt{n} \Lambda^{-1/2}R_{n}(\bm{x}^{(d)})\in\mathsf{R}_{t}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} \big\vert\\ & \quad\quad\quad +\big\vert P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \big\vert\\ & \quad\quad\quad +P\left\{ \sqrt{n} \|(\Lambda^{-1/2}-\hat{\Lambda}_{n}^{-1/2})R_{n}(\bm{x}^{(d)})\|_{\infty}\geq t\right\} , \end{flalign} and similarly \begin{flalign} & P\left\{ \sqrt{n} \hat{\Lambda}_n^{-1/2}R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \nonumber\\ & \quad\quad\leq\vert P\left\{ \sqrt{n} \Lambda^{-1/2}R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}_{t}\mid\mathbf{D}_{n}\right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} \vert\\ & \quad\quad\quad+\vert P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \vert\\ & \quad\quad\quad +P\left\{ \sqrt{n} \|(\Lambda^{-1/2}-\hat{\Lambda}_{n}^{-1/2})R_{n}^{*}(\bm{x}^{(d)})\|_{\infty}\geq t\mid\mathbf{D}_{n}\right\} , \end{flalign} respectively. Analogous lower bounds will follow from the same argument, with the enlarged rectangle $\mathsf{R}_{t}$ replaced with the shrunken rectangle $[a_{l}+t\bm{1}_{d},a_{u}-t\bm{1}_{d}]$. It thereby remains to obtain suitable bounds for each term, (ref) through (ref). To bound the Gaussian approximation errors (ref) and (ref), we apply the following Theorem, which establishes a generic quantitative central limit for the statistic $R_{n}$ in addition to a generic quantitative conditional central limit theorem for the half-sample bootstrap. \begin{theorem} Suppose that the moment function $M(x;\theta_0, g_0)$ satisfies (ref) and Part (i) of (ref) and that (ref) hold. \textbf{(i)} The inequality \begin{flalign} \sup_{R\in\mathcal{R}}\sup_{P\in\mathbf{P}} \Big\vert P\left\{ \sqrt{n} R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{ Z\in\mathsf{R}\right\} \Big\vert & \lesssim \frac{\varphi^{1/2}}{\underline{\lambda}^{1/2}} \left(\frac{\log^{5}(dn)}{n}\right)^{1/4} + \delta_n\sqrt{\log{d}} + \rho_n \end{flalign} holds. \textbf{(ii)} If the bootstrap root is constructed with the Half-Sample bootstrap, then the inequality \begin{flalign} \sup_{R\in\mathcal{R}}\sup_{P\in\mathbf{P}} \Big\vert P\left\{ \sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} -P\left\{ Z\in\mathsf{R}\right\} \Big\vert & \lesssim \frac{\varphi^{1/2}}{\underline{\lambda}^{1/2}}\left(\frac{\log^{5}\left(dn\right)}{n}\right)^{1/4}+ \delta_n\sqrt{\log{d}} \end{flalign} holds with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$. \end{theorem} In particular, (ref) implies that \begin{align} & \big\vert P\left\{ \sqrt{n} \Lambda^{-1/2}R_{n}(\bm{x}^{(d)})\in\mathsf{R}_{t}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} \big\vert \nonumber \\ &= \big\vert P\left\{ \sqrt{n} R_{n}(\bm{x}^{(d)})\in \Lambda^{1/2} \mathsf{R}_{t}\right\} -P\left\{ Z\in \Lambda^{1/2} \mathsf{R}_{t}\right\} \big\vert \nonumber \\ &\quad\quad \lesssim \frac{\varphi^{1/2}}{\underline{\lambda}^{1/2}} \left(\frac{\log^{5}(dn)}{n}\right)^{1/4} + \delta_n\sqrt{\log{d}} + \rho_n \end{align} and analogously \begin{align} & \vert P\left\{ \sqrt{n} \Lambda^{-1/2}R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}_{t}\mid\mathbf{D}_{n}\right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} \vert \nonumber \\ & = \big\vert P\left\{ \sqrt{n} R^*_{n}(\bm{x}^{(d)})\in \Lambda^{1/2} \mathsf{R}_{t}\mid \mathbf{D}_{n}\right\} -P\left\{ Z\in \Lambda^{1/2} \mathsf{R}_{t}\right\} \big\vert \nonumber \\ & \quad\quad \lesssim \frac{\varphi^{1/2}}{\underline{\lambda}^{1/2}} \left(\frac{\log^{5}(dn)}{n}\right)^{1/4} + \delta_n\sqrt{\log{d}} + \rho_n \end{align} with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$, as $\Lambda^{1/2} \mathsf{R}_{t}$ is a hyper-rectangle. To bound the differences in the Gaussian probabilities ((ref)) and ((ref)), we apply the following anti-concentration inequality, stated in chernozhukov2017detailed and often referred to as Nazarov's inequality. \begin{lemma}[{chernozhukov2017detailed}] Let $Z=(Z_{j})_{j=1}^{d}$ be a centered Gaussian random vector in $\mathbb{R}^{d}$ such that $\mathbb{E}[Z_{j}^{2}]\geq c$ for all $j$ in $[d]$ and some constant $c$. For every $z\in\mathbb{R}^{d}$ and $t>0$, the inequality \[ P\left\{ Z\leq z+t\right\} -P\left\{ Z\leq z\right\} \lesssim\frac{t}{c}\sqrt{\log d} \] holds. \end{lemma} In particular, we have that \begin{equation} \vert P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}_{t}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}\right\} \vert\lesssim t\sqrt{\log d} \end{equation} for all $t>0$. Finally, we bound the terms ((ref)) and ((ref)) resulting from variance estimation. Observe that \begin{align} \|(\Lambda^{-1/2} - \hat{\Lambda}_n^{-1/2}) R_n(\bm{x}^{(d)})\|_{\infty} &= \|(I - \hat{\Lambda}_n^{-1/2}\Lambda^{1/2}) \Lambda^{-1/2} R_n(\bm{x}^{(d)})\|_{\infty} \nonumber\\ &\leq \sup_{j\in[d]} |\hat{\lambda}_{n,j}/\lambda_j - 1|\cdot \|\Lambda^{-1/2}R_n(\bm{x}^{(d)})\|_{\infty} \end{align} and analogously \begin{align} \|(\Lambda^{-1/2} - \hat{\Lambda}_n^{-1/2}) R^*_n(\bm{x}^{(d)})\|_{\infty} \leq \sup_{j\in[d]} |\hat{\lambda}_{n,j}/\lambda_j - 1|\cdot \|\Lambda^{-1/2}R^*_n(\bm{x}^{(d)})\|_{\infty} . \end{align} We give probability bounds for the terms on the right-hand-sides of (ref) and (ref). Observe that the Borell-TIS inequality adler2009random implies that \[ P\left\{ \|\Lambda^{-1/2}Z\|_{\infty}\geq C \sqrt{\log{dn}}\right\} \le n^{-1}~. \] Thus, (ref) implies that \begin{flalign} P\left\{ \|\Lambda^{-1/2}R_{n}(\bm{x}^{(dn)})\|_{\infty}\geq C \sqrt{\log{dn}}\right\} & \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{d}} + \rho_n \end{flalign} and that \begin{flalign} P\left\{ \|\Lambda^{-1/2}R^*_{n}(\bm{x}^{(d)})\|_{\infty}\geq C \sqrt{\log{dn}}\mid\mathbf{D}_{n}\right\} & \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{d}} \end{flalign} with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$. In turn, to bound the discrepancy between the bootstrap variance estimate $\hat{\lambda}_{n,j}$ and $\lambda_j$, we apply the following Lemma. \begin{lemma}Suppose that (ref) holds. If the bootstrap root is constructed with the Half-Sample bootstrap, then \begin{equation} P\left\{\sup_{j\in[d]} \bigg\vert \frac{\hat{\lambda}_{n,j}}{\lambda_{j}} - 1\bigg\vert \geq C \frac{\varphi^2}{\underline{\lambda}^2 n} \log^2(dn) + C\frac{1}{n}\delta_n^2 \right\} \lesssim 1-C(\rho_n+n^{-1}) . \end{equation} \end{lemma} Thus, by combining the bounds (ref) and (ref) with Lemma (ref), we find that \begin{flalign} &P\left\{ \|(\Lambda^{-1/2}-\hat{\Lambda}_{n}^{-1/2})R_{n}(\bm{x}^{(d)})\|_{\infty} \gtrsim \left(\frac{\varphi^2}{\underline{\lambda}^2 n}\log^2(dn) + \frac{1}{n}\delta_n^2\right)\sqrt{\log{dn}}\right\}\nonumber\\ & \quad\quad\quad\quad\quad\quad\quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{d}} + \rho_n . \end{flalign} Similarly, by combining the bounds (ref) and (ref) with Lemma (ref), we find that \begin{flalign} &P\left\{ \|(\Lambda^{-1/2}-\hat{\Lambda}_{n}^{-1/2})R_{n}^{*}(\bm{x}^{(d)})\|_{\infty} \gtrsim\left(\frac{\varphi^2}{\underline{\lambda}^2 n}\log^2(dn) + \frac{1}{n}\delta_n^2\right)\sqrt{\log{dn}}\mid\mathbf{D}_{n}\right\} \nonumber \\ & \quad\quad\quad\quad\quad\quad\quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log{d}} \end{flalign} with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$. To put the pieces together, recall the upper bounds (ref) and (ref). By setting \[ t =\left(\frac{\varphi^2}{\underline{\lambda}^2 n}\log^2(dn) + \frac{1}{n}\delta_n^2\right)\sqrt{\log{dn}}~, \] the bounds (ref), (ref), and (ref) imply that \begin{flalign} & P\left\{ \hat{\Lambda}_{n}^{-1/2}R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{ Z\in\mathsf{R}\right\} \nonumber\\ & \quad\quad\quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n \sqrt{\log(dn)} + \left(\frac{\varphi^2\log^2(dn)}{\underline{\lambda}^2 n} + \frac{1}{n}\delta_n^2\right)\log(dn) + \rho_n \nonumber \\ &\quad\quad\quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n \sqrt{\log(dn)} + \rho_n \end{flalign} where the second inequality follows from the normalization (ref). Similarly, the bounds (ref), (ref), and (ref) imply that \begin{flalign} & P\left\{ \hat{\Lambda}_{n}^{-1/2}R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} -P\left\{ Z\in\mathsf{R}\right\} \nonumber\\ & \quad\quad\quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} +\left(\frac{\varphi^2\log^2(dn)}{\underline{\lambda}^2 n} + \frac{1}{n}\delta_n^2\right)\log(dn) \nonumber \\ & \quad\quad\quad\quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n \sqrt{\log(dn)} + \rho_n \end{flalign} with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$. Hence, the upper bounds (ref) and (ref) verify the upper bounds encoded in (ref) and (ref), respectively. Again, the corresponding lower bounds follow from an analogous argument. \qed \section{Proof of (ref)} Proofs for Lemmas stated in this appendix are given in (ref). The result follows from an application of (ref). We begin by verifying the requisite assumptions. (ref) requires that the moment function $M(\cdot;\theta_0,g_0)$ is Neyman Orthogonal and satisfies (ref). Neyman Orthogonality is also imposed in the conditions of (ref), and so is satisfied. (ref) requires that the moment function $M(x;\theta_0,g_0)$ is twice continuously differentiable and has a Jacobian function \begin{equation} M^{(1)}(x; \theta, g) = \frac{\partial}{\partial \theta^\prime} M(\cdot;\theta_0,g_0) \mid_{\theta^\prime = \theta} \end{equation} that satisfies the bounds (ref) and (ref). Twice continuous differentiability follows immediately from (ref), which states that the moment function $m(\cdot;\theta,g)$ satisfies the linear representation \begin{equation} m(D_i;\theta, g) = m^{(1)}(D_i;g)\cdot \theta + m^{(2)}(D_i;g) , \end{equation} with bounded functions $m^{(1)}(\cdot;g)$ and $m^{(2)}(\cdot;g)$. In particular, the representation (ref) implies that the Jacobian can be written \begin{equation} M^{(1)}(x; \theta, g) = \mathbb{E}[ m^{(1)}(D_i;g)] \end{equation} and that the Hessian can be written \begin{equation} H(x; \theta, g) = \frac{\partial^2}{\partial^2 \theta^\prime} M(\cdot;\theta_0,g_0) \mid_{\theta^\prime = \theta} = 0 . \end{equation} Here, our ability to exchange integration and differentiation follows from boundedness of $m(\cdot;\theta, g)$. Consequently, the bounds (ref) and (ref) follow from the bounds (ref) and (ref) stated as Part (iii) of (ref). (ref) also requires that the centered empirical moment $\bar{M}_n(x; \theta, g)$ satisfy (ref). (ref) imposes the restriction that $\bar{M}_n(x; \theta, g)$ is twice continuously differentiable and that its Hessian \begin{align} \bar{H}_n(x;\theta, g)&= \frac{\partial^2}{\partial^2 \theta^\prime} \bar{M}_n(x;\theta^\prime, g_0) \vert_{\theta^\prime = \theta} \end{align} is uniformly bounded almost surely as $x$, $\theta$, and $g$ vary over their respective domains. Recall that, in our case, we have that \begin{equation} \bar{M}_n(x; \theta,g) = \sum_{i=1}^n \left( K(x,X_i)m(D_i; \theta, g) - \mathbb{E}\left[K(x,X_i) m(D_i; \theta, g)\right]\right) . \end{equation} Twice continuously differentiability again follows from (ref), i.e., the linear representation (ref). In turn, we have that $\bar{H}_n(x;\theta, g) = 0$ almost surely, verifying the uniform boundedness condition. Moreover, for future reference, the empirical Jacobian is given by \begin{equation} \bar{M}_n^{(1)}(x; g) = \sum_{i=1}^n \left( K(x,X_i)m^{(1)}(D_i; g) - \mathbb{E}\left[K(x,X_i) m^{(1)}(D_i; g)\right]\right) \end{equation} again by (ref). We now quantify the generic sequences $\delta_n^{u}$ and $\rho_n^{u}$ defined in (ref). Recall the normalized statistic $U_n(x)$ defined in (ref). By the definition (ref), this quantity can be written \begin{align} U_{n}(x) & = - \frac{1}{r} \sum_{q=1}^r u(x; D_{\mathsf{s}_q}, \xi_{\mathsf{s}}, \theta_0, g_0) ,\quad\text{where}\nonumber\\ u(x; D_\mathsf{s}, \xi_{\mathsf{s}}, \theta, g) &= M^{(1)}(x;g_0)^{-1}\sum_{i\in\mathsf{s}} \big( \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g)- \mathbb{E}\big[ \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) m(D_i; \theta, g)\big]\big) .\nonumber \end{align} Define the de-randomized kernel function and H\'{a}jek projection \begin{align} \tilde{u}(x; D) &= \mathbb{E}\left[u(x; D_{\mathsf{s}}, \xi_{\mathsf{s}}, \theta_0(x), g_0) \mid D_\mathsf{s} = D \right]\quad\text{and}\\ \tilde{u}^{(1)}(x; D) &= \mathbb{E}\left[u(x; D_{\mathsf{s}}, \xi_{\mathsf{s}}, \theta_0(x), g_0) \mid i\in\mathsf{s}, D_i = D \right] , \end{align} respectively, in addition to the quantities \begin{equation} \sigma^2_{b,j} = \Var(\tilde{u}(x^{(j)}, D_i))\quad\text{and}\quad \underline{\sigma}^2_b = \min_{j \in [d]} \sigma^2_{b,j} . \end{equation} The result below follows from an application of (ref). \begin{lemma}[Asymptotic Linearity] Suppose that the de-randomized kernel function (ref) is bounded by $\phi \geq 1$ almost surely and is invariant to permutations of its second argument. Suppose that $b$ and $r$ are chosen to satisfy $n\leq \sqrt{r} b$. There exists some constant $C_1 < 1$ such that if \begin{equation} \frac{(1+\|\theta_0(\bm{x}^{(d)})\|_\infty) \phi b\log(dn)}{n} \leq C_1 \quad\text{and}\quad b^{-C_2} \leq \underline{\sigma}^2_b \end{equation} for some positive constant $C_2$, then \begin{align} \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} \Big\| U_{n}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \Big\|_{\infty} \lesssim \left( \frac{(1+\|\theta_0(\bm{x}^{(d)})\|_\infty)^2 \phi^2 \log(dn)}{n \underline{\sigma}^2_b} \right)^{1/2} \end{align} with probability greater than $1 - C/n$ for all $b$ larger than $(7 + C_2) \log(n)$. \end{lemma} In particular, observe that (ref) implies that the de-randomized kernel function (ref) is bounded by $\phi$, up to a constant that depends only on $\mathbf{P}$. Permutation invariance again follows by assumption. We may assume that the first condition in (ref) holds, as otherwise the desired bound is vacuously true. The second condition in (ref) follows, with $C_2 = 1$, from incrementality. Consequently, by incrementality, the fact that $\|\theta_0(\bm{x}^{(d)})\|_\infty$ is bounded as $P$ varies over $\mathbf{P}$, and the assumption that $b$ greater than $C\log(n)$, we can choose \begin{equation} \delta_{n,u} = C \left( \frac{ b \phi^2 \log(dn)}{n} \right)^{1/2} \end{equation} and $\rho_{n,u}= Cn^{-1}$, respectively. Next, we quantify the sequences introduced in (ref). The Lemma below follows from arguments similar to those used in wager2018estimation and oprescu2019orthogonal. \begin{lemma}[Bias, Consistency, and Stochastic Equicontinuity] Suppose that the conditions of (ref) are satisfied. Moreover, suppose that there exists some constant $c<1$ such that the inequality \begin{equation} \frac{\phi^2 (1 + \| \theta(\bm{x}^{(d)})\|_\infty)^2 b\log^2(dn)}{n} < c \end{equation} holds. \textbf{(i)} The bound \begin{align} & \| \mathsf{Bias}_n(\bm{x}^{(d)};\theta(\bm{x}^{(d)}), \hat{g}_n) \|_\infty \lesssim (1 + \| \theta(\bm{x}^{(d)})\|_\infty) \varepsilon_{b} \end{align} holds. \textbf{(ii)} The bound \begin{align} &P \left\{ \| \bar{M}^{(1)}_n(\bm{x}^{(d)}; g_0)\|_\infty \geq C \left(\frac{b \phi^2 \log^2(dn)}{n}\right)^{1/2} \right\} \lesssim \frac{1}{n} \end{align} holds for each $b\geq2$. \textbf{(iii)} The bounds \begin{align} &P\left\{ \big\| \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), \hat{g}_n) - \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), g_0) \big\|_\infty \geq C \tau_{n,S} \right\} \lesssim \frac{1}{n} \quad\text{and}\\ &P\left\{ \big\| \bar{M}^{(1)}_n(\bm{x}^{(d)}; \hat{g}_n) - \bar{M}^{(1)}_n(\bm{x}^{(d)}; g_0) \big\|_\infty \geq C \tau_{n,S} \right\} \lesssim \frac{1}{n} ,\quad\text{where}\\ &\tau_{n,S} = \sqrt{\frac{b\log(dn)}{n}} \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon_b + \frac{b}{n}(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty) \phi \log^2(dn) .\nonumber \end{align} hold for each $b\geq2$. \textbf{(iv)} If, additionally, there exists some constant $c < 1$ such that \begin{equation} \varepsilon_{b}\leq c \quad\text{and}\quad \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \leq c , \end{equation} then the bound \begin{align} & P\left\{ \|\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)})\|_\infty \geq C \tau_{n,\theta} \right\} \lesssim \frac{1}{n} ,\quad\text{where}\nonumber\\ &\tau_{n,\theta} = \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + (1 + \| \theta_0(\bm{x}^{(d)})\|_\infty) \varepsilon_{b} + \sqrt{\frac{b}{n}}\log(dn)(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty)\phi\nonumber , \end{align} holds for each $b\geq2$. \end{lemma} Observe that the choice \[ \bar{u}(x, D_i) = b\cdot \tilde{u}(x, D_i)~, \] suggested by (ref) implies that \begin{equation} b \gtrsim \underline{\lambda}^2 = b^2 \underline{\sigma}^2_b \gtrsim b , \end{equation} where the first inequality is established in the Proof of (ref) and the second inequality follows from incrementality. We may assume that there exists some small constant $c$ such that \begin{equation} \varepsilon_n \leq c,\quad \delta_{n,g} \leq c,\quad\text{and}\quad \frac{b \phi^4\log^5(dn)}{n}\leq c , \end{equation} as otherwise the bound is vacuously true. Consequently, all of the conditions of (ref) are satisfied. Thus, (ref) indicates that we can set the various sequences defined in (ref) as \begin{align} \delta_{n,b} & = C \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} \varepsilon_{b} \lesssim \sqrt{\frac{n}{b}} \varepsilon_{b} ,\\ \delta_{n,m} & = C \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} \frac{b \phi^2 \log^2(dn)}{n} \lesssim \sqrt{\frac{b}{n}}\phi^2 \log^2(dn) \\ \delta_{n,\theta} & = C \left(\frac{n}{b^2 \underline{\sigma}^2_b}\right)^{1/4} \left(\left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon_{b} + \sqrt{\frac{b}{n}} \log(dn)\phi\right) ,\nonumber \\ & \lesssim \delta_{n,g} + \left(\frac{n}{b}\right)^{1/4} \varepsilon_{b} + \left(\frac{b}{n}\right)^{1/4} \log(dn)\phi , \quad\text{and}\\ \delta_{n,S} = \delta_{n,J} & = C \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \log^{1/2}(dn) + \varepsilon_{b} + C \sqrt{\frac{b}{n}}\log^2(dn)\phi . \end{align} with \begin{equation} \rho_{n,m} = \rho_{n,g} = \rho_{n,\theta} = \rho_{n,S} = \rho_{n,J} = C n^{-1} \end{equation} respectively, where we have repeatedly used the fact that $\|\theta_0(\bm{x}^{(d)})\|_\infty$ is bounded as $P$ varies over $\mathbf{P}$. With these results in place, we apply (ref). We begin by giving a suitable upper bound for the sequence \begin{equation} \delta_n = \delta^2_{n,g} + \delta^2_{n,\theta} + \delta^2_{n,m} + \delta_{n,B} + \delta_{n,S} + \delta_{n,u} + \underline{\lambda}^{1/2}n^{-1/4} \delta_{n,\theta}\left(\delta_{n,B} + \delta_{n,J}\right) \end{equation} introduced in the statement of (ref). Observe that \begin{align} \frac{\underline{\lambda}^{1/2}}{n^{1/4}}(\delta_{n,B} + \delta_{n,J}) \delta_{n,\theta} \lesssim \frac{b^{1/4}}{n^{1/4}}(\delta_{n,B} + \delta_{n,J}) \delta_{n,\theta} \lesssim \delta_{n,B} + \delta_{n,J} , \end{align} as otherwise the desired bound would be vacuous, where the first equality follows from (ref). Consequently, we find that, if $\delta_{n,J} = \delta_{n,S}$, then \begin{equation} \delta_n \lesssim \delta^2_{n,g} + \delta^2_{n,\theta} + \delta^2_{n,m} + \delta_{n,B} + \delta_{n,S} + \delta_{n,u} . \end{equation} Plugging in the choices specified above gives \begin{align} \delta_{n} & \lesssim \delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon_{b} + \sqrt{\frac{b}{n}} \log^{1/2}(dn) \phi + \frac{b}{n} \log^{4}(dn) \phi^4 \\ & + \delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon^2_{b} + \sqrt{\frac{b}{n}} \log^2(dn)\phi^2 \nonumber \\ & + \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \log^{1/2}(dn) + \varepsilon_{b} + \sqrt{\frac{b}{n}}\log^2(dn)\phi \nonumber \end{align} where the first line collects the terms associated with $\delta^2_{n,g}$, $\delta_{n,B}$, $\delta_{n,u}$, and $\delta^2_{n,m}$, the second line collects the term associated with $\delta^2_{n,\theta}$, and the final line collects the term associated with $\delta_{n,S}$. Immediately, we can simplify the bound (ref) to get \begin{align} \delta_{n} & \lesssim \delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon_{b} + \sqrt{\frac{ b}{n} \log^{4}(dn) \phi^4} + \frac{b}{n} \log^{4}(dn) \phi^4 \\ & + \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \log^{1/2}(dn) \nonumber \\ & \lesssim \delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon_{b} + \sqrt{\frac{ b}{n} \log^{4}(dn) \phi^4} + \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \log^{1/2}(dn) , \nonumber \end{align} where, to get the second inequality, we apply the normalization (ref). Now, observe that \begin{align} \left(\frac{b}{n}\right)^{1/4} \delta_{n,g}\log^{1/2}(dn) & \lesssim \left(\frac{b \log^{2}(dn)}{n}\right)^{1/2} + \delta^2_{n,g} . \end{align} Consequently, we find that \begin{align} \delta_{n} \lesssim \delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon_{b} + \sqrt{\frac{b}{n} \log^{4}(dn) \phi^4} \end{align} and that $\rho_{n} = Cn^{-1}$. Hence, the inequality (ref), the Orlicz-norm bound \begin{equation*} \| \bar{u}(x^{(j)}, D) \|_{\psi_1} \leq b\cdot (1 + |\theta(x^{(j)})|) \phi \lesssim b \phi , \end{equation*} and (ref) imply that \begin{align} & \sup_{P\in\mathbf{P}} \big\vert P\left\{ \theta_0(\bm{x}^{(d)}) \in \hat{\mathcal{C}}(\bm{x}^{(d)}) \right\} - (1 - \alpha)\big\vert \nonumber \\ & \lesssim \left( \frac{b \phi^2 \log^{5}\left(dn\right)}{n}\right)^{1/4} + \left(\delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon_{b}\right)\sqrt{\log{d}} + \frac{\sqrt{\log(d)}}{n} + \sqrt{\frac{b}{n} \log^{5}(dn) \phi^4} \nonumber\\ & \lesssim \left( \frac{b \phi^4 \log^{5}\left(dn\right)}{n}\right)^{1/4} + \left(\delta^2_{n,g} + \sqrt{\frac{n}{b}} \varepsilon_{b}\right)\sqrt{\log{d}} , \end{align} as required. \qed \section{Additional Results and Discussion} \subsection{A Small Survey of Heterogeneity Assessment} We conduct a small-scale survey of treatment effect heterogeneity estimation in applied economics. We review the 45 papers published in the \textit{American Economic Review} between January and June of 2023. We consider only the main text of each article. First, we categorize each paper according to whether it was empirical. Of the empirical papers, we determine whether any of the figures or tables display estimates of treatment effect heterogeneity. (We exclude intertemporal effect heterogeneity, e.g., event-studies). We then categorize each of the papers that display estimates of treatment effect heterogeneity according to whether their report is nonparametric, based on interacted linear regression, based on the interaction of treatment with binary covariates, or involves a structural model. We categorize 38 papers as empirical. Of these, 30 report treatment effect heterogeneity. Two papers report treatment effect heterogeneity nonparametrically. Nine papers use interacted linear regression. Seven papers use structural modeling. Twelve papers report coefficient on interactions of binary covariates. Many of these papers discretize a continuous covariate into a binary covariate, e.g., age into indicators for age above and below 50. \subsection{Binomial-Sample Bootstrap} Recall that the half-sample bootstrap root is given by \begin{equation} R^*_n(\mathbf{x}^{(d)}) = \hat{\theta}_\mathsf{h}(\mathbf{x}^{(d)}) - \hat{\theta}_n(\mathbf{x}^{(d)}) \end{equation} where $\mathsf{h}$ denotes a random element of $\mathcal{S}_{n,n/2}$ and $\hat{\theta}_n(\mathbf{x}^{(d)})$ denotes a version of the estimator $\hat{\theta}_n(\mathbf{x}^{(d)})$ evaluated on the data $D_\mathsf{h}$. Our theoretical analysis of the half-sample bootstrap is based on the observation that if the estimator $\hat{\theta}_n(\mathbf{x}^{(d)})$ admits a linear representation \begin{equation} \hat{\theta}_n(\mathbf{x}^{(d)}) = \frac{1}{n} \sum_{i=1}^n \bar{u}(\mathbf{x}^{(d)}; D_i) \end{equation} for some function $\bar{u}(\cdot;\cdot)$, then the root $R^*_n(\mathbf{x}^{(d)})$ admits the representation \begin{equation} R^*_n(\mathbf{x}^{(d)}) = \frac{1}{n} \sum_{i = 1}^n V_i (\bar{u}(\mathbf{x}^{(d)}, D_i) - \theta_0(\mathbf{x}^{(d)})) \end{equation} where $V_i$ is equal to 1 if $i$ is in $\mathsf{h}$ and is equal to $-1$ otherwise. The weights $V_1, \ldots, V_n$ are exchangeable Rademacher random variables. In this appendix, we discuss an alternative subsampling procedure that induces analogous weights that are fully independent. That is, for statistics that admit the linear representation (ref), the sampling procedure considered here is equivalent to the Rademacher bootstrap. We refer to this procedure as the “Binomial-Sample" bootstrap. The Binomial-Sample bootstrap root is given by \begin{equation} R^*_n(\mathbf{x}^{(d)}) = \frac{2Q_n}{n}( \hat{\theta}_\mathsf{s}(\mathbf{x}^{(d)}) - \hat{\theta}_n(\mathbf{x}^{(d)})) , \end{equation} where $Q_n$ is an independent random variable with a $\mathsf{Bin}(n,1/2)$ distribution and $\mathsf{s}$ denotes a random element of $\mathcal{S}_{n,Q_n}$. That is, $\mathsf{s}$ is a random set in $[n]$ of cardinality $Q_n$. If the estimator $\hat{\theta}_n(\mathbf{x}^{(d)})$ admits a linear representation (ref), then the root $R^*_n(\mathbf{x}^{(d)})$ admits the representation \begin{equation} R^*_n(\mathbf{x}^{(d)}) = \frac{1}{n} \sum_{i = 1}^n \tilde{V}_i (\bar{u}(\mathbf{x}^{(d)}, D_i) - \theta_0(\mathbf{x}^{(d)})) \end{equation} where $\tilde{V}_i$ is equal to 1 if $i$ is in $\mathsf{s}$ and is equal to $-1$ otherwise. The weights $\tilde{V}_1,\ldots,\tilde{V}_n$ are fully independent. A bound exactly analogous to (ref) holds if the confidence region formulated in (ref) is constructed with the Binomial-Sample bootstrap. This follows immediately from the following Theorem, which gives results analogous to (ref), Part (ii), and (ref), stated in the proof of (ref). Proofs are given in (ref). \begin{theorem} Suppose that the moment function $M(x;\theta_0, g_0)$ satisfies (ref) and Part (i) of (ref) and that (ref) hold. \textbf{(i)} If the bootstrap root is constructed with the Binomial-Sample bootstrap, then the inequality \begin{flalign} \sup_{R\in\mathcal{R}}\sup_{P\in\mathbf{P}} \Big\vert P\left\{ \sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} -P\left\{ Z_n\in\mathsf{R}\right\} \Big\vert & \lesssim \frac{\varphi^{1/2}}{\underline{\lambda}^{1/2}}\left(\frac{\log^{5}\left(dn\right)}{n}\right)^{1/4}+ \delta_n\sqrt{\log{d}} \end{flalign} holds with probability greater than $1-Cn^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) - \rho_n$. \textbf{(ii)} Moreover, in this case, we have that \begin{equation} P\left\{\sup_{j\in[d]} \bigg\vert \frac{\hat{\lambda}^2_{n,j}}{\lambda^2_{j}} - 1\bigg\vert \geq C \frac{\varphi^2}{\underline{\lambda}^2 n} \log^2(dn) + C\frac{1}{n}\delta_n^2 \right\} \lesssim 1-C(\rho_n+n^{-1}) . \end{equation} \end{theorem} \begin{remark} (ref) displays upper and lower confidence bounds for the CATE (ref) on post-treatment assets. These bounds are built with the confidence region formulated in (ref), implemented with the binomial-sample bootstrap. The qualitative and quantitative features of this figure are very similar to the features of (ref).$\blacksquare$ \end{remark} \subsection{Stochastic Equicontinuity without Sample Splitting} In the main text, we assume that the nuisance parameter estimator $\hat{g}_n$ is computed on a separate sample, independent of the data $\mathbf{D}_n$. In this section, we give additional conditions under which the nuisance parameter estimator $\hat{g}_n$ can be computed on the same data used to construct the estimator $\hat{\theta}_n(\bm{x}^{(d)})$. Our analysis builds on a similar result given in chen2022debiased, who give a pointwise analysis of unconditional moment estimators. By contrast, we give a high-dimensional analysis of conditional moment estimators constructed with subsampled kernels. In the proof of (ref), the statistical independence of the estimator $\hat{g}_n$ and the data $\mathbf{D}_n$ is only needed in the proof of (ref), Part (iii). Thus, our objective is to prove an analogous result, without imposing this restriction. In particular, consider the functions \begin{flalign*} a(x;D_{i},D_{\mathsf{s}},g) & =\kappa(x,X_{i};D_{\mathsf{s}})m(D_{i},g)\quad\text{and}\quad A(x;g)=\mathbb{E}\left[\kappa(x,X_{i};D_{\mathsf{s}})m(D_{i},g)\right], \end{flalign*} where $\kappa(\cdot,\cdot;D_{\mathsf{s}})$ is a kernel function and $m(D_{i},g)$ is a moment function. We are interested in the complete, deterministic, $U$-statistic \begin{flalign} & \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}F(\bm{x}^{(d)};D_{\mathsf{s}},\hat{g}_{n}) \end{flalign} where the kernel function $F(x;D_{\mathsf{s}},g)$ is given by \begin{flalign*} F(x;D_{\mathsf{s}},g) & =\sum_{i\in\mathsf{s}}f(x;D_{i},D_{\mathsf{s}},g),\\ f(x;D_{i},D_{\mathsf{s}},g) & =a(x;D_{i},D_{\mathsf{s}},g)-a(x;D_{i},D_{\mathsf{s}},g_{0})-\left(A(x;g)-\mathbb{E}\left[A(x;g_{0})\right]\right), \end{flalign*} and $\hat{g}_{n}$ is some estimator of the nuisance parameter $g_{0}$ computed with the data $\mathbf{D}_{n}$. That is, for the sake of simplicity, we have restricted our attention to complete, deterministic subsampled kernel estimators. Incomplete, random, subsampled kernel estimators can be accommodated through the same methods applied repeatedly throughout the arguments supporting (ref). To state our additional conditions, we require some additional notation. Let $\mathbf{D}_{n}^{\prime}=(D_{i}^{\prime})_{i=1}^{n}$ denote an independent copy of $\mathbf{D}_{n}$. Let $\hat{g}_{n}^{(-\mathsf{s})}$ denote a version of the estimator $\hat{g}_{n}$ formed with all observations in $\mathbf{D}_{n}$, except that the observations $D_{\mathsf{s}}$ are replaced by the observations $D_{\mathsf{s}}^{\prime}$. Define the norm \[ \|g-\tilde{g}\|_{2q,\infty}=\max_{k\in[h]}\max_{j\in[d]}\left(\mathbb{E}\left[\left(g^{(k)}(Z_{i})-\tilde{g}^{(k)}(Z_{i})\right)^{2q}\mid X_{i}=x^{(j)}\right]\right)^{1/2q} \] for any nuisance parameters $g=(g^{(k)})_{k=1}^{h}$ and $\tilde{g}=(\tilde{g}^{(k)})_{k=1}^{h}$ in the space $\mathcal{G}$ and any positive integer $q>0$. First, we require that moment $m(D_{i},g)$ satisfies a pair of smoothness conditions. \begin{assumption} Define the higher-order variogram \[ V^{(q)}(x;g,g^{\prime})=\mathbb{E}\left[\left(m(D_{i},g)-m(D_{i},g^{\prime})\right)^{2q}\mid X_{i}=x\right] \] for each $g$ and $g^{\prime}$ in $\mathcal{G}$. For each integer $q>0$, the Lipschitz condition \begin{equation} \vert V^{(q)}(x;g,g^{\prime})-V^{(q)}(x^{\prime};g,g^{\prime})\vert\lesssim\|x-x^{\prime}\|_{\infty}^{q} \end{equation} holds for each $x$ and $x^{\prime}$ in $\mathcal{X}$ and the mean-square continuity condition \begin{equation} \sup_{j\in[d]}V^{(q)}(x^{(j)};g,g^{\prime})\lesssim\|g-g^{\prime}\|_{2q,\infty}^{2q} \end{equation} holds for each $g$ and $g^{\prime}$ in $\mathcal{G}$. \end{assumption} (ref) is a higher-order analogue to Part (ii) of (ref). Second, we impose the following higher-order moment stability conditions. \begin{assumption} There exists some increasing sequence $\gamma(q)$, which may depend on the parameters $n$, $b$, and $d$, such that the $L_{q}$-norm stability bounds \begin{flalign} \|m(D_{1},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\|_{q} & \lesssim q\gamma(q)\frac{b}{n}\quad\text{and}\\ \|m(D_{1}^{\prime\prime},\hat{g}_{n})-m(D_{1}^{\prime\prime},\hat{g}_{n}^{(-[b])})\|_{q} & \lesssim q\gamma(q)\frac{b}{n} \end{flalign} hold, where $D_{1}^{\prime\prime}$ is an independent copy of $D_{1}$. \end{assumption} (ref) quantifies the sensitivity of the moment $m(D_{1},\hat{g}_{n})$ to re-sampling $b$ elements of the data $\mathbf{D}_n$. Analogous conditions are studied in abou2019exponential and chen2022debiased. We discuss moments and estimators that satisfy (ref) in (ref). The following Theorem gives a suitable large deviation bound for the U-statistic (ref). \begin{theorem} Suppose that the kernel $\kappa(x,X_{i};D_{\mathsf{s}})$ satisfies (ref) and has shrinkage rate $\varepsilon_{b}$, that the moment function $m(D_{i},g)$ satisfies (ref), and that the nuisance parameter estimator satisfies the high-order moment bound \begin{equation} \mathbb{E}\left[\|\hat{g}_{n}-g_{0}\|_{q,\infty}^{q}\right]\lesssim\left(q\left(\frac{b}{n}\right)^{1/4}\delta_{n,g}\right)^{q} \end{equation} for some sequence $\delta_{n,g}$. If (ref) holds, then \begin{flalign} \bigg\|\frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}F(\bm{x}^{(d)};D_{\mathsf{s}},\hat{g}_{n})\bigg\|_{\infty} & \lesssim\sqrt{\frac{b}{n}}\left(\left(\frac{b}{n}\right)^{1/4}\delta_{n,g} +\gamma_{n,b}^{\star}\right)\log(dn) +\varepsilon_{b} \\ & \quad+\frac{b}{n}\gamma_{n,b}^{\star}\log^2(dn)\nonumber \end{flalign} with probability greater than $1-Cn^{-1}$, where $\gamma_{n,b}^{\star}=\gamma(2\lceil\log(dn)\rceil)b^{1/\log(dn)}$. \end{theorem} \begin{remark} Compare the rate (ref) to the rate given in (ref), Part (iii). The rates match, up to the additional term $\gamma_{n,b}^{\star}$. Thus, under the conditions of (ref), if the quantity $\gamma_{n,b}^{\star}$ decreases to zero as $n$ and $b$ increase, a result analogous to (ref) will hold. We discuss conditions under which $\gamma_{n,b}^{\star}$ decreases in the (ref).$\blacksquare$ \end{remark} \begin{remark} (ref) follows from an argument similar to the argument developed in chen2022debiased. There is one important difference. chen2022debiased apply a “double-centering trick,” due to kumar2013near, to get a variance bound. To obtain a high-dimensional bound, with a logarithmic dependence on the dimension $d$, we replace this step with an application of a generalized Efron-Stein inequality, due to boucheron2005moment.$\blacksquare$ \end{remark} \subsubsection{Verifying (ref)} In this section, we study conditions under which (ref) is satisfied. First, we impose an additional smoothness condition on the moment function $m(D_{i},g)$. Throughout, we write the nuisance parameter estimator $\hat{g}_{n}$ as $\hat{g}_{n}=(\hat{g}_{k,n})_{k=1}^{h}$. \begin{assumption} The inequalities \begin{equation} \|m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\|_{2q}\lesssim\max_{k\in[h]}\mathbb{E}\left[\sup_{z\in\mathcal{Z}}\left(\hat{g}_{k,n}(z)-\hat{g}_{k,n}^{(-[b])}(z)\right)^{2q}\right]^{1/2q} \end{equation} and \begin{equation} \|m(D_{i}^{\prime\prime},\hat{g}_{n})-m(D_{i}^{\prime\prime},\hat{g}_{n}^{(-[b])})\|_{2q}\lesssim\max_{k\in[h]}\mathbb{E}\left[\mathbb{E}\left[\left(\hat{g}_{k,n}(Z_{i}^{\prime})-\hat{g}_{k,n}^{(-[b])}(Z_{i}^{\prime})\right)^{2q}\mid D_{i}^{\prime}\right]\right]^{1/2q} \end{equation} hold for any positive integer $q$. \end{assumption} The bound ((ref)) stipulates that the higher-order moments of $m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})$ are smaller than the supremum of the higher order-moments of $\hat{g}_{k,n}(z)-\hat{g}_{k,n}^{(-[b])}(z)$ over the space $\mathcal{Z}$. The bound ((ref)) is weaker, as we only need to control the distance $\hat{g}_{k,n}(z)-\hat{g}_{k,n}^{(-[b])}(z)$ at $z=Z_{i}^{\prime}$. Bounds of the form specified by (ref) are satisfied by many standard choices of moment function. For example, consider the moment function \begin{equation} m(D_{i},g)=\mu(Z_{i},1)-\mu(Z_{i},0)+\beta(W_{i},Z_{i})(Y_{i}-\mu(Z_{i},W_{i})) \end{equation} where the nuisance function $g$ collects the moment parameters $g=(\mu,\beta)$. Observe that (ref) is analogous to the moment function (ref) used as a running example in the main text. \begin{lemma} If the quantities \begin{equation} \vert\hat{\beta}_{n}(W_{i},Z_{i})\vert\quad\text{and}\quad\vert Y_{i}-\hat{\mu}_{n}(Z_{i},W_{i})\vert \end{equation} are bounded almost surely, then it holds that \begin{flalign} & \|m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\|_{2p}\nonumber \\ & \lesssim\max\bigg\{\max_{w\in\left\{ 0,1\right\} }\left\{ \|\hat{\mu}_{n}(w,Z_{i})-\hat{\mu}_{n}^{(-[b])}(w,Z_{i})\|_{2p}\right\} ,\\ & \quad\quad\quad\quad\quad\quad\|\hat{\mu}_{n}(W_{i},Z_{i})-\hat{\mu}_{n}^{(-[b])}(w,Z_{i})\|_{2p},\hat{\beta}_{n}(W_{i},Z_{i})-\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i})\|_{2p}\bigg\},\quad\text{and}\nonumber \\ & \|m(D_{i}^{\prime\prime},\hat{g}_{n})-m(D_{i}^{\prime\prime},\hat{g}_{n}^{(-i)})\|_{2p}\nonumber \\ & \lesssim\max\bigg\{\max_{w\in\left\{ 0,1\right\} }\left\{ \|\hat{\mu}_{n}(w,Z_{i}^{\prime\prime})-\hat{\mu}_{n}^{(-[b])}(w,Z_{i}^{\prime\prime})\|_{2p}\right\} ,\\ & \quad\quad\quad\quad\quad\quad\|\hat{\mu}_{n}(W_{i},Z_{i}^{\prime\prime})-\hat{\mu}_{n}^{(-[b])}(w,Z_{i}^{\prime\prime})\|_{2p},\hat{\beta}_{n}(W_{i},Z_{i}^{\prime\prime})-\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i}^{\prime\prime})\|_{2p}\bigg\},\nonumber \end{flalign} respectively, where $D_{i}^{\prime\prime}$ is an independent copy of $D_{i}$ and $Z_{i}^{\prime\prime}$ is an independent copy of $Z_{i}$. \end{lemma} Now, we consider the higher-order stability of the nuisance parameter estimator $\hat{g}_{n}(z)$. For the sake of simplicity, we assume that $\hat{g}_{n}(z)$ is scalar valued and takes the form of a complete, deterministic, $U$-statistic \begin{equation} \hat{g}_{n}(z)=\frac{1}{N_{b^{\prime}}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b^{\prime}}}u_{z}(D_{\mathsf{s}}), \end{equation} where $u_{z}(D_{\mathsf{s}})$ is some deterministic kernel function of order $b^{\prime}$. Again, let $\hat{g}^{(-[b])}_{n}(z)$ be constructed analogously to $\hat{g}_{n}(z)$ using the $\mathbf{D}_{n}$, but replacing $D_{[b]}$ with an independent copy $D_{[b]}^{\prime}$. \begin{lemma} Suppose that the kernel function $u_{z}(D_{\mathsf{s}})$ satisfies the bound $\vert u_{z}(D_{\mathsf{s}})\vert\leq\phi$ almost surely for each $z$ in $\mathcal{Z}$. If the estimator $\hat{g}_{n}(z)$ is given by (ref), then it holds that \begin{flalign} \|\hat{g}_{n}(z)-\hat{g}^{(-[b])}_{n}(z)\|_{q} & \lesssim\frac{\sqrt{b^{\prime}b}}{n}q\phi+q^{3/2}\phi^{2}n\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\log(n). \end{flalign} for each $z$ in $\mathcal{Z}$. \end{lemma} \begin{remark} Under (ref), (ref) implies that \begin{equation} \|m(D_{1}^{\prime\prime},\hat{g}_{n})-m(D_{1}^{\prime\prime},\hat{g}_{n}^{(-[b])})\|_{q}\lesssim\frac{\sqrt{b^{\prime}b}}{n}q\phi+q^{3/2}\phi^{2}n\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\log(n). \end{equation} In terms of the notation introduced in (ref), the bound (ref) implies that \[ \gamma(q)=\sqrt{\frac{b^{\prime}}{b}}\phi+\frac{q^{1/2}n^{2}\phi^{2}}{b}\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\log(n). \] At the choice $q=\log(dn)$, invoked in (ref), and assuming that $b^{\prime}=o(n)$ by at least a logarithmic factor, we have that \[ \gamma(\log(dn))\lesssim\frac{b^{\prime}}{b}\phi. \] Consequently, (ref) suggests that the bound (ref) is $o_p(\sqrt{b/n})$ if $b^{\prime}=o(b)$.$\blacksquare$ \end{remark} \begin{remark} (ref) entails two stability bounds, (ref) and (ref). Under (ref), (ref) quantifies the sequence $\gamma(q)$ introduced on the right-hand side of (ref). By the discussion above, if $b^{\prime}=o(b)$, this sequence is sufficiently small for a result analogous to (ref), Part (iii) to hold. Handling the bound (ref) requires developing a more refined argument. In particular, under (ref), to bound the right-hand-side of (ref), we must account for the supremum over the space $\mathcal{Z}$ within the expectation. Such a generalization should be reasonably straightforward, potentially through a chaining argument. Roughly speaking, we should expect the supremum to contribute a term like $\log(p)$ to the stability, where $p$ is the dimension of $\mathcal{Z}$. In this case, in settings where $p$ is not high-dimensional, this would not generate any issues. This should be contrasted with analogous supremums taken over the parameter space of a decision tree (which are generated if stochastic equicontinuity is controlled with a union bound). Here the dimension of the parameter space is roughly $2^{\text{Depth of Tree}}$, which may large enough to make a material difference. However, in order to operationalize this intuition, we would need to place further restrictions sufficient for the smoothness of the H\'{a}jek projection of $\hat{g}_n(z)$. We leave this for future work. $\blacksquare$ \end{remark} \subsection{Cross-Fitting} (ref) holds under the assumption that the nuisance parameter estimator $\hat{g}_n$ is computed on a data set that is statistically independent of the data used to construct the confidence region $\hat{\mathcal{C}}(\bm{x}^{(d)})$. This condition can be achieved by splitting the data into two parts, at the cost of reducing statistical precision and introducing randomness independent of the observed data. In (ref), we introduce further restrictions that allow the nuisance parameter estimator $\hat{g}_n$ and the confidence region $\hat{\mathcal{C}}(\bm{x}^{(d)})$ to be computed using the same data. However, in practice, these conditions are more stringent, and it may be unclear whether they are satisfied for a given nuisance parameter estimator $\hat{g}_n$. In this section, we detail an alternative procedure for constructing a confidence region similar to the confidence region specified in (ref), whose validity will hold under the same conditions imposed in (ref). The procedure is based on a cross-fit estimator. To introduce this estimator, we require some additional notation. Let $\mathcal{R}_{n,k}$ denote the set of partitions of $[n]$ into $k$ equally sized and mutually exclusive subsets. That is, for each $\mathsf{r} = (\mathsf{s}_1,\ldots,\mathsf{s}_k)$ in $\mathcal{R}_{n,k}$, the sets $\mathsf{s}_1,\ldots,\mathsf{s}_k$ are mutually exclusive, have union equal to $[n]$, and are each of size $n/k$. Throughout, we set $q=n/k$ and let $\tilde{\mathsf{s}}$ denote the complement of the set $\mathsf{s}$ in $[n]$. Finally, for each subset $\mathsf{s}$ of $[n]$, let $\hat{g}_{\mathsf{s}}$ denote a version of the nuisance parameter estimator $\hat{g}_n$ computed with the data $D_{\mathsf{s}}$. The cross-fit estimator is computed as follows. For each subset $\mathsf{s}$ in $\mathcal{S}_{n,q}$, let the estimator $\hat{\theta}_\mathsf{s}(\bm{x}^{(d)})$ be constructed by solving the empirical conditional moment equality (ref) using the data $D_{\mathsf{s}}$ and the nuisance parameter estimator $\hat{g}_{\tilde{\mathsf{s}}}$. That is, the nuisance parameter estimator is computed on the sample of data whose indices are not in $\mathsf{s}$. The $k$-fold cross-fit estimator is given by \begin{equation} \hat{\theta}_{\mathsf{r}}(\bm{x}^{(d)}) = \frac{1}{k} \sum_{\mathsf{s} \in \mathsf{r}} \hat{\theta}_\mathsf{s}(\bm{x}^{(d)}) , \end{equation} where $\mathsf{r}$ denotes a random element of $\mathcal{R}_{n,k}$. It is somewhat unclear how to implement the half-sample bootstrap with the estimator $\hat{\theta}_\mathsf{r}(\bm{x}^{(d)})$. We propose the following computationally efficient variant. Let $\mathcal{H}(\mathsf{s})$ denote the set of half-samples of the set $\mathsf{s}$, i.e., the set of subsets of $\mathsf{s}$ that contain exactly half of its elements. We say that the collection $\mathsf{H} = (\mathsf{h}_l)_{l=1}^k$ is a half-sample of the partition $\mathsf{r} = (\mathsf{s}_l)_{l=1}^k$ if each $\mathsf{h}_l$ is an element of $\mathcal{H}(\mathsf{s}_l)$, i.e., if each $\mathsf{h}_l$ is a half-sample of $\mathsf{s}_l$. For each $l$ in $[k]$, let the estimator $\hat{\theta}_\mathsf{\mathsf{h}_l}(\bm{x}^{(d)})$ be constructed by solving the empirical conditional moment equality (ref) using the data $D_{\mathsf{h}_l}$ and the nuisance parameter estimator $\hat{g}_{\tilde{\mathsf{s}}_l}$, i.e., using the same nuisance parameter estimates used to construct (ref). The half-sample $k$-fold cross-split estimator is given by \begin{equation} \hat{\theta}_{\mathsf{H}}(\bm{x}^{(d)}) = \frac{1}{k} \sum_{\mathsf{h} \in \mathsf{H}} \hat{\theta}_\mathsf{h}(\bm{x}^{(d)}) . \end{equation} In this way, once the $k$-fold cross-fit estimator (ref) has been computed, the nuisance parameter estimates $\hat{g}_{\tilde{\mathsf{s}}_1}, \ldots, \hat{g}_{\tilde{\mathsf{s}}_k}$ do not need to be recomputed to construct the half-split $k$-fold cross-split estimator (ref). Simultaneous confidence intervals can then be constructed analogously to the intervals introduced in (ref) by using the half-sample $k$-fold cross-split bootstrap root \begin{equation} R^*_n(\bm{x}^{(d)} = \hat{\theta}_{\mathsf{H}}(\bm{x}^{(d)}) - \hat{\theta}_{\mathsf{r}}(\bm{x}^{(d)}) \end{equation} in the place of the half-sample bootstrap root (ref). In other words, when implementing the half-sample bootstrap based on an estimator constructed with $k$-fold cross-fitting, one can avoid recomputing nuisance parameters if the half-samples are “stratified” across the $k$-folds. The error bound given in (ref) will generalize to the confidence region based on the bootstrap root (ref) through a straightforward argument, so long as $k$ is bounded. (ref) and (ref) gives versions of (ref) and (ref), constructed using the $2$-fold cross-split estimator (ref) and the $2$-fold cross-split bootstrap root (ref). The estimates and confidence bounds are quantitatively and qualitatively very similar to the estimates and confidence bounds reported in the main text. (ref) displays results for the simulation presented in (ref), analogous to (ref), for the $2$-fold cross-split estimator (ref) and the $2$-fold cross-split bootstrap root (ref). The confidence region is very slightly anti-conservative, so long as $b/n$ is decreasing as $n$ increases. The measurements of bias and variance exhibit patterns very similar to the patterns displayed in (ref). \subsection{Proof of (ref)} To ease notation, we drop the dependence on $x^{(j)}$. The argument will follow by first re-expressing the $U$-statistic of interest in terms of its Hoeffding expansion, stated in the proof of (ref). In particular, (ref) implies that there exist functions $u^{(1)}(\cdot),\ldots,u^{(b)}(\cdot)$ such that \begin{equation} u(D_{[b]}) = \sum_{l = 1}^b \sum_{\mathsf{s}\in\mathcal{S}_{b,l}} u^{(l)}(D_{\mathsf{s}}) \end{equation} and that all $2^{b} - 1$ terms on the right-hand side of (ref) are mean-zero and uncorrelated. Thus, we have that \begin{equation} \Var(u(D_{[b]})) = \sum_{l=1}^b {b \choose l} \Var(u^{(l)}(D_{[l]})) \end{equation} and that thereby \begin{equation} {b \choose l} \Var(u^{(l)}(D_{[l]})) \leq \Var(u(D_\mathsf{[b]})) . \end{equation} Moreover, again by (ref), we can write \begin{align} \frac{1}{N_b} \sum_{\mathsf{s} \in \mathsf{S}_{n,b}} u(D_\mathsf{s}) & = {n \choose b}^{-1} \sum_{\mathsf{s} \in \mathsf{S}_{n,b}} \sum_{l = 1}^b \sum_{\mathsf{r}\in\mathcal{S}_{l}(\mathsf{s})} u^{(l)}(D_{\mathsf{r}}) \\ & = \sum_{l = 1}^b {n \choose b}^{-1} {n - l \choose b - l} \sum_{r \in \mathcal{S}_{n,l}} u^{(l)}(D_{\mathsf{r}}) \nonumber\\ & = \sum_{l = 1}^b {b \choose l} {n \choose l}^{-1} \sum_{\mathsf{r}\in\mathcal{S}_{n,l}} u^{(l)}(D_{\mathsf{r}}) ,\nonumber \end{align} where $\mathcal{S}_{l}(\mathsf{s})$ denotes the set of $\mathsf{s}$ of cardinality $l$. Thereby, by the mutual orthogonality of the terms in the Hoeffding expansion (ref), it holds that \begin{align} \Var\left(\frac{1}{N_b} \sum_{\mathsf{s} \in \mathsf{S}_{n,b}} u(D_\mathsf{s}) - \frac{b}{n} \sum_{i=1}^n u^{(1)}(D_i) \right) & = \Var\left(\sum_{l = 2}^b {b \choose l} {n \choose l}^{-1} \sum_{\mathsf{r}\in\mathcal{S}_{n,l}} u^{(l)}(D_{\mathsf{r}}) \right)\nonumber\\ & = \sum_{l=2}^b {b \choose l}^{2} {n \choose l}^{-2} \sum_{\mathsf{r}\in\mathcal{S}_{n,l}} \Var(u^{(l)}(D_{\mathsf{r}}))\nonumber\\ & = \sum_{l=2}^b {b \choose l}^2 {n \choose l}^{-1} \Var(u^{(l)}(D_{[l]})) . \end{align} Consequently, we have that \begin{align} \Var\left(\frac{1}{N_b} \sum_{\mathsf{s} \in \mathsf{S}_{n,b}} u(D_\mathsf{s}) - \frac{b}{n} \sum_{i=1}^n u^{(1)}(D_i) \right)\nonumber & = \sum_{l=2}^b {b \choose l}^2 {n \choose l}^{-1} \Var(u^{(l)}(D_{[l]}))\nonumber\\ & \leq \Var(u(D_{[b]})) \sum_{l=2}^b {b \choose l} {n \choose l}^{-1} \nonumber\\ & \leq \Var(u(D_{[b]})) \sum_{l=2}^b \left(\frac{b}{n}\right)^l \nonumber\\ & \leq \Var(u(D_{[b]})) \left(\frac{b}{n}\right)^2\left(1 - \frac{b}{n}\right)^{-1} , \end{align} where the first inequality follows from the inequality (ref) and the third equality follows from the geometric series formula. Now, consider the decomposition \begin{align} \sqrt{\frac{n}{\sigma^2_{b,j} b^2}} \left(\frac{1}{N_b} \sum_{\mathsf{s} \in \mathsf{S}_{n,b}} u(D_\mathsf{s})\right) &= \sqrt{\frac{1}{\sigma^2_{b,j} n}} \sum_{i=1}^n u^{(1)}(D_\mathsf{s}) \\ &- \sqrt{\frac{n}{\sigma^2_{b,j} b^2}} \left(\frac{1}{N_b} \sum_{\mathsf{s} \in \mathsf{S}_{n,b}} u(D_\mathsf{s}) - \frac{b}{n} \sum_{i=1}^n u^{(1)}(D_i) \right) . \end{align} The normalization $\nu^2_j/n\sigma^2_{b,j} \to 0$, the bound (ref), and Chebychev's inequality imply that the term (ref) is $o_p(1)$ as $n\to\infty$. The result then follows by applying the central limit theorem to the term (ref).\qed \subsection{Proof of (ref), Part (i)} Observe that \begin{align} \sqrt{\frac{n}{b^2}} \Sigma^{-1/2} \bar{U}_{n,b}(\bm{x}^{(d)}) & = \sqrt{\frac{1}{n}} \sum_{i=1}^n \Sigma^{-1/2} u^{(1)}(\bm{x}^{(d)}; D_i)\\ & + \sqrt{\frac{n}{b^2}} \Sigma^{-1/2} \left(\bar{U}_{n,b}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n u^{(1)}(\bm{x}^{(d)}; D_i)\right) . \end{align} (ref) implies that \begin{align} \Big\| \sqrt{\frac{1}{n}} \sum_{i=1}^{n} \Sigma^{-1/2} \tilde{u}^{(1)}(\bm{x}^{(d)}, D_i) \Big\|_{\infty} \lesssim \log^{1/2}(dn) + \frac{\phi\log^2(dn)}{\underline{\sigma}_b n^{1/2}} \end{align} with probability greater than $1-C/n$. We show in the proof of (ref) that there exists a constant $C_1$ such that if $b\log(dn)/n <C_1 < 1$, then (ref) implies that (ref) is smaller than $\phi n^{-1}$, with probability greater than $1-Cn^{-1}$, so long as $b\gtrsim C_2 \log(n)$. The result then follows by aggregating these two bounds.\qed \subsection{Proof of (ref), Part (ii)} Fix a rectangle $\mathsf{R} = [a_l, a_u]$ in $\mathcal{R}$, where $a_l$ and $a_u$ are vectors in $\mathbb{R}^d$ with $a_l\leq a_u$, interpreted componentwise. Define the enlarged rectangle $\mathsf{R}_t=[a_l-t\bm{1}_d,a_u+t\bm{1}_d]$ for each $t>0$. Define the normalized quantity \begin{equation} \hat{u}^{(1)}(\bm{x}^{(d)}; D_i) = \Sigma^{-1/2}\tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) . \end{equation} For reasons of space, we give the details of the proof of the upper bound encoded in (ref). The lower bound will follow from an analogous argument, and we note the differences where they occur. Observe that the decomposition (ref) implies the upper bound \begin{align} & P\left\{\sqrt{\frac{n}{b^2}}\Sigma^{-1/2} \bar{U}_{n,b}(\bm{x}^{(d)}) \in \mathsf{R}\right\} - P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}\right\} \\ & \leq \bigg\vert P\left\{\frac{1}{\sqrt{n}} \sum_{i=1}^n \hat{u}^{(1)}(\bm{x}^{(d)}; D_i) \in \mathsf{R}_t\right\} - P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}_t\right\}\bigg\vert\\ & + \bigg\vert P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}_t\right\} - P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}\right\} \bigg\vert\\ & + \bigg\vert P\left\{ \big \|\sqrt{\frac{n}{b^2}}\Sigma^{-1} \bar{U}_{n,b}(\bm{x}^{(d)})- \frac{1}{\sqrt{n}} \sum_{i=1}^n \hat{u}^{(1)}(\bm{x}^{(d)}; D_i)\big \|_\infty> t\right\} \bigg\vert . \end{align} A matching lower bound follows from an analogous argument, by considering the shrunken rectangle $[a_l+t\bm{1}_d,a_u-t\bm{1}_d]$. We bound the normal approximation term (ref) through the application of (ref). In particular, observe that \begin{equation} \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[\left(\hat{u}^{(1)}(x^{(j)}; D_{i})\right)^2\right]=1 , \quad\ \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[\left(\hat{u}^{(1)}(x^{(j)},D_{i})\right)^4\right] \leq \left(\phi/\underline{\sigma}_b\right)^{2} \end{equation} and $\| \hat{u}^{(1)}(x^{(j)}, D_{i}) \|_{\psi_1} \leq (\phi/\underline{\sigma}_b)$. Consequently, as \begin{equation} \Var(\hat{u}^{(1)}(x^{(j)}, D_{i}))=\Sigma^{-1/2}\Var\left(Z\right)\Sigma^{-1/2}, \end{equation} Lemma (ref) implies that \begin{equation} \bigg\vert P\left\{\frac{1}{\sqrt{n}} \sum_{i=1}^n \hat{u}^{(1)}(\bm{x}^{(d)}; D_i) \in \mathsf{R}_t\right\} - P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}_t\right\}\bigg\vert \lesssim \left(\frac{\phi^{2}\log^{5}(dn)}{\underline{\sigma}_b^{2}n}\right)^{1/4} . \end{equation} Now, to bound the difference (ref), (ref) gives \begin{equation} \Big\vert P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}_t\right\} - P\left\{ \Sigma^{-1/2}Z \in \mathsf{R}\right\} \Big\vert \lesssim t \sqrt{\log(d)} . \end{equation} Finally, as in the previous Part of the Lemma, recall that there exists a constant $C_1$ such that if $b\log(dn)/n <C_1 < 1$, then (ref) implies that (ref) is smaller than $\phi n^{-1}$, with probability greater than $1-Cn^{-1}$, so long as $b\gtrsim C_2 \log(n)$. Thus, the proof is complete by setting $t = \phi n^{-1}$. \qed \section{Details Concerning Data and Simulations} In this appendix, we document our treatment of the banerjee2015multifaceted data. (ref) details our acquisition and cleaning of these data. In (ref), we give further details concerning the construction of (ref) and (ref). (ref) discusses our simulation calibration. \subsection{Data} The data from banerjee2015multifaceted were acquired from \url{https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/NHIXNT} on September 10, 2021. The data from the graduation program implemented in Pakistan are considered in chen2023semiparametric and ritzwoller2023reproducible. Here, we consider the data from the graduation program implemented in Ghana, as it has a larger sample size. The data record measurements of many pre-treatment and post-treatment outcomes for 2,606 individuals in the northern region of Ghana. Baseline survey measurements were made prior to the allocation of treatment. A multifaceted treatment was randomly allocated to 632 of the individuals. banerjee2015multifaceted consider data from two endline surveys, made two years and three years after the initial asset transfer, respectively. For the purpose of this paper, we consider only data from the baseline survey, records of the treatment allocation, and measurements from the first endline survey. We omit data from 164 attrited individuals, none of whom were assigned to the treatment. The covariate vector $Z_i$ is composed of measurements of 16 pre-treatment outcomes. Five of these outcomes are associated with consumption: total monthly consumption and total monthly consumption on food, non-food, and durable commodities. Each consumption variable is measured in 2014 US dollars. We transform total monthly consumption to logs, base 10. Three of the variables are associated with assets, each of which is an index constructed from survey data measuring total assets, total productive assets, and total household assets. We transform the total assets measurement to logs, base 10. Five of the outcomes are associated with food security. These consist of four binary variables indicating different aspects of food security, e.g. did a child skip a meal, in addition to an index aggregating these measurements.\footnote{There are $2$ individuals with missing values for the food security index. We impute these values with the median values of the food security index.} The final four variables are associated with finance and income: the total amount of outstanding loans, the total amount of savings, income from agriculture, and total income from business. The covariate vector $X_i$ collects the total monthly consumption and assets for each individual. The outcome $Y_i$ measures the total assets two years after the initial asset transfer. Again, we transform these measurements to logs, base 10. \subsection{Parameter Choices} In (ref) and (ref), we set the subsample proportion $b/n$ equal to 0.05. In constructing the nuisance parameter estimate, we use set $b/n$ = 0.025. We use $r=200$ bootstrap replicates throughout. We use 20,000 trees to construct (ref) and 2,000 trees in each bootstrap replicate to construct (ref) and throughout the simulation. \subsection{Simulation Calibration} We calibrate a simulation to the banerjee2015multifaceted data using a collection of Generalized Adversarial Networks (GAN) goodfellow2014generative. This approach to simulation design was proposed by athey2021using. Roughly speaking, a GAN is a pair of neural networks. The objective of the first network, the generator, is to generate data that looks like the banerjee2015multifaceted data. The objective of the second network, the discriminator, is to discriminate between the real banerjee2015multifaceted data and the data generated by the generator. These networks compete iteratively until convergence. The idea is that, after convergence, the generator is a good proxy for the true process that generated the banerjee2015multifaceted data. We use the “WGAN” package associated with athey2021using. To calibrate our simulation, we estimate three GANs. The first GAN estimates the distribution of the covariates $X_i$, i.e., baseline consumption and baseline total assets. The second GAN estimates the distribution of $Z_i$ conditional on $X_i$. Recall that $Z_i$ collects all baseline covariates, other than the covariates in $X_i$. The third GAN estimates the distribution of $Y_i$ conditioned on $Z_i$, $X_i$, and $W_i$. To generate an observation $D_i$, we generate $X_i$, generate $Z_i$ conditioned on $X_i$, and generate the potential outcomes $Y_i(1)$ and $Y_i(0)$ from the estimated distributions of $Y_i$ conditioned on $Z_i$, $X_i$, and $W_i = 1$ and $W_i = 0$, respectively. The treatment indicator is $W_i$ is drawn i.i.d., Bernoulli with the observed probability in the banerjee2015multifaceted data and we set $Y_i = Y_i(W_i)$. In this way, we know the true treatment effect $Y_i(1) - Y_i(0)$ for each unit in our simulation. We draw 10 million observations $D_i$ with this process. In the simulation, datasets of various sizes are sampled from this collection. We use a related procedure to determine the true CATE $\theta_0(x)$ at each value $x$ in the query-vector $\mathbf{x}^{(d)}$ (i.e., the centers of each of the rectangles displayed in (ref)). Specifically, for each $x$ in $\mathbf{x}^{(d)}$, we draw 100,000 observations from the distribution of $Z_i$ conditioned on $X_i = x$. We then draw observations $Y_i(1)$ and $Y_i(0)$ for each of these replicates, and compute the average of the true treatment effects $Y_i(1) - Y_i(0)$. (ref) displays these pseudo-true values of the CATE $\theta_0(x)$. Our simulation design captures many of the same features of the data recovered by GRF, but gives a somewhat smoother picture of the CATE. (ref) displays a scatterplot comparing the moments of the data from the banerjee2015multifaceted data to the data generated by our calibrated simulation. The distributions match quite closely. (ref) compares a scatter plot of the observed values of baseline consumption and baseline assets in the banerjee2015multifaceted data with a heat-map of the distribution of these covariates in our simulation. The limits of the horizontal and vertical axes in this Figure match (ref) displayed in the main text. Some observations fall outside of the limits of this figure. The quartiles of baseline log consumption are 3.33, 3.76, and 4.20. The quartiles of baseline assets are -0.45, -0.71, and 0.03. \section{Proofs for Lemmas Supporting the Proof of (ref)} \subsection{Proof of (ref), Part (i)} Throughout, we take $\theta_0(x)=0$ for all $x$ without loss of generality. We begin by giving a high-probability bound on the difference \begin{equation} \| R_n(\bm{x}^{(d)}) -U(\bm{x}^{(d)}) \|_\infty \end{equation} which will be needed at a later point in the proof. To this end, let $x$ be any component of the vector $\bm{x}^{(d)}$. By a Taylor expansion about $\theta_0(x)$, we have that \begin{align} M(x; \hat{\theta}_n(x), \hat{g}_n) - M(x ;\theta_0(x), \hat{g}_n) & = (\hat{\theta}_n(x) - \theta_0(x)) M^{(1)}(x; \theta_0(x), \hat{g}_n) \nonumber \\ & + (\hat{\theta}_n(x) - \theta_0(x))^2 H(x; \tilde{\theta}_0(x), \hat{g}_n) \end{align} for some $\tilde{\theta}_0(x)$ between $\hat{\theta}_n(x)$ and $\theta_0(x)$. Moreover, we can write \begin{align} &(\hat{\theta}_n(x) - \theta_0(x)) M^{(1)}(x; \theta_0(x), \hat{g}_n)\nonumber \\ & = (\hat{\theta}_n(x) - \theta_0(x)) M^{(1)}(x; \theta_0(x), g_0)\nonumber \\ & + (\hat{\theta}_n(x) - \theta_0(x)) \left(M^{(1)}(x; \theta_0(x), \hat{g}_n) - M^{(1)}(x; \theta_0(x), g_0) \right) \end{align} and \begin{align} M(x;\hat{\theta}_n(x),\hat{g}_n) - M(x;\theta_0(x),\hat{g}_n) & = (M(x;\hat{\theta}_n(x),\hat{g}_n) - M_n(x;\hat{\theta}_n(x),\hat{g}_n,\mathbf{D}_n)) \nonumber\\ & \quad+ (M(x;\theta_0(x),g_0) - M(x;\theta_0(x),\hat{g}_n)) \nonumber\\ & = \left(M(x;\hat{\theta}_n(x),\hat{g}_n) - \mathbb{E}\left[M_n(x;\hat{\theta}_n(x),\hat{g}_n,\mathbf{D}_n)\right]\right) \nonumber\\ & \quad+ \left(\mathbb{E}\left[M_n(x;\hat{\theta}_n(x),\hat{g}_n,\mathbf{D}_n)\right] - M_n(x;\hat{\theta}_n(x),\hat{g}_n,\mathbf{D}_n)\right) \nonumber\\ & \quad+ (M(x;\theta_0(x),g_0) - M(x;\theta_0(x),\hat{g}_n)) . \end{align} Thus, by the identity \begin{align} \bar{M}_n(x;\hat{\theta}_n(x),\hat{g}_n) &= \bar{M}_n(x;\hat{\theta}_n(x),\hat{g}_n) - \bar{M}_n(x;\theta_0(x),\hat{g}_n) \nonumber\\ & + \bar{M}_n(x;\theta_0(x),\hat{g}_n) - \bar{M}_n(x;\theta_0(x),g_0) + \bar{M}_n(x;\theta_0(x),g_0) , \end{align} the equalities (ref), (ref), and (ref) imply that \begin{align} &M^{(1)}(x;\theta_0(x),g_0) (\hat{\theta}_n(x) - \theta_0(x))\nonumber\\ & = M^{(1)}(x;\theta_0(x),\hat{g}_n) (\hat{\theta}_n(x) - \theta_0(x)) \nonumber\\ & - \left(M^{(1)}(x;\theta_0(x),\hat{g}_n) - M^{(1)}(x;\theta_0,g_0)\right) (\hat{\theta}_n(x) - \theta_0(x)) \nonumber\\ & = -\bar{M}_n(x;\theta_0(x),g_0) \\ & \quad+ \mathsf{Bias}(x;\hat{\theta}_n(x),\hat{g}_n) + \mathsf{Nuis}(x;\theta_0(x),\hat{g}_n) \\ & \quad+ \mathsf{Stoch}^{(1)}(x; \hat{\theta}_n(x),\hat{g}_n) + \mathsf{Stoch}^{(2)}(x; \hat{g}_n) \\ & \quad- (\hat{\theta}_n(x) - \theta_0(x))^2 H(x; \tilde{\theta}_0(x), \hat{g}_n) \\ & \quad- (\hat{\theta}_n(x) - \theta_0(x)) \left(M^{(1)}(x; \theta_0(x), \hat{g}_n) - M^{(1)}(x; \theta_0(x), g_0) \right) \end{align} where \begin{align} \mathsf{Bias}(x;\hat{\theta}_n(x),\hat{g}_n) &= M(x;\hat{\theta}_n(x),\hat{g}_n) - \mathbb{E}\left[M_n(x;\hat{\theta}_n(x),\hat{g}_n)\right] ,\\ \mathsf{Nuis}(x;\theta_0(x),\hat{g}_n) &= M(x;\theta_0(x),g_0) - M(x;\theta_0(x),\hat{g}_n) ,\\ \mathsf{Stoch}^{(1)}(x; \hat{\theta}_n(x),\hat{g}_n) &= \bar{M}_n(x;\hat{\theta}_n(x),\hat{g}_n) - \bar{M}_n(x;\theta_0(x),\hat{g}_n) ,\quad\text{and}\\ \mathsf{Stoch}^{(2)}(x; \hat{g}_n)&= M(x;\theta_0(x),g_0) - M(x;\theta_0(x),\hat{g}_n) , \end{align} respectively. We now give bounds for the terms (ref), (ref), (ref), and (ref). To handle (ref), observe that Parts (i) and (ii) of (ref) imply that \begin{equation} \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \vert \mathsf{Bias}(\bm{x}^{(d)};\hat{\theta}_n(x),\hat{g}_n)\vert \lesssim (1 + \|\hat{\theta}_n(\bm{x}^{(d)})\|_\infty ) \delta_{n,B} \lesssim \delta_{n,B}\left(1 + \frac{\underline{\lambda}^{1/2}}{n^{1/4}} \delta_{n,\theta}\right) \end{equation} with probability greater than $1-\rho_{n,\theta}$. Moreover, a Taylor expansion, second-order smoothness, i.e., (ref), and (ref), Part (ii), give that \begin{align} \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \mathsf{Nuis}(\bm{x}^{(d)};\theta_0(x),\hat{g}_n) & = \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \partial_g M(\bm{x}^{(d)}; \theta_0(x), g_0)[g-g_0] \nonumber\\ &+ \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \partial_{g,g} M(\bm{x}^{(d)}; \theta_0(x), g_0)[g-g_0] \lesssim \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \|g - g_0\|_{2,\infty}^2 \lesssim \delta^2_{n,g} . \end{align} with probability greater than $1-\rho_{n,g}$. Next, we handle the term (ref). By (ref), a Taylor expansion gives \begin{align} \mathsf{Stoch}^{(1)}(x; \hat{\theta}_n(x),\hat{g}_n) & = (\hat{\theta}_n(x) - \theta_0(x)) \bar{M}_n^{(1)}(x; \theta_0(x), \hat{g}_n) \nonumber \\ & + (\hat{\theta}_n(x) - \theta_0(x))^2 \bar{H}_n(x; \tilde{\theta}_0(x), \hat{g}_n) \end{align} for some, potentially different, $\tilde{\theta}_0(x)$ between $\hat{\theta}_n(x)$ and $\theta_0(x)$. To bound this term, observe that \begin{align} &\vert \bar{M}_n^{(1)}(x; \theta_0(x), \hat{g}_n) \vert \leq \vert \bar{M}_n^{(1)}(x; \theta_0(x), g_0)\vert +\vert \bar{M}_n^{(1)}(x; \theta_0(x), \hat{g}_n) - \bar{M}_n^{(1)}(x; \theta_0(x), g_0)\vert . \end{align} Hence, (ref) and (ref), Parts (ii) and (iii) imply that \begin{equation} \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \| \mathsf{Stoch}^{(1)}(\bm{x}^{(d)}; \hat{\theta}_n(\bm{x}^{(d)}),\hat{g}_n)\|_\infty \leq \delta_{n,m}\delta_{n,\theta} + \frac{\underline{\lambda}^{1/2} }{ n^{1/4}} \delta_{n,\theta} \delta_{n,J} + \delta^2_{n,\theta} \end{equation} with probability greater than $1 - \rho_{n,m} - \rho_{n,\theta} - \rho_{n,J}$. Moreover, we have that \begin{equation} \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \| \mathsf{Stoch}^{(2)}(\bm{x}^{(d)}; \hat{g}_n)\|_\infty \leq \delta_{n,S} \end{equation} with probability greater than $1-\rho_{n,S}$. Finally, we handle the terms (ref) and (ref). (ref), Part (ii), and (ref) imply that \begin{align} \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \|(\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}))^2 H(\bm{x}^{(d)} ; \tilde{\theta}_0(\bm{x}^{(d)}), \hat{g}_n)\|_\infty &\lesssim \delta^2_{n,\theta} \quad\text{and}\\ \sqrt{ \frac{n}{ \underline{\lambda}^2 } } \|(\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)})) (M^{(1)}(\bm{x}^{(d)}; \theta_0(x), \hat{g}_n) - M^{(1)}(\bm{x}^{(d)}; \theta_0(x), g_0)) \|_\infty &\lesssim \delta_{n,\theta} \delta_{n,g} \end{align} with probabilities greater than $1-\rho_{n,\theta}$ and $1-\rho_{n,g}$, respectively. Putting the pieces together, the decomposition (ref) and the lower-boundedness of the Jacobian $M^{(1)}(\cdot;\theta,g)$ imply that \begin{align} &\sqrt{ \frac{n}{ \underline{\lambda}^2 } } \| R_n(\bm{x}^{(d)}) -U(\bm{x}^{(d)}) \|_\infty \\ & \lesssim \delta^2_{n,g} + \delta^2_{n,\theta} + \delta_{n,B} + \delta_{n,S}+ \delta_{n,\theta}\left(\delta_{n,m} + \delta_{n,g} + \underline{\lambda}^{1/2}n^{-1/4} \left(\delta_{n,B} + \delta_{n,J}\right)\right)\nonumber\\ & \lesssim \delta^2_{n,g} + \delta^2_{n,\theta} + \delta^2_{n,m} + \delta_{n,B} + \delta_{n,S} + \underline{\lambda}^{1/2}n^{-1/4} \delta_{n,\theta}\left(\delta_{n,B} + \delta_{n,J}\right)\nonumber \end{align} with probability greater than $1-\rho_{n,m} + \rho_{n,g} + \rho_{n,\theta} + \rho_{n,S} + \rho_{n,J}$, by the bounds (ref), (ref), (ref), (ref), (ref), and (ref). With this in place, we turn to the proof of the normal approximation to $R_{n}(\bm{x}^{(d)})$ on hyper-rectangles. Fix a rectangle $\mathsf{R}=[a_{l},a_{u}]$ in $\mathcal{R}$. For the sake of exposition, we give the details of the proof of the upper bound \begin{align} P\left\{\sqrt{n} R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{Z\in\mathsf{R} \right\} \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log(d)} + \rho_n . \end{align} The matching lower bound will follow from a very similar argument. We comment on the differences where they occur. Consider the decomposition \begin{flalign} R_n(x) & = \left(\frac{1}{n}\sum_{i=1}^n \bar{u}(x, D_i) - U_n(x)\right) - \frac{1}{n}\sum_{i=1}^n \bar{u}(x, D_i) + \Delta_n(x),\quad\text{where}\\ \Delta_n(x) & = R_n(x) -U(x) , \end{flalign} and we recall that the function $\bar{u}(\cdot,\cdot)$ is defined in (ref). Define the normalized functions \begin{equation*} \hat{u}(\bm{x}^{(d)}, D_i) = \Lambda^{-1/2} \bar{u}(\bm{x}^{(d)}, D_i)\quad\text{and}\quad\hat{U}(\bm{x}^{(d)}) = \Lambda^{-1/2} U(\bm{x}^{(d)}) \end{equation*} and the analogously normalized rectangle $\tilde{\mathsf{R}} = [\Lambda^{-1/2}a_l, \Lambda^{-1/2}a_u]$ and the enlarged rectangle $\tilde{\mathsf{R}}_t = [\Lambda^{-1/2}a_l - \mathbf{1}_dt, \Lambda^{-1/2}a_u + \mathbf{1}_dt]$. Observe that the decomposition (ref) yields the upper bound \begin{flalign} & P\left\{\sqrt{n} R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{Z\in\mathsf{R} \right\} \nonumber \\ & = P\left\{\sqrt{n} \Lambda^{-1/2} R_{n}(\bm{x}^{(d)})\in\tilde{\mathsf{R}}\right\} -P\left\{\Lambda^{-1/2}Z\in\tilde{\mathsf{R}} \right\} \nonumber \\ & \quad \leq\Big\vert P\left\{ \frac{1}{\sqrt{n}}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i) \in \tilde{\mathsf{R}}_t \right\} -P\left\{\Lambda^{-1/2}Z\in \tilde{\mathsf{R}}_t \right\} \Big\vert \\ & \quad\quad +\Big\vert P\left\{\Lambda^{-1/2}Z\in\tilde{\mathsf{R}_{t}}\right\} -P\left\{\Lambda^{-1/2}Z\in\tilde{\mathsf{R}}\right\} \Big\vert \\ & \quad\quad +P\left\{\sqrt{n}\|\hat{U}_n(\bm{x}^{(d)}) - \frac{1}{n} \sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i) \|_\infty \ge \frac{1}{2}t \right\} \\ & \quad\quad +P\left\{\sqrt{n} \| \Lambda^{-1/2} \Delta_n(\bm{x}^{(d)})\|_\infty \ge \frac{1}{2}t \right\} \end{flalign} for each $t>0$. An analogous lower bound follows by considering a shrunken rectangle $[\Lambda^{-1/2}a_l + \mathbf{1}_dt, \Lambda^{-1/2}a_u - \mathbf{1}_dt]$. We proceed by providing appropriate bounds for the terms (ref) through (ref). We begin by bounding the normal approximation term (ref) through the application of the following quantitative central limit theorem, stated as Theorem 2.1 of chernozhuokov2022improved. \begin{lemma}[{chernozhuokov2022improved}] Let $X_{1},\ldots,X_{n}$ be a collection of independent, centered, random vectors in $\mathbb{R}^{d}$ and let $Z$ be a centered Gaussian random vector with covariance matrix \begin{equation} \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[X_{i}X_{i}^{\top}\right]. \end{equation} If there exist absolute constants $c$, $C_1$, and $\varphi$ such that the bounds \begin{equation} \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[X_{i,j}^{2}\right]\geq c ,\quad \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[X_{i,j}^{4}\right]\leq C_1 \varphi^{2} , \quad\text{and}\quad \|X_{i,j} \|_{\psi_1} \leq \varphi \end{equation} hold, then the inequality \begin{equation} \sup_{\mathsf{R}\in\mathcal{R}} \Big\vert P\left\{ \frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_{i}\in \mathsf{R}\right\} -P\left\{ \sqrt{n}Z\in \mathsf{R}\right\} \Big\vert\leq C_2 \left(\frac{\varphi^{2}\log^{5}(dn)}{n}\right)^{1/4} \end{equation} holds for some constant $C_2$ that depends only on $c$ and $C_1$. \end{lemma} In particular, observe that \begin{equation} \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[\hat{u}^{2}(x^{(j)}, D_{i})\right]=1 \end{equation} by definition and that \begin{equation} \frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left[\hat{u}^{4}(x^{(j)},D_{i})\right] \leq \left(\varphi/\underline{\lambda}\right)^{2} \end{equation} by (ref). Similarly we have that \begin{equation} \| \hat{u}\left(x^{j}), D_{i}\right) \|_{\psi_1} \leq (\varphi /\underline{\lambda}) \end{equation} by (ref). Consequently, as \begin{equation} \Var(\hat{u}(x^{(j)}, D_{i}))=\Lambda^{-1/2}\Var\left(Z\right)\Lambda^{-1/2}, \end{equation} by definition, Lemma (ref) implies that the bound \begin{equation} \Big\vert P\left\{ \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\hat{u}\left(D_{i}\right)\in \mathsf{\tilde{R}}_t\right\} -P\left\{ \Lambda^{-1/2}Z\in \mathsf{\tilde{R}}_t\right\} \Big\vert \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} . \end{equation} holds. In turn, to bound the term (ref), we have that \begin{equation} \Big\vert P\left\{\Lambda^{-1/2}Z\in\mathsf{R}_{t}\right\} -P\left\{\Lambda^{-1/2}Z\in\mathsf{R}\right\} \Big\vert \lesssim t \sqrt{\log d} . \end{equation} by (ref). Moreover, to bound the term (ref), recall that \begin{equation} P\left\{\sqrt{n}\|\hat{U}_n(\bm{x}^{(d)}) - \frac{1}{n} \sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i) \|_\infty \ge \delta_{n,u} \right\} \leq \rho_{n,u} \end{equation} by (ref). Thus, by choosing $t=C\delta_n$, the bound (ref) implies that the sum of term (ref) and term (ref) is upper bounded by $C\rho_{n}$. Hence, by plugging this choice of $t$ into (ref) and (ref), we can conclude that \begin{equation} \Big\vert P\left\{\sqrt{n} R_{n}(\bm{x}^{(d)})\in\mathsf{R}\right\} -P\left\{ Z\in\mathsf{R}\right\} \Big\vert \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^{2}n}\right)^{1/4} + \delta_n\sqrt{\log(d)} + \rho_n , \end{equation} as required.\qed \subsection{Proof of (ref), Part (ii)} The result follows from an argument whose structure is similar to the proof of (ref), Part (i). Again, we take $\theta_0(x)=0$ for all $x$, without loss of generality. We are interested in studying the discrepancy \begin{flalign*} R_{n}^{*}(x) & =\hat{\theta}_\mathsf{h}(x)-\hat{\theta}_n(x) = (\hat{\theta}_\mathsf{h}(x)-\theta_{0}(x))-R_{n}(x). \end{flalign*} In terms of the decomposition (ref), we can write \begin{flalign} R_n(x) = \left(\frac{1}{n}\sum_{i=1}^n \bar{u}(x, D_i) - U_n(x)\right) - \frac{1}{n}\sum_{i=1}^n \bar{u}(x, D_i) + \Delta_n(x) , \end{flalign} where $\Delta_n(x)$ is defined in (ref). On the other hand, as $\hat{\theta}_\mathsf{h}(x)$ is constructed with a random half-sample $\mathsf{h}$ of the data $\mathbf{D}_{n}$, we have that \begin{flalign} \hat{\theta}_\mathsf{h}(x)-\theta_{0}(x) = \left(\frac{2}{n}\sum_{i\in\mathsf{h}}^n \bar{u}(x, D_i) - U_\mathsf{h}(x)\right) - \frac{2}{n}\sum_{i\in\mathsf{h}}^n \bar{u}(x, D_i) + \Delta_\mathsf{h}(x) , \end{flalign} where $U_\mathsf{h}(x)$ and $\Delta_\mathsf{h}(x)$ are constructed with the half-sample $\mathsf{h}$. The proof of (ref), Part (i), worked by giving a high probability bound for the first and third term in (ref) and showing that the the second term satisfies a central limit theorem. Here, as we are interested in giving a bound conditioned on the data $\mathbf{D}_n$, we show that the difference between the second terms in (ref) and (ref) satisfies a central limit theorem on the event that the first and third terms in (ref) and (ref) satisfy a specified bound, which we show holds with high probability. In particular, let $\mathcal{F}_n(t)$ and $\mathcal{F}_\mathsf{h}(t)$ denote the events that \begin{equation} \sqrt{\frac{n}{\underline{\lambda}^2}} \| \Delta_n(\bm{x}^{(d)})\|_\infty \leq t/4 \quad\text{and}\quad \sqrt{\frac{n}{\underline{\lambda}^2}} \| \Delta_\mathsf{h}(\bm{x}^{(d)})\|_\infty \leq t/4 , \end{equation} respectively. Similarly, let $\mathcal{H}_n(t)$ and $\mathcal{H}_\mathsf{h}(t)$ denote the events that \begin{flalign} &\sqrt{n}\| \frac{1}{n}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i) - \hat{U}_n(\bm{x}^{(d)}) \|_\infty \leq t/4 \quad\text{and}\\ &\sqrt{n}\| \frac{2}{n}\sum_{i\in\mathsf{h}}^n \hat{u}(\bm{x}^{(d)}, D_i) - \hat{U}_\mathsf{h}(\bm{x}^{(d)}) \|_\infty \leq t/4\nonumber \end{flalign} respectively, where $\hat{U}_\mathsf{h}(\bm{x}^{(d)})$ is defined analogously to $\hat{U}_n(\bm{x}^{(d)})$. Define the event $\mathcal{E}_n(t) = \mathcal{F}_n(t) \cap \mathcal{F}_\mathsf{h}(t) \cap \mathcal{H}_n(t) \cap \mathcal{H}_\mathsf{h}(t)$. Fix a hyper-rectangle $\mathsf{R}$ in $\mathcal{R}$. As before, we prove only the requisite upper bound. The lower bound follows from an analogous argument. Recall the definitions of the transformed rectangle $\tilde{\mathsf{R}}$ and the enlarged transformed rectangle $\tilde{\mathsf{R}}_t$. On the event $\mathcal{E}_n(t) $, we have \begin{flalign} & P\left\{\sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} - P\left\{ Z\in\mathsf{R}\right\} \nonumber \\ & = P\left\{\sqrt{n} \Lambda^{-1/2} R_{n}^{*}(\bm{x}^{(d)})\in\tilde{\mathsf{R}}\mid\mathbf{D}_{n}\right\} - P\left\{\Lambda^{-1/2} Z\in\tilde{\mathsf{R}}\right\} \nonumber \\ & \quad\leq P\left\{ \frac{2}{\sqrt{n}}\sum_{i\in\mathsf{h}}^n \hat{u}(\bm{x}^{(d)}, D_i) - \frac{1}{\sqrt{n}}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i)\in\tilde{\mathsf{R}}_{t} \mid\mathbf{D}_{n}\right\} -P\left\{ \Lambda^{-1/2}Z\in\tilde{\mathsf{R}}_{t}\right\} \nonumber \\ & \quad+\vert P\left\{ \Lambda^{-1/2}Z\in\tilde{\mathsf{R}}\right\} -P\left\{ \Lambda^{-1/2}Z\in\tilde{\mathsf{R}}_t\right\} \vert\nonumber \end{flalign} for each $t>0$. As the data $D_\mathsf{h}$ are drawn independently and identically with distribution $P$ in $\mathbf{P}$ and we have assumed that $\delta_{n/2}\lesssim \delta_n$ and $\rho_{n/2}\lesssim \rho_n$, by setting $t = C\delta_n$, (ref) and the bound (ref) imply that the event $\mathcal{E}_n(t)$ occurs with probability greater than $1-C\rho_n$. Thus, (ref) implies that \begin{flalign} & P\left\{\sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} - P\left\{ Z\in\mathsf{R}\right\} \\ & \quad \leq P\left\{ \frac{2}{\sqrt{n}}\sum_{i\in\mathsf{h}}^n \hat{u}(\bm{x}^{(d)}, D_i) - \frac{1}{\sqrt{n}}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i)\in\tilde{\mathsf{R}}_{t} \mid\mathbf{D}_{n}\right\} -P\left\{ \Lambda^{1/2}Z\in\tilde{\mathsf{R}}_{t}\right\} \\ & \quad\quad+ \delta_n \sqrt{\log(d)} \nonumber \end{flalign} with probability greater than $1-C\rho_n$. Hence, it suffices to bound the term (ref). To this end, we apply a coupling argument introduced in yadlowsky2023evaluating, which is similar to a Poissonization technique studied in praestgaard1993exchangeably (see also Section 3.6.2 of van1996weak). In particular, let $V_{i}$ be a random variable taking the value $1$ when $i$ is an element of the subset $\mathsf{h}$ and taking the value $-1$ otherwise. Observe that \begin{flalign} \frac{2}{n}\sum_{i\in\mathsf{h}} \hat{u}(\bm{x}^{(d)}, D_i) - \frac{1}{n}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i) = \frac{1}{n} \sum_{i=1}^n V_i \hat{u}(\bm{x}^{(d)}, D_i) . \end{flalign} Let $\tilde{V}_{1},\ldots,\tilde{V}_{n}$ denote a collection of random variables valued on $\{-1,1\}$. We define their joint distribution as follows. Let $Q_{n}$ denote a random variable with distribution $\mathsf{Bin}\left(n,1/2\right)$. If $Q_{n}\geq n/2$, then choose $Q_{n}-n/2$ indices $i$ in $[n]$ with $V_{i}=-1$ and set $\tilde{V}_{i}=1$. If $Q_{n}<n/2$, then choose $n/2-Q_{n}$ indices with $V_{i}=1$ and set $\tilde{V}_{i}=-1$. Set $\tilde{V}_{i}=V_{i}$ for all other units. Observe that the collection $\tilde{V}_{i}$ are independent and identically distributed Rademacher random variables. With this in place, we obtain the decomposition \begin{equation} \frac{1}{n}\sum_{i=1}^{n}V_{i}\hat{u}(\bm{x}^{(d)}, D_i) =\frac{1}{n}\sum_{i=1}^{n}\tilde{V}_{i} \hat{u}(\bm{x}^{(d)}, D_i) +\frac{1}{n}\sum_{i=1}^{n}\left(V_{i}-\tilde{V}_{i}\right)\hat{u}(\bm{x}^{(d)}, D_i) . \end{equation} Let $\mathcal{V}(t)$ denote the event that \begin{equation} \sqrt{n} \Big\| \frac{1}{n}\sum_{i=1}^{n}\left(V_{i}-\tilde{V}_{i}\right)\hat{u}(\bm{x}^{(d)}, D_i) \Big\|_\infty > t . \end{equation} Fix any rectangle $\mathsf{R}^\prime = [a^\prime_l, a^\prime_u]$ and define the enlarged rectangle $\mathsf{R}^\prime_t = [a_l - t\mathbf{1}_d,a_u + t\mathbf{1}_d]$. On the event $\mathcal{V}(t)$, the decomposition (ref) implies that \begin{flalign} & P\left\{ \frac{1}{\sqrt{n}}\sum_{i=1}^{n}V_{i}\hat{u}(\bm{x}^{(d)}, D_i) \in \mathsf{R}^\prime \mid \mathbf{D}_n \right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}^\prime \right\} \nonumber\\ & \leq P\left\{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\tilde{V}_{i} \hat{u}(\bm{x}^{(d)}, D_i) \in \tilde{\mathsf{R}}_t \mid \mathbf{D}_n \right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}^\prime_t\right\} \\ & \quad\quad + \vert P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}^\prime_{t}\right\} -P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}^\prime\right\} \vert \end{flalign} for all $t>0$. (ref) implies that (ref) is less than $t\sqrt{\log(d)}$. To handle the term (ref), we apply the following quantitative central limit theorem, stated as Lemma 4.6 in chernozhuokov2022improved. \begin{lemma}[{chernozhuokov2022improved}] Consider the setting and assumptions of (ref). Let $\bm{X}_{n}=(X_{1},\ldots,X_{n})$ collect the observed data and let $\tilde{V}_{1},\ldots,\tilde{V}_{n}$ be a collection of independent Rademacher random variables. We have that \begin{equation} \sup_{\mathsf{R}\in\mathcal{R}}\Big\vert P\left\{ n^{-1/2}\sum_{i=1}^{n}X_{i}\in \mathsf{R}\right\} -P\left\{ n^{-1/2}\sum_{i=1}^{n}\tilde{V}_{i}X_{i}\in R\mid\bm{X}_{n}\right\} \Big\vert \leq C_2 \left(\frac{\varphi^{2}\log^{5}(dn)}{n}\right)^{1/4}. \end{equation} with probability greater than \begin{equation} 1-C\frac{\varphi \log^{3/2}(dn)}{n^{1/2}} \end{equation} for some constant $C_2$ that depends only on the constants $C_2$ and $c$ defined in the statement of (ref). \end{lemma} Thus, on the event $\mathcal{V}(t)$, (ref) and (ref) imply that \begin{flalign*} P\left\{ \frac{1}{\sqrt{n}}\sum_{i=1}^{n}V_{i}\hat{u}(\bm{x}^{(d)}; D_i) \in \mathsf{R}^\prime \mid \mathbf{D}_n \right\} - P\left\{ \Lambda^{-1/2} Z\in\mathsf{R}^\prime\right\} \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^2 n}\right)^{1/4} + t\sqrt{\log(d)} , \end{flalign*} with probability greater than $1-Cn^{-1/2}\underline{\lambda}^{-1}\varphi \log^{3/2}(dn)$. Hence, it suffices to give a high probability bound on $\mathcal{V}(t)$ for a suitable choice of $t$. To this end, observe that \begin{flalign} G_{n} & =\Big\|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(V_{i}-\tilde{V}_{i}\right)\hat{u}(\bm{x}^{(d)}; D_i)\Big\|_{\infty} \quad \text{is equidistributed with} \quad\\ &\quad \Big\|\frac{2}{\sqrt{n}}\sum_{i=1}^{\vert Q_{n} - n/2\vert}\hat{u}(\bm{x}^{(d)};D_{i})\Big\|_{\infty} . \end{flalign} Consider the decomposition \begin{flalign} P\left\{ G_{n}\geq t \right\} & \leq P\left\{ G_{n}\geq t,\vert Q_{n} - n/2 \vert \leq\delta\frac{n}{2}\right\} +P\left\{ \vert Q_{n} - n/2 \vert \geq\delta\frac{n}{2}\right\} \nonumber \\ & \leq P\left\{ \max_{1\leq k\leq\delta\frac{n}{2}}\Big\|\frac{2}{\sqrt{n}}\sum_{i=1}^{k}\hat{u}\left(D_{i}\right)\Big\|_{\infty}\geq t\right\} +P\left\{ \vert Q_{n} - n/2\vert \geq\delta\frac{n}{2}\right\} , \end{flalign} for some $\delta>0$ to be chosen. Observe that \begin{equation} P\left\{ \vert Q_{n} - n/2\vert \geq\frac{\delta n}{2}\right\} \le2\exp\left(-\frac{\delta^{2}n}{6}\right) \end{equation} by the multiplicative Chernoff bound. To bound the first term in ((ref)), we combine two inequalities. The first inequality is the following standard Bernstein-type bound, stated, e.g., in song2019approximating. \begin{lemma}[Lemma A.2, song2019approximating] Let $Z_{1},\ldots,Z_{n}$ be independent, centered, random vectors in $\mathbb{R}^{d}$. Define the quantity \begin{equation} \sigma^2 = \max_{j\in[d]} \sum_{i=1}^n \mathbb{E}\left[Z^2_{i,j}\right] \end{equation} and assume that $\|Z_{ij}\|_{\psi_{1}}\leq \phi $ for all $i\in[n]$ and $j\in[d]$. The inequality \begin{flalign*} & P\left\{ \Big\|\sum_{i=1}^{n}Z_{i}\Big\|_{\infty} \geq C \left(\sigma \log^{1/2}(dg) + \phi \log(dn)\left(\log(dn) + \log(g)\right)\right) \right\} \lesssim \frac{1}{g} \end{flalign*} holds for any constant $g>0$. \end{lemma} The second inequality is a L\'{e}vy type inequality for independent random vectors, due to montgomery1993comparison. See Chapter 1 of de2012decoupling for a textbook treatment. \begin{lemma}[Theorem 1.1.5, de2012decoupling] Let $Z_{1},\ldots,Z_{n}$ be independent random vectors in $\mathbb{R}^{d}$. There exists a universal constants $C_1$ and $C_2$ such that \begin{equation} P\left\{ \max_{1\leq k \leq n} \| \sum_{i=1}^k Z_i \|_\infty > t \right\} \leq C_1 P\left\{ \| \sum_{i=1}^n Z_i \|_\infty > \frac{t}{C_2} \right\} \end{equation} for all $t>0$. \end{lemma} In particular, as \begin{equation*} \|\frac{2}{\sqrt{n}}\hat{u}(x^{(j)}; D_{i}) \|_{\psi_{1}} \leq\frac{2}{\sqrt{n}}\frac{\varphi}{\underline{\lambda}} , \end{equation*} and \begin{equation} \max_{j \in [d]} \sum_{k=1}^{\delta n / 2} \mathbb{E}\left[ \frac{2}{\sqrt{n}}\hat{u}(x^{(j)}; D_{i}) \right] = 2\delta , \end{equation} (ref) and (ref) imply that \begin{align} P\left\{ \max_{1\leq k\leq\delta\frac{n}{2}}\Big\|\frac{2}{\sqrt{n}}\sum_{i=1}^{k}\hat{u}\left(\bm{x}^{(d)};D_{i}\right)\Big\|_{\infty} \geq C \left(\delta \log^{1/2}(dn) + \frac{2}{\sqrt{n}}\frac{\varphi}{\underline{\lambda}} \log^2(dn)\right) \right\} \lesssim \frac{1}{n} . \end{align} Now, the choice \begin{equation} \delta=C \sqrt{\frac{\log n}{n}} \end{equation} gives \begin{equation} P\left\{ Q_{n}\geq\frac{\delta n}{2}\right\} \lesssim \frac{1}{n} \end{equation} by ((ref)). Plugging this choice into (ref) yields \begin{flalign} P\left\{ \max_{1\leq k\leq\delta\frac{n}{2}}\Big\|\frac{2}{\sqrt{n}}\sum_{i=1}^{k}\hat{u}\left(\bm{x}^{(d)};D_{i}\right)\Big\|_{\infty} \geq C \frac{1}{\sqrt{n}}\frac{\varphi}{\underline{\lambda}} \log^2(dn) \right\} \lesssim \frac{1}{n} . \end{flalign} Hence, we find that the inequality \begin{flalign*} P\left\{ \Big\|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(V_{i}-\tilde{V}_{i}\right)\hat{u}(\bm{x}^{(d)}, D_{i})\Big\|_{\infty} \geq C \left(\frac{\varphi^2\log^{4}\left(dn\right)}{\underline{\lambda}^2 n}\right)^{1/2} \right\} & \lesssim \frac{1}{n} \end{flalign*} holds for all $n$ sufficiently large. Thus, by setting \begin{equation*} t = C \left( \frac{\varphi^2\log^{4}\left(dn\right)}{\underline{\lambda}^2 n}\right)^{1/2} , \end{equation*} we find that \begin{align} & P\left\{ \frac{1}{n}\sum_{i=1}^{n}V_{i}\hat{u}(\bm{x}^{(d)}, D_i) \in \mathsf{R}^{\prime} \mid \mathbf{D}_n \right\} - P\left\{ \Lambda^{-1/2} Z\in \mathsf{R}^\prime \right\} \nonumber\\ & \quad\quad\quad\quad \lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^2 n}\right)^{1/4} + \sqrt{\log (d)}\left( \frac{\varphi^2 \log^{4}\left(dn\right)}{\underline{\lambda}^2 n}\right)^{1/2}\\ & \quad\quad\quad\quad \lesssim \left( \frac{\varphi^2 \log^{5}\left(dn\right)}{\underline{\lambda}^2 n}\right)^{1/4} , \end{align} with probability greater than $1-C n^{-1/2}\underline{\lambda}^{-1}\varphi \log^{3/2}(dn)$. Putting the pieces together, the inequalities (ref) and (ref) imply that \begin{flalign*} & P\left\{\sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} - P\left\{ Z\in\mathsf{R}\right\} \lesssim \left( \frac{\varphi^2 \log^{5}\left(dn\right)}{\underline{\lambda}^2 n}\right)^{1/4}+ \delta_n\sqrt{\log(d)} \end{flalign*} with probability greater than $1-C(n^{-1/2}\varphi \underline{\lambda}^{-1} \log^{3/2}(dn) + \rho_n)$, as required.\qed \subsection{Proof of (ref)} Again, we take $\theta_0(x)=0$ for all $x$, without loss of generality. Recall from the proof of (ref), Part (ii), that $V_{i}$ is a random variable taking the value $1$ when $i$ is an element of the subset $\mathsf{h}$, and taking the value $-1$ otherwise, and that \begin{flalign} \frac{2}{n}\sum_{i\in\mathsf{h}}^n \bar{u}(x, D_i) - \frac{1}{n}\sum_{i=1}^n \bar{u}(x, D_i) = \frac{1}{n} \sum_{i=1}^n V_i \bar{u}(x, D_i) . \end{flalign} To ease notation, define the objects \begin{align*} T_n(x) &= \frac{1}{n}\sum_{i=1}^n \bar{u}(x, D_i) - U_n(x)\quad\text{and}\quad T_\mathsf{h}(x)= \frac{2}{n}\sum_{i\in\mathsf{h}} \bar{u}(x, D_i) - U_\mathsf{h}(x) . \end{align*} We are interested in studying \begin{align*} \hat{\lambda}^2_{n,j} & = n\mathbb{E}_{V}\left[\left(R^*_n(x^{(j)})\right)^2 \right]\\ & = n\mathbb{E}_{V}\left[\left(\frac{1}{n} \sum_{i=1}^n V_i \bar{u}(x^{(j)}, D_i) + T_\mathsf{h}(x^{(j)}) - T_n(x^{(j)}) + \Delta_\mathsf{h}(x^{(j)}) - \Delta_n(x^{(j)})\right)^2\right] , \end{align*} where the notation $\mathbb{E}_{V}\left[\cdot\right]$ denotes that the expectation is evaluated only over the random variables $V_1,\ldots,V_n$ and we recall that the object $ \Delta_\mathsf{h}(x^{(j)})$ and $\Delta_n(x^{(j)})$ introduced in (ref) and (ref), respectively. On the event $\mathcal{E}(t)$, defined as the intersection of the events (ref) and (ref), we have that \begin{flalign*} & \sup_{j\in[d]} \big\vert \hat{\lambda}^2_{n,j} - \bar{\lambda}^2_{n,j} \big\vert \leq t^2 , \quad\text{where}\quad \bar{\lambda}^2_{n,j} = n\mathbb{E}_V\left[\left(\frac{1}{n} \sum_{i=1}^n V_i \bar{u}(x^{(j)}, D_i)\right)^2\right] . \end{flalign*} We can evaluate \begin{flalign} \bar{\lambda}^2_{n,j} &= \frac{1}{n}\sum_{i=1}^n \bar{u}^2(x^{(j)}, D_i) + \frac{1}{n}\sum_{i=1}^n \sum_{i^\prime \neq i} \mathbb{E}\left[V_i V_{i^\prime}\right] \bar{u}(x^{(j)}, D_i) \bar{u}(x^{(j)}, D_{i^\prime}) .\nonumber \end{flalign} Observe that \begin{flalign*} \mathbb{E}\left[V_i V_{i^\prime}\right] &= \frac{1}{2} \mathbb{E}\left[V_i \mid V_{i^\prime} = 1\right] - \frac{1}{2} \mathbb{E}\left[V_{i^\prime} \mid V_i = -1\right]\\ &= \frac{1}{2}\left(\frac{n/2-1}{n-1} - \frac{n/2}{n-1}\right) - \frac{1}{2} \left(\frac{n/2}{n-1} - \frac{n/2-1}{n-1}\right) = -\frac{1}{n-1} \end{flalign*} and thereby \begin{equation} \bar{\lambda}^2_{n,j} = \frac{1}{n}\sum_{i=1}^n \bar{u}^2(x^{(j)}, D_i) - \frac{1}{n}\frac{1}{n-1} \sum_{i=1}^n \sum_{i^\prime \neq i} \bar{u}(x^{(j)}, D_i) \bar{u}(x^{(j)}, D_{i^\prime}) . \end{equation} Now, observe that the first term in (ref) satisfies \begin{equation*} \sup_{j\in[d]} \Big\vert \frac{1}{n}\sum_{i=1}^n \bar{u}^2(x^{(j)}, D_i) - \lambda^2_{j} \Big\vert \lesssim \frac{\varphi}{n}\log(dn) \end{equation*} with probability greater than $1-n^{-1}$, by Bernstein's inequality (see e.g., Theorem 2.8.1 of vershynin2018high). To handle the second term in (ref), we apply the following sub-exponential formulation of the hanson1971bound exponential concentration inequality for quadratic forms, due to gotze2021concentration. \begin{lemma}[Proposition 1.1, gotze2021concentration] Let $X_1,\ldots,X_n$ be independent, centered, random variables satisfying $\mathbb{E}\left[X_i^2\right] = \sigma_{i}^2$ and $\|X_i\|_{\psi_1}\leq\varphi$. If $A=(a_{i,i^\prime})$ is any symmetric $n\times n$ matrix, then the inequality \begin{equation} P\left\{ \big\vert \sum_{i=1}^n \sum_{i^\prime=1}^n a_{i,i^\prime} X_i X_{i^\prime} - \sum_{i=1}^n \sigma_i^2 a_{i,i} \big\vert \geq t \right\} \leq 2 \exp\left( \frac{1}{C} \min \left(\frac{t^2}{\varphi^4 \|A\|^2_{F}}, \frac{t^{1/2}}{\varphi \|A\|_{\mathsf{op}}^{1/2} }\right) \right) \end{equation} holds for any $t\geq0$, where $\| \cdot \|_F$ and $\| \cdot \|_{\mathsf{op}}$ denote the Frobenius and $\ell_2$ operator norms, respectively. \end{lemma} In particular, if $A$ denotes the $n \times n$ matrix with zeroes on the diagonal and $(n(n-1))^{-1}$ in every other entry, then \begin{align*} \|A\|^2_{F} & = \frac{1}{n}\frac{1}{n-1}\quad\text{and}\quad \|A\|^{1/2}_{\mathsf{op}} = \frac{1}{n} \end{align*} and so (ref) implies that \begin{equation*} \sup_{j\in[d]} \Big\vert \frac{1}{n}\frac{1}{n-1} \sum_{i=1}^n \sum_{i^\prime\neq i} \bar{u}(x^{(j)}, D_i) \bar{u}(x^{(j)}, D_{i^\prime})\Big\vert \lesssim \frac{\varphi^2}{n} \log^2(dn) \end{equation*} with probability greater than $1-n^{-1}$. Thus, on the event $\mathcal{E}_n(t)$, we have that \begin{equation*} \sup_{j\in[d]} \big\vert \hat{\lambda}^2_{n,j} - \lambda^2_{j} \big\vert \lesssim \frac{\varphi^2}{n}\log^2(dn) + t^2 \end{equation*} with probability greater than $1-Cn^{-1}$, as $\varphi \geq 1$. By setting \[ t=C \sqrt{\frac{\underline{\lambda}^2}{n}} \delta_n~, \] (ref) and the bound (ref) imply that the event $\mathcal{E}_n(t)$ occurs with probability greater than $1-\rho_n$. Thus, we can conclude that \begin{equation*} \sup_{j\in[d]} \big\vert \hat{\lambda}_{n,j}/\lambda_{n,j} - 1\big\vert \lesssim \sup_{j\in[d]} \big\vert \hat{\lambda}^2_{n,j}/\lambda^2_{n,j} - 1\big\vert \lesssim \frac{\varphi^2}{\underline{\lambda}^2n}\log^2(dn) + \frac{1}{n}\delta_n^2 \end{equation*} with probability greater than $1-C(\rho_n+n^{-1})$, as required.\qed \section{Proofs for Lemmas Supporting the Proof of (ref)} \subsection{Proof of (ref)} To ease notation, we define the parameter \[ \phi(\theta_0) = (1+\|\theta_0(\bm{x}^{(d)})\|)\phi \] and drop dependence on $\theta(x)$ and $g$ when writing $u(x; D_{\mathsf{s}},\xi_{\mathsf{s}}, \theta(x), g)$, as these values will be taken to be $\theta_0(x)$ and $g_0$ throughout. We are interested in studying the discrepancy \begin{equation} U_n(\bm{x}^{(d)}) - \frac{b}{n}\sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) . \end{equation} Define the quantities \begin{flalign} \hat{U}_{n}(\bm{x}^{(d)}) & =\frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}} u(\bm{x}^{(d)}; D_{\mathsf{s}},\xi_{\mathsf{s}}) \quad\text{and}\quad \bar{U}_{n}(\bm{x}^{(d)}) = \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \tilde{u}(\bm{x}^{(d)}; D_\mathsf{s}) \end{flalign} for some collection of independent random variables $\bm{\xi} = (\xi_{\mathsf{s}})_{\mathsf{s}\in\mathcal{S}_{n,b}}$ having the same distribution as $\xi$. The statistics $\hat{U}_{n}(\bm{x}^{(d)})$ and $\bar{U}_{n}(\bm{x}^{(d)})$ are the complete, randomized and de-randomized, $U$-statistics associated with the randomized $d$-dimensional kernel $u(\bm{x}^{(d)};\cdot,\cdot)$, respectively. Consider the decomposition \begin{align} & \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} U_n(\bm{x}^{(d)}) - \frac{b}{n}\sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \nonumber\\ & \quad\quad= \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} (U_{n}(\bm{x}^{(d)})-\hat{U}_{n}(\bm{x}^{(d)})) +\sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} (\hat{U}_{n}(\bm{x}^{(d)})-\bar{U}_{n}(\bm{x}^{(d)})) \nonumber \\ & \quad\quad\quad+\sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} (\bar{U}_{n}(\bm{x}^{(d)})-\frac{b}{n}\sum_{i=1}^{n}\tilde{u}^{(1)}(\bm{x}^{(d)}; D_{i})) . \end{align} A high-probability bound for the third term in (ref) is obtained by applying (ref), stated in (ref). In particular, (ref) implies that \begin{align} \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} & \Big\| \bar{U}_{n}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \Big\|_{\infty} \nonumber\\ & \quad\quad \lesssim \left(\frac{C^\star b\log(dn)}{n}\right)^{b/2} \left( \left(\frac{n}{b^2 \underline{\sigma}^2_{b}}\right)^{1/2} +\left(\frac{b\log^4(dn)}{\underline{\sigma}^2_{b}}\right)^{1/2} \right) \phi(\theta_0) \end{align} with probability greater than $1-Cn^{-1}$, for some constant $C^\star > 1$. By choosing $C_1 = (C^\star)^{-1} (C^{\star\star})<1$ for some $(C^{\star\star})< 1$, the conditions \[ \frac{b\log(dn)}{n} \leq C_1 < 1 \quad\text{and}\quad b^{-C_2} \leq \underline{\sigma}^2_b \] imply that \begin{align} & \left(\frac{C^\star b\log(dn)}{n}\right)^{b/2} \left( \left(\frac{n}{b^2 \underline{\sigma}^2_{b}}\right)^{1/2} +\left(\frac{b\log^4(dn)}{\underline{\sigma}^2_{b}}\right)^{1/2} \right) \phi(\theta_0)\nonumber \\ & \quad\quad \lesssim \left(C{\star\star}\right)^{b/2} \left( \left(n b^{C_2 - 2}\right)^{1/2} +\left(b^{1+C_2}n^4\right)^{1/2} \right) \phi(\theta_0)\nonumber \\ & \quad\quad \lesssim \left(C^{\star\star}\right)^{b/2} n^{(5+C_2)/2} \phi(\theta_0) . \end{align} Observe that the conditions \begin{equation} q \log(C^{\star\star}) + \log(n) \leq 0\quad\text{and}\quad (C^{\star\star})^q \leq n^{-1} \end{equation} are equivalent. Consequently, if \begin{equation} b \geq (7 + C_2) \frac{-\log(n)}{\log(C^{\star\star})} \end{equation} then \begin{align} (C^{\star\star})^{b/2} & \lesssim \left((C^{\star\star})^{\frac{-\log(n)}{\log(C^{\star\star})}}\right)^{(7 + C_2)/2} \lesssim n^{-(7 + C_2)/2} . \end{align} Hence, if (ref) holds, then the bounds (ref) and (ref) imply that \begin{align} \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} & \Big\| \bar{U}_{n}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \Big\|_{\infty} \lesssim n^{-1} \phi(\theta_0) \end{align} holds with probability greater than $1-Cn^{-1}$. Bounds for the first two terms in (ref) are obtained through the application of the following Lemma. \begin{lemma} Let $\mathbf{D}_n = (D_i)_{i=1}^n$ and $\bm{\xi} = (\xi_{\mathsf{s}})_{\mathsf{s}\in\mathcal{S}_{n,b}}$ denote two independent collections of independent and identically distributed random variables. Consider the function $u(D_{\mathsf{s}}, \xi_{\mathsf{s}})$ valued in $\mathbb{R}^d$. Assume that the absolute value of each component of $u(D_{\mathsf{s}}, \xi_{\mathsf{s}})$ is bounded by the constant $\phi\geq1$ almost surely. \textbf{(i)} If there exists some constant $c < 1 $ such that the bound \begin{equation} \frac{\phi b\log^{1/2}(dn)}{n} < c \end{equation} holds, then the bound \begin{equation} \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \left(u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})-\mathbb{E}\left[u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}}) \mid \mathbf{D}_{\mathsf{s}_q} \right]\right) \lesssim \frac{ b \phi \log^{1/2}(dn)}{n} \end{equation} holds with probability greater than $1-n^{-1}$ for all $b>2$. \textbf{(ii)} Moreover, if, in addition, the sets $(\mathsf{s}_q)_{q=1}^r$ are drawn independently and identically from $\mathcal{S}_{n,b}$ and $n\leq \sqrt{r} b$, then the bound \begin{equation} \frac{1}{r}\sum_{q=1}^{r} \left(u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})-\mathbb{E}\left[u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}}) \mid \mathbf{D}_{n}, \bm{\xi} \right]\right) \lesssim \frac{ b \phi \log^{1/2}(dn)}{n} \end{equation} holds with probability greater than $1-n^{-1}$. \end{lemma} In particular, observe that \[ \hat{U}_{n,b} =\frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}u(D_{\mathsf{s}},\xi_{\mathsf{s}}) =\mathbb{E}\left[u(D_{\mathsf{s}_{g}},\xi_{\mathsf{s}_{g}})\mid\mathbf{D}_{n},\bm{\xi}\right] \] and that therefore we can write \[ U_{n}-\hat{U}_{n} =\frac{1}{r}\sum_{q=1}^{r}Z_{q}, \quad\text{with}\quad Z_{q}=u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})-\mathbb{E}\left[u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})\mid\mathbf{D}_{n},\bm{\xi}\right]~. \] Consequently, by the normalization $\phi(\theta_0) b\log(dn)/n \leq C_1 <1$, Part (ii) of (ref) implies that \begin{align} \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} (U_{n}(\bm{x}^{(d)})-\hat{U}_{n}(\bm{x}^{(d)})) & \lesssim \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} \frac{ b \phi(\theta_0) \log^{1/2}(dn)}{n} = \left(\frac{\phi(\theta_0)^2 \log(dn)}{\underline{\sigma}^2_b n}\right)^{1/2} \end{align} with probability greater than $1 - n^{-1}$. In turn, we can similarly write \[ \hat{U}_{n}-\bar{U}_{n} =\frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}Z_{\mathsf{s}} \quad\text{with}\quad Z_{\mathsf{s}} =u(D_{\mathsf{s}},\xi_{\mathsf{s}}) -\mathbb{E}\left[u(D_{\mathsf{s}},\xi_{\mathsf{s}})\mid D_{\mathsf{s}} \right]~. \] Thus, again by the normalization $\phi(\theta_0) b\log(dn)/n \leq C_1 <1$, Part (i) of (ref) implies that \begin{align} \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} (\hat{U}_{n}(\bm{x}^{(d)})-\bar{U}_{n}(\bm{x}^{(d)})) & \lesssim \left(\frac{\phi(\theta_0)^2 \log(dn)}{\underline{\sigma}^2_b n}\right)^{1/2} \end{align} with probability greater than $1 - n^{-1}$. Putting the pieces together, the bounds (ref), (ref), and (ref) imply that \begin{align} & \sqrt{\frac{n}{b^2 \underline{\sigma}^2_b}} \Big\| \bar{U}_{n}(\bm{x}^{(d)}) - \frac{b}{n} \sum_{i=1}^n \tilde{u}^{(1)}(\bm{x}^{(d)}; D_i) \Big\|_{\infty} \nonumber\\ & \lesssim \frac{\phi(\theta_0)}{n} + \left(\frac{\phi(\theta_0)^2 \log(dn)}{\underline{\sigma}^2_b n}\right)^{1/2} \lesssim \left( \frac{\phi(\theta_0)^2\log(dn)}{n \underline{\sigma}^2_b} \right)^{1/2} , \end{align} with probability greater than $1-C/n$, where the second inequality follows from the fact that $\underline{\sigma}^2_b \lesssim b^{-1}$, which is established in the proof of (ref).\qed \subsection{Proof of (ref), Part (i)} Observe that \begin{align*} \mathbb{E}\left[M_n(x; \theta, g) \right] &= \mathbb{E}\left[\sum_{i=1}^n K(x, X_i) m(D_i; \theta, g) \right]\\ &= \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \sum_{i\in \mathsf{s}} \mathbb{E}\left[\kappa(x, X_i, \mathsf{s}, \xi) m(D_i; \theta, g) \right]\\ &= \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \sum_{i\in \mathsf{s}} \mathbb{E}\left[ \mathbb{E}\left[\kappa(x, X_i, \mathsf{s}, \xi) \mid X_i, D_{\mathsf{s}_{-i}}\right] \mathbb{E}\left[ m(D_i; \theta, g)\mid X_i, D_{\mathsf{s}_{-i}}\right] \right]\tag{Honesty}\\ &= \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \sum_{i\in \mathsf{s}} \mathbb{E}\left[ \kappa(x, X_i, \mathsf{s}, \xi) \mathbb{E}\left[ m(D_i; \theta, g)\mid X_i\right] \right]\\ &= \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \sum_{i\in \mathsf{s}} \mathbb{E}\left[ \kappa(x, X_i, \mathsf{s}, \xi) M(X_i; \theta, g) \right] . \end{align*} Therefore, the normalization \begin{equation} \sum_{i \in \mathsf{s}} \kappa(x, X_i, \mathsf{s}, \xi) = 1 \end{equation} implies that \begin{equation*} \mathsf{Bias}_n(x; \theta, g) = \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \sum_{i\in \mathsf{s}} \mathbb{E}\left[ \kappa(x, X_i, \mathsf{s}, \xi) \left(M(X_i; \theta, g) - m(x; \theta, g)\right)\right] . \end{equation*} By the boundedness part of (ref) and Part (iii) of (ref), we find that \begin{equation} \mathsf{Bias}_n(x; \theta, g) \lesssim (1 + \vert\theta\vert) \mathbb{E} \left[ \kappa(x, X_i, \mathsf{s}, \xi) \|X_i - x\|_\infty \right] \leq (1 + \vert\theta\vert) \varepsilon_{b} , \end{equation} where final inequality follows from the definition of the shrinkage rate $\varepsilon_n$ and the normalization (ref). \subsection{Proof of (ref), Part (ii)} Define the function \begin{equation*} J(x; D_\mathsf{s},\xi_\mathsf{s}) = \sum_{i\in \mathsf{s}} \left(\kappa(x,X_i,\mathsf{s},\xi_\mathsf{s})m^{(1)}(D_i; g_0) - \mathbb{E}\left[\kappa(x,X_i \mathsf{s},\xi_\mathsf{s}) m^{(1)}(D_i; g_0)\right]\right) \end{equation*} and observe that \begin{equation} \bar{M}_n^{(1)}(x; g_0) = \frac{1}{r} \sum_{q = 1}^n J(x; D_{\mathsf{s}_q},\xi_{\mathsf{s}_q}) . \end{equation} Consider the decomposition \begin{equation} \bar{M}_n^{(1)}(x,g_0) = \tilde{A}(x) + \hat{A}(x) + \bar{A}(x) , \end{equation} where \begin{align} \tilde{A}(x) & = \frac{1}{r} \sum_{q = 1}^n \left(J(x; D_{\mathsf{s}_q},\xi_{\mathsf{s}_q}) - \mathbb{E}\left[J(x; D_{\mathsf{s}_q},\xi_{\mathsf{s}_q}) \mid \mathbf{D}_n, \bm{\xi}\right]\right) ,\\ \hat{A}(x) & = \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \left(J(x; D_{\mathsf{s}},\xi_\mathsf{s}) - \mathbb{E}\left[J(x; D_{\mathsf{s}},\xi_{\mathsf{s}} \mid D_\mathsf{s}\right]\right) ,\quad\text{and}\\ \bar{A}(x) & = \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \mathbb{E}\left[J(x; D_{\mathsf{s}},\xi_{\mathsf{s}}) \mid D_\mathsf{s}\right] , \end{align} respectively. We again apply (ref) to bound (ref) and (ref). In particular, by (ref), the normalization (ref), and the restriction $n\leq b \sqrt{r}$, (ref) implies that \begin{align} P\left\{ \| \tilde{A}(\bm{x}^{(d)}) \|_\infty \geq C \frac{b\phi \log^{1/2}(dn)}{n} \right\} & \lesssim \frac{1}{n}\quad\text{and}\\ P\left\{ \| \hat{A}(\bm{x}^{(d)}) \|_\infty \geq C\frac{b\phi \log^{1/2}(dn)}{n} \right\} & \lesssim \frac{1}{n} \end{align} respectively. In turn, we apply (ref), stated in (ref), to bound the term (ref). In this case, by (ref), (ref) imply that \begin{align} \| \bar{A}_n(\bm{x}^{(d)}) \|_\infty & \lesssim \left(\frac{b \phi^2 \log(dn)}{n}\right)^{1/2} + \left(\frac{b \phi \log^{2}(dn)}{n} \right)^{1/2} \left(\frac{b \phi \log^{2}(dn)}{n}\right)^{1/2}\nonumber\\ & \lesssim \left(\frac{b \phi^2 \log^2(dn)}{n}\right)^{1/2} \end{align} with probability greater than $1-Cn^{-1}$, where the second inequality follows from the normalization (ref). Thus, the bounds (ref), (ref) and (ref) imply that \begin{equation} P\left\{ \| \bar{M}_n^{(1)}(\bm{x}^{(d)};g_0) \|_\infty \geq C \left(\frac{b \phi^2 \log^2(dn)}{n}\right)^{1/2} \right\} \lesssim \frac{1}{n} , \end{equation} as required.\qed \subsection{Proof of (ref), Part (iii)} We give the details of the proof of the stated probability bound on the discrepancy \begin{equation} \big\| \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), \hat{g}_n) - \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), g_0) \big\|_\infty \end{equation} only. The argument giving the analogous bound associated with the term $\bar{M}^{(1)}_n(\bm{x}^{(d)}; g)$ is identical. Define the functions \begin{align} W(x; D_{\mathsf{s}}, \xi_{\mathsf{s}}, g) & = \sum_{i\in\mathsf{s}} f(x, X_i; D_{\mathsf{s}}, \xi_{\mathsf{s}}, g) \nonumber \quad\text{and}\quad \bar{W}(x; D_{\mathsf{s}}, g) = \mathbb{E}_{\xi_{\mathsf{s}}}\left[ f(x; D_{\mathsf{s}}, \xi_{\mathsf{s}}, g)\right] , \end{align} where \begin{align} f(x, X_i; D_{\mathsf{s}}, \xi_{\mathsf{s}}, g) & = \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) \left( m(D_i ; \theta_0, g) - m(D_i ; \theta_0, g_0) \right)\nonumber \\ & \quad\quad\quad - \mathbb{E}[\kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) \left( m(D_i ; \theta_0, g) - m(D_i ; \theta_0, g_0) \right) ] \end{align} and the notation $\mathbb{E}_{A}[\cdot]$ indicates that we are evaluating the expectation over the randomness in $A$. Define the quantities \begin{align} \tilde{W}_n(x; g) &= \frac{1}{r}\sum_{q=1}^r \left(W(x; D_{\mathsf{s}_q}, \xi_{\mathsf{s}_q}, g) - \mathbb{E}_\mathsf{s}\left[W(x; D_{\mathsf{s}}, \xi_{\mathsf{s}}), g\right]\right) ,\\ \hat{W}_n(x; g) &= \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \left(W(x; D_{\mathsf{s}}, \xi_{\mathsf{s}}, g) - \bar{W}(x; D_{\mathsf{s}}, g) \right) ,\quad\text{and}\\ \bar{W}_n(x; g) &= \frac{1}{N_b} \sum_{\mathsf{s}\in\mathcal{S}_{n,b}} \bar{W}(x; D_{\mathsf{s}}, g) . \end{align} and consider the decomposition \begin{align} \bar{M}_n(x;\theta_0(x), g) - \bar{M}_n(x;\theta_0(x), g_0) = \tilde{W}_n(x, g) + \hat{W}_n(x, g) + \bar{W}_n(x, g) . \end{align} We again apply (ref) to bound (ref) and (ref). In particular, by (ref), the normalization (ref), and the restriction $n\leq b \sqrt{r}$, (ref) imply that \begin{align} P\left\{ \| \tilde{W}(\bm{x}^{(d)}; g) \|_\infty \geq C \frac{b(1 + \| \theta_0(\bm{x}^{(d)}) \|_\infty) \phi \log^{1/2}(dn)}{n} \right\} & \lesssim \frac{1}{n}\quad\text{and}\\ P\left\{ \| \hat{W}(\bm{x}^{(d)}; g) \|_\infty \geq C\frac{b(1 + \| \theta_0(\bm{x}^{(d)}) \|_\infty) \phi \log^{1/2}(dn)}{n} \right\} & \lesssim \frac{1}{n} \end{align} respectively. To bound the term (ref), we apply (ref). In particular, observe that \begin{equation*} \mathbb{E}[\bar{W}(\bm{x}^{(d)}; D_{\mathsf{s}}, g)] = 0 \quad\text{and}\quad \|\bar{W}(\bm{x}^{(d)}; D_{\mathsf{s}}, g)\|_{\psi_1} \lesssim (1 + \|\theta(\bm{x}^{(d)})\|_\infty) \phi \end{equation*} by the boundedness part of (ref). In turn, observe that \begin{align*} \mathbb{E}\left[ \left( \bar{W}(x; D_{\mathsf{s}}, g) \right)^2 \right] & \leq \mathbb{E} \left[ \sum_{i\in\mathsf{s}} \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) \mathbb{E}\left[\left( m(D_i ; \theta_0, g) - m(D_i ; \theta_0, g_0) \right)^2\mid X_i \right] \right]\\ & \lesssim \mathbb{E} \left[ \sum_{i\in\mathsf{s}} \kappa(x, X_i, D_{\mathsf{s}}, \xi_{\mathsf{s}}) V(x, g) \right] + \varepsilon_{b}\\ & \lesssim \|g - g_0 \|_\infty^2+ \varepsilon_{b} \end{align*} where the first inequality follows from Honesty and Jensen's inequality, the second inequality follows from (ref), Part (ii), and the definition of the shrinkage rate $\varepsilon_{b}$, and the third inequality follows from (ref), Part (ii), and the normalization that $\sum_{i\in\mathsf{s}} \kappa(x,X_i,D_\mathsf{s},\xi) = 1$ almost surely. Thus, we obtain \begin{align} &P\left\{ \| \bar{W}_n(\bm{x}^{(d)}; g) \| \geq C \xi^\prime_{n}(g) \right\} \lesssim \frac{1}{n} ,\quad\text{where}\\ & \xi^\prime_{n}(g) = \sqrt{\frac{b (\|g - g_0 \|_{2,\infty}^2+ \varepsilon_{b})\log(dn)}{n}} + \frac{b(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty) \phi \log^2(dn)}{n} .\nonumber \end{align} Consequently, as $\hat{g}_n$ is statistically independent of $\mathbf{D}_n$, the decomposition (ref) and the bounds (ref), (ref), and (ref), imply that \begin{align} \| \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), \hat{g}_n) - \bar{M}_n(x;\theta_0(\bm{x}^{(d)}), g_0) \|_\infty \lesssim \xi^\prime_{n}(\hat{g}_n) , \end{align} with probability greater than $1-Cn^{-1}$. Now, observe that \begin{align} \sqrt{\frac{b (\|\hat{g}_n - g_0 \|_{2,\infty}^2+ \varepsilon_{n})\log(dn)}{n}} & \lesssim \sqrt{\frac{b \log(dn)}{n}}\|\hat{g}_n - g_0 \|_{2,\infty} + \sqrt{\frac{b\log(dn)}{n}} \varepsilon^{1/2}_{b} \nonumber \\ & \lesssim \sqrt{\frac{b\log(dn)}{n}} \left(\left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon^{1/2}_{b}\right) \end{align} with probability greater than $1-n^{-1}$. Putting the pieces together, the bounds (ref) and (ref) imply that \begin{align} &\big\| \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), \hat{g}_n) - \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}), g_0) \big\|_\infty \nonumber \\ &\lesssim \sqrt{\frac{b\log(dn)}{n}} \left(\left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon^{1/2}_{b}\right) + \frac{b}{n}(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty) \phi \log^2(dn) \nonumber \\ &\lesssim \sqrt{\frac{b\log(dn)}{n}} \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon_b + \frac{b}{n}(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty) \phi \log^2(dn) \end{align} with probability greater than $1-n^{-1}$, as required.$\blacksquare$ \subsection{Proof of (ref), Part (iv)} Recall the decomposition \begin{align} &M^{(1)}(x;\theta_0(x),g_0) (\hat{\theta}_n(x) - \theta_0(x))\\ & = -\bar{M}_n(x;\theta_0(x),g_0) \\ & \quad+ \mathsf{Bias}(x;\hat{\theta}_n(x),\hat{g}_n) + \mathsf{Nuis}(x;\theta_0(x),\hat{g}_n) \\ & \quad+ \mathsf{Stoch}^{(1)}(x; \hat{\theta}_n(x),\hat{g}_n) + \mathsf{Stoch}^{(2)}(x; \hat{g}_n) \\ & \quad- (\hat{\theta}_n(x) - \theta_0(x)) \left(M^{(1)}(x; \theta_0(x), \hat{g}_n) - M^{(1)}(x; \theta_0(x), g_0) \right) \end{align} stated as (ref) in the Proof of (ref), Part (i), where we note that the various terms appearing in (ref) are defined in (ref) through (ref). Here, we have used the fact that the Hessian $H(x; \theta, g) = 0$ almost surely, by (ref). First, observe that, by an argument identical to the argument used to establish Part (ii) of this Lemma, i.e., (ref), we have that \begin{equation} \| \bar{M}_n(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}),g_0) \|_\infty \lesssim \left(\frac{b \phi^2(1 + \| \theta(\bm{x}^{(d)})\|_\infty)^2 \log^2(dn)}{n}\right)^{1/2} \end{equation} with probability greater than $1-Cn^{-1}$. Moreover, as before, we have that \begin{align} \bigg\|(\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)})) \left(M^{(1)}(\bm{x}^{(d)}; \theta_0(\bm{x}^{(d)}), \hat{g}_n) - M^{(1)}(\bm{x}^{(d)}; \theta_0(\bm{x}^{(d)}), g_0) \right)\bigg\|_\infty \nonumber \\ \lesssim \| \hat{g}_n - g_0 \|_{2,\infty} \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \nonumber \\ \lesssim \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \end{align} with probability greater than $1-n^{-1}$, where the first inequality follows from Part (iii) of (ref) and the second inequality follows from the rate condition (ref). In turn, observe that Part (i) of this Lemma gives that \begin{align} \| \mathsf{Bias}_n(\bm{x}^{(d)};\theta(\bm{x}^{(d)}), \hat{g}_n) \|_\infty \lesssim (1 + \| \theta(\bm{x}^{(d)})\|_\infty) \varepsilon_{b} \end{align} and that Neyman orthogonality give that \begin{align} \| \mathsf{Nuis}(\bm{x}^{(d)};\theta_0(\bm{x}^{(d)}),\hat{g}_n)\|_\infty \lesssim \| \hat{g}_n - g_0 \|^2_{2,\infty} \lesssim \sqrt{\frac{b}{n}} \delta^2_{n,g} \end{align} with probability greater than $1-n^{-1}$ via the expansion (ref) and the the rate condition (ref). Moreover, as before, by (ref), a Taylor expansion gives \begin{align} \vert \mathsf{Stoch}^{(1)}(x; \hat{\theta}_n(x),\hat{g}_n) \vert & = \vert (\hat{\theta}_n(x) - \theta_0(x)) \bar{M}_n^{(1)}(x; \theta_0(x), \hat{g}_n) \vert \nonumber\\ & = \vert \hat{\theta}_n(x) - \theta_0(x) \vert \bar{M}_n^{(1)}(x; \theta_0(x), g_0)\vert \nonumber\\ & + \vert \hat{\theta}_n(x) - \theta_0(x) \vert \vert \bar{M}_n^{(1)}(x; \theta_0(x), \hat{g}_n) - \bar{M}_n^{(1)}(x; \theta_0(x), g_0)\vert \end{align} where we have used the fact that the empirical Hessian $\bar{H}_n(x; \tilde{\theta}_0(x), \hat{g}_n) = 0$ almost surely, by (ref). Consequently, we find that \begin{align} & \| \mathsf{Stoch}^{(1)}(\bm{x}^{(d)}; \hat{\theta}_n(\bm{x}^{(d)}),\hat{g}_n) \|_\infty \nonumber\\ & \lesssim \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \| \bar{M}^{(1)}_n(\bm{x}^{(d)}; g_0)\|_\infty \nonumber\\ & \lesssim \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \big\| \bar{M}^{(1)}_n(\bm{x}^{(d)}; \hat{g}_n) - \bar{M}^{(1)}_n(\bm{x}^{(d)}; g_0) \big\|_\infty \nonumber\\ & \lesssim \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \Bigg(\left(\frac{b \phi^2 \log^2(dn)}{n}\right)^{1/2} \nonumber\\ & \quad\quad+ \sqrt{\frac{b\log(dn)}{n}} \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon_{n} + \frac{b\log^2(dn)}{n}(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty) \phi \Bigg) \nonumber \\ & \lesssim \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \left( \left(\frac{b \phi^2 (1 + \|\theta_0(\bm{x}^{(d)}\|_\infty)^2 \log^2(dn)}{n}\right)^{1/2} + \varepsilon_b\right) \end{align} with probability greater than $1-C/n$, where the second to last inequality follows from Parts (ii) and (iii) of this Lemma and the final inequality follows from the normalizations (ref) and (ref). Similarly, Part (ii) of this Lemma gives that \begin{align} \| \mathsf{Stoch}^{(2)}(\bm{x}^{(d)}; \hat{g}_n) \|_\infty & \lesssim \sqrt{\frac{b\log(dn)}{n}} \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} + \varepsilon_{b} + \frac{b\log^2(dn)}{n}(1 + \|\theta_0(\bm{x}^{(d)})\|_\infty) \phi \nonumber \\ &\lesssim \left(\left(\frac{b \phi^2 (1 + \|\theta_0(\bm{x}^{(d)}\|_\infty)^2 \log^2(dn)}{n}\right)^{1/2}+ \varepsilon_{b}\right) \end{align} with probability greater than $1-n^{-1}$, where the second inequality follows from the normalizations (ref) and (ref). Consequently, plugging the bounds (ref), (ref), (ref), (ref), (ref), and (ref) into the decomposition (ref), we find that \begin{align} \|\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)})\|_{\infty} & \lesssim \sqrt{\frac{b \log^2(dn)}{n}} (1 + \| \theta(\bm{x}^{(d)})\|_\infty)\phi \\ & + \left(\frac{b}{n}\right)^{1/4} \delta_{n,g} \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \\ &+ (1 + \| \theta_0(\bm{x}^{(d)})\|_\infty) \varepsilon_{b} + \sqrt{\frac{b}{n}} \delta^2_{n,g} \\ &+ \| \hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)}) \|_\infty \sqrt{\frac{b \log^2(dn)}{n}} (1 + \| \theta(\bm{x}^{(d)})\|_\infty)\phi \\ & + \sqrt{\frac{b \log^2(dn)}{n}} (1 + \| \theta(\bm{x}^{(d)})\|_\infty)\phi \end{align} with probability greater than $1-Cn^{-1}$, where we have used the fact that $M^{(1)}(x;\theta_0(x),g_0)$ is bounded from below by Part (iii) of (ref). Consequently, we find that \begin{align} \|\hat{\theta}_n(\bm{x}^{(d)}) - \theta_0(\bm{x}^{(d)})\|_{\infty} & \lesssim \sqrt{\frac{b \log^2(dn)}{n}} (1 + \| \theta(\bm{x}^{(d)})\|_\infty)\phi \\ & + (1 + \| \theta_0(\bm{x}^{(d)})\|_\infty) \varepsilon_{b} + \sqrt{\frac{b}{n}} \delta^2_{n,g} , \end{align} with probability greater than $1-Cn^{-1}$, as required.\qed \subsection{Proofs of (ref)} We begin by considering the quantity \begin{equation} \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}} Z_{\mathsf{s}} \quad\text{where}\quad Z_{\mathsf{s}} = u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})-\mathbb{E}\left[u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}}) \mid D_{\mathsf{s}} \right] . \end{equation} Conditioned on the data $\mathbf{D}_n$, the observations $Z_{\mathsf{s}}$, $\mathsf{s}\in\mathcal{S}_{n,b}$, are centered and mutually independent. Moreover, each component of $Z_{\mathsf{s}}$ is bounded by $2\phi$ almost surely. Thus, (ref) implies that \begin{align} \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}} Z_{\mathsf{s}} & \lesssim \frac{\phi \log^{1/2}(dn)}{N_b^{1/2}} + \frac{\phi ^2\log(dn)}{N_b} \end{align} with probability greater than $1 - n^{-1}$. As $(n/b)^b \lesssim N_b$, for each $b$, we have that \begin{align} & \frac{\phi \log^{1/2}(dn)}{N_b^{1/2}} + \frac{\phi ^2\log(dn)}{N_b} \nonumber \\ & \quad \lesssim \frac{b \phi \log^{1/2}(dn)}{n} + \frac{b\phi \log^{1/2}(dn)}{n}\frac{b \phi \log^{1/2}(dn)}{n} \nonumber \\ & \quad \lesssim \frac{b \phi \log^{1/2}(dn)}{n} \end{align} for each $b\geq 2$, where in the second inequality we have used the normalization $b \phi \log^{1/2}(dn) / n <c$. The desired inequality (ref) follows from (ref) and (ref). Next, we consider the quantity \[ \frac{1}{r}\sum_{q=1}^{r}Z_{q}, \quad\text{with}\quad Z_{q}=u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})-\mathbb{E}\left[u(D_{\mathsf{s}_{q}},\xi_{\mathsf{s}_{q}})\mid\mathbf{D}_{n},\bm{\xi}\right]~. \] Conditioned on the data $\mathbf{D}_n$ and the residual randomness $\bm{\xi}$, the observations $Z_{q}$, $q\in[r]$, are centered and mutually independent. Moreover, each component of $Z_{q}$ is bounded by $\phi$ almost surely. Consequently, (ref) implies that \begin{align} \frac{1}{r}\sum_{q=1}^{r}Z_{q} & \lesssim \frac{\phi(\theta_0) \log^{1/2}(dn)}{r^{1/2}} + \frac{\phi(\theta_0) ^2\log(dn)}{r} \end{align} with probability greater than $1 - n^{-1}$. Now, the restriction $n\leq \sqrt{r} b$ implies that \begin{align} & \frac{\phi \log^{1/2}(dn)}{r^{1/2}} + \frac{\phi ^2\log(dn)}{r} \nonumber \\ & \quad \lesssim \frac{b \phi \log^{1/2}(dn)}{n} + \frac{b\phi \log^{1/2}(dn)}{n}\frac{b \phi \log^{1/2}(dn)}{n} \nonumber \\ & \quad \lesssim \frac{b \phi \log^{1/2}(dn)}{n} , \end{align} where in the second inequality we have used the normalization $b \phi \log^{1/2}(dn) / n <c$. The desired inequality (ref) follows from (ref) and (ref).\qed \section{Proofs and Figures for (ref)} \subsection{Proof of (ref), Part (i)} The result follows from an argument very similar to the proof of (ref), Part (ii). Again, we take $\theta_0(x)=0$ for all $x$, without loss of generality. Here, we are interested in studying the discrepancy \begin{flalign*} R_{n}^{*}(x) & = \left(\frac{2Q_n}{n}\right)\hat{\theta}_\mathsf{s}(x)-\hat{\theta}_n(x) = \left(\frac{2Q_n}{n}\right) \left(\hat{\theta}_\mathsf{s}(x)-\theta_{0}(x)\right)-R_{n}(x) . \end{flalign*} Analogously to (ref), we can write \begin{flalign} \left(\frac{2Q_n}{n}\right) \left(\hat{\theta}_\mathsf{s}(x)-\theta_{0}(x)\right) & = \frac{2Q_n}{n} \left(\frac{1}{Q_n}\sum_{i\in\mathsf{s}} \bar{u}(x, D_i) - U_\mathsf{s}(x)\right) \\ & \quad- \frac{2}{n}\sum_{i\in\mathsf{s}} \bar{u}(x, D_i) + \frac{2Q_n}{n} \Delta_\mathsf{s}(x) , \end{flalign} where $U_\mathsf{s}(x)$ is analogous to $U_n(x)$, but is constructed with the data with indices in the subsample $\mathsf{s}$, and \begin{flalign} \Delta_\mathsf{s}(x) & = R_{n}^{*}(x) -U_\mathsf{s}(x) . \end{flalign} Let $\mathcal{Q}_n(t_0)$ denote the event that \begin{equation} \bigg\vert \frac{2Q_n}{n} - 1\bigg\vert \leq t_0 . \end{equation} Similarly, let $\mathcal{F}^\prime_n(t)$ and $\mathcal{H}^\prime_n(t)$ denote the events that \begin{flalign} \sqrt{n}\| \Lambda^{-1/2} \Delta_\mathsf{s}(\bm{x}^{(d)})\|_\infty & \leq t/4,\quad\text{and}\\ \sqrt{n}\| \frac{2}{n}\sum_{i\in\mathsf{s}}^n \hat{u}(\bm{x}^{(d)}, D_i) - \hat{U}_\mathsf{s}(\bm{x}^{(d)}) \|_\infty &\leq t/4 , \end{flalign} respectively, where $\hat{U}_\mathsf{s}(\bm{x}^{(d)}) $ is again defined analogously to $\hat{U}_n(\bm{x}^{(d)}) $. Define the event $\mathcal{E}^\prime_n(t,t_0) = \mathcal{F}_n(t) \cap \mathcal{F}_\mathsf{h}(t) \cap \mathcal{H}_n(t) \cap \mathcal{H}_\mathsf{h}(t)\cap \mathcal{Q}_n(t_0)$. Fix a hyper-rectangle $\mathsf{R}$ in $\mathcal{D}$, and again recall the normalized and enlarged hyper-rectangles $\tilde{\mathsf{R}}$ and $\tilde{\mathsf{R}}_t$. On the event $\mathcal{E}^\prime_n(t,t_0)$, we have \begin{flalign} & \vert P\left\{\sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} - P\left\{ Z\in\mathsf{R}\right\} \vert\nonumber \\ & = \vert P\left\{\sqrt{n} \Lambda^{-1/2} R_{n}^{*}(\bm{x}^{(d)})\in\tilde{\mathsf{R}} \mid\mathbf{D}_{n}\right\} - P\left\{ \Lambda^{-1/2} Z\in\tilde{\mathsf{R}}\right\} \vert\nonumber \\ & \quad\leq\vert P\left\{ \frac{2}{\sqrt{n}}\sum_{i\in\mathsf{s}}^n \hat{u}(\bm{x}^{(d)}, D_i) - \frac{1}{\sqrt{n}}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i)\in\tilde{\mathsf{R}}_{t(1+t_0)} \mid\mathbf{D}_{n}\right\} -P\left\{ \Lambda^{1/2} Z\in\tilde{\mathsf{R}}_{t(1+t_0)}\right\} \vert\nonumber \\ & \quad+\vert P\left\{ \Lambda{Z}\in\tilde{\mathsf{R}}_{t(1+t_0)}\right\} -P\left\{ \Lambda^{1/2} Z\in\tilde{\mathsf{R}}\right\} \vert\nonumber \end{flalign} for each $t>0$. Let $\tilde{V}_i$ be a random variable taking the value $1$ with $i$ is an element of the subset $\mathsf{s}$ and taking the value $-1$ otherwise. Observe that \begin{flalign} \frac{2}{n}\sum_{i\in\mathsf{s}} \hat{u}(\bm{x}^{(d)}, D_i) - \frac{1}{n}\sum_{i=1}^n \hat{u}(\bm{x}^{(d)}, D_i) = \frac{1}{n} \sum_{i=1}^n \tilde{V}_i \hat{u}(\bm{x}^{(d)}, D_i) . \end{flalign} and that the weights $\tilde{V}_i$ are independent and identically distributed Rademacher random variables. Thus, on the event $\mathcal{E}^\prime_n(t,t_0)$, we have \begin{align} & \vert P\left\{\sqrt{n} R_{n}^{*}(\bm{x}^{(d)})\in\mathsf{R}\mid\mathbf{D}_{n}\right\} - P\left\{ Z\in\mathsf{R}\right\} \vert\nonumber \\ & \quad\lesssim \left(\frac{\varphi^{2}\log^{5}(dn)}{\underline{\lambda}^2 n}\right)^{1/4} + t(1+t_0)\sqrt{\log (d)} \end{align} with probability greater than $1-Cn^{-1/2}\underline{\lambda}^{-1}\varphi \log^{3/2}(dn)$ by (ref). Hence, it suffice to give a high probability bound on $\mathcal{E}^\prime_n(t,t_0)$ for suitable choices of $t$ and $t_0$. To this end, observe that a multiplicative Chernoff bound implies that \begin{equation} \bigg\vert \frac{2Q_n}{n} - 1\bigg\vert \lesssim \sqrt{\frac{\log(n)}{n}} \end{equation} with probability greater than $1-n^{-1}$. Thus, as the data $D_\mathsf{s}$ are drawn independently and identically with distribution $P$ in $\mathbf{P}$ and we have assumed that $\delta_{\varepsilon n} \lesssim \delta_n$ and $\rho_{\varepsilon n} \lesssim \rho_n$ for any fixed $0<\varepsilon<1$, by setting $t = \delta_n$ and $t_0 = (\log(n)/n)^{1/2}$, (ref) and the bound (ref) imply that the event $\mathcal{E}^\prime_n(t,t_0)$ occurs with probability greater than $1-C(\rho_n + n^{-1})$, as required. \qed \subsection{Proof of (ref), Part (ii)} Again, we take $\theta_0(x)=0$ for all $x$, without loss of generality. Recall from the proof of (ref), Part (iii), that $\tilde{V}_{i}$ is a random variable taking the value $1$ when $i$ is an element of the subset $\mathsf{s}$, and taking the value $-1$ otherwise, and that \begin{flalign} \frac{2}{n}\sum_{i\in\mathsf{s}} \bar{u}(\bm{x}^{(d)}, D_i) - \frac{1}{n}\sum_{i=1}^n \bar{u}(\bm{x}^{(d)}, D_i) = \frac{1}{n} \sum_{i=1}^n \tilde{V}_i \bar{u}(\bm{x}^{(d)}, D_i) . \end{flalign} Define the object \begin{align*} T_\mathsf{s}(x)= \frac{1}{Q_n}\sum_{i\in\mathsf{h}} \bar{u}_n(x, D_i) - U_\mathsf{s}(x) . \end{align*} We are interested in studying \begin{align*} \hat{\lambda}^2_{n,j} & = n\mathbb{E}_{\tilde{V}}\left[\left(R^*_n(x^{(j)})\right)^2 \right]\\ & = n\mathbb{E}_{\tilde{V}}\left[\left(\frac{1}{n} \sum_{i=1}^n \tilde{V}_i \bar{u}(x^{(j)}, D_i) + \frac{2Q_n}{n} T_\mathsf{s}(x^{(j)}) - T_n(x^{(j)}) + \frac{2Q_n}{n} \Delta_\mathsf{s}(x^{(j)}) - \Delta_n(x^{(j)})\right)^2\right] , \end{align*} where the notation $\mathbb{E}_{\tilde{V}}\left[\cdot\right]$ denotes that the expectation is evaluated only over the random variables $\tilde{V}_1,\ldots,\tilde{V}_n$. On the event $\mathcal{E}_n^\prime(t_0, t)$, defined in the proof of (ref), Part (iii), we have that \begin{flalign*} & \sup_{j\in[d]} \big\vert \hat{\lambda}^2_{n,j} - \bar{\lambda}^2_{n,j} \big\vert \leq (t(1+ t_0))^2 , \quad\text{where}\quad \bar{\lambda}^2_{n,j} = n\mathbb{E}_V\left[\left(\frac{1}{n} \sum_{i=1}^n \tilde{V}_i \bar{u}(x^{(j)}, D_i)\right)^2\right] . \end{flalign*} We can evaluate \begin{flalign} \bar{\lambda}^2_{n,j} &= \frac{1}{n}\sum_{i=1}^n \bar{u}^2(x^{(j)}, D_i).\nonumber \end{flalign} as the weights $\tilde{V}_i$ are mutually independent. Now, observe that \begin{equation*} \sup_{j\in[d]} \Big\vert \frac{1}{n}\sum_{i=1}^n \bar{u}^2(x^{(j)}, D_i) - \lambda^2_{j} \Big\vert \lesssim \frac{\varphi}{n}\log(dn) \end{equation*} with probability greater than $1-n^{-1}$ by Bernstein's inequality. Thus, on the event $\mathcal{E}_n^\prime(t_0, t)$, we find that \begin{flalign*} & \sup_{j\in[d]} \big\vert \hat{\lambda}^2_{n,j} - \lambda^2_{n,j} \big\vert \lesssim \frac{\varphi}{n}\log(dn) + (t(1+ t_0))^2 , \end{flalign*} with probability greater than $1-n^{-1}$. By setting \[ t=C \sqrt{\frac{\underline{\lambda}^2}{n}} \delta_n~, \] and $t_0 = (\log(n)/n)^{1/2}$, (ref) and the bound (ref) imply that the event $\mathcal{E}^\prime_n(t,t_0)$ occurs with probability greater than $1-C\rho_n-n^{-1}$. Consequently, we find that \begin{flalign*} & \sup_{j\in[d]} \big\vert \hat{\lambda}^2_{n,j} - \lambda^2_{n,j} \big\vert \lesssim \frac{\varphi}{n}\log(dn) + \frac{\underline{\lambda}^2}{n}\delta_n^2 , \end{flalign*} with probability greater than $1-C(\rho_n+n^{-1})$, as required.$\blacksquare$. \subsection{Proof of (ref)} Throughout, we let $x$ denote an arbitrary element of $\bm{x}^{(d)}$. We are interested in giving a high probability bound for the quantity \begin{flalign} \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}F(\bm{x}^{(d)};D_{\mathsf{s}},\hat{g}_{n}) & =\frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}\sum_{i\in\mathsf{s}}a(\bm{x}^{(d)};D_{i},D_{\mathsf{s}},g)-a(\bm{x}^{(d)};D_{i},D_{\mathsf{s}},g_{0})\nonumber \\ & \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-\left(A(\bm{x}^{(d)};g)-\mathbb{E}\left[A(\bm{x}^{(d)};g_{0})\right]\right). \end{flalign} Building on an argument used in chen2022debiased, we decompose ((ref)) into three terms that will be easier to handle in isolation. In particular, by (ref), the quantity ((ref)) can be re-expressed as \begin{flalign} \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}F(\bm{x}^{(d)};D_{\mathsf{s}},\hat{g}_{n}) & =\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{1}(\bm{x}^{(d)};\mathsf{s}_{\pi,l})\\ & -\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{2}(\bm{x}^{(d)};\mathsf{s}_{\pi,l})\\ & +\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{3}(\bm{x}^{(d)};\mathsf{s}_{\pi,l}) \end{flalign} where \begin{flalign*} F_{1}(x;\mathsf{s}) & =\sum_{i\in\mathsf{s}}a(x;D_{i},D_{\mathsf{s}},\hat{g}_{n})-a(x;D_{i},D_{\mathsf{s}},\hat{g}_{n}^{(-\mathsf{s})})\\ F_{2}(x;\mathsf{s}) & =\sum_{i\in\mathsf{s}}A(x;\hat{g}_{n})-A(x;\hat{g}_{n}^{(-\mathsf{s})}),\quad\text{and}\\ F_{3}(x;\mathsf{s}) & =\sum_{i\in\mathsf{s}}a(\bm{x}^{(d)};D_{i},D_{\mathsf{s}},\hat{g}_{n}^{(-\mathsf{s})})-a(\bm{x}^{(d)};D_{i},D_{\mathsf{s}},g_{0})\\ & \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-(A(\bm{x}^{(d)};\hat{g}_{n}^{(-\mathsf{s})})-A(\bm{x}^{(d)};g_{0})), \end{flalign*} respectively. We give suitable probability bounds for the terms in ((ref)) through ((ref)). We begin by considering the term ((ref)). Fix any integer $q\geq1$. It will suffice to show that \begin{equation} \Big\|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{1}(x;\mathsf{s}_{\pi,l})\Big\|_{2q}\lesssim2qn^{-1}b\gamma(2q)b^{1/2q}. \end{equation} In particular, if ((ref)) holds, then we have that \begin{flalign} & P\left\{ \bigg\vert\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{1}(x;\mathsf{s}_{\pi,l})\bigg\vert>\frac{2}{\sqrt{e}}qn^{-1}b\gamma(2q)b^{1/2q}\right\} \nonumber \\ & \leq\frac{1}{\exp(q)}\frac{\mathbb{E}\left[\vert\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{1}(x;\mathsf{s}_{\pi,l})\vert^{2q}\right]}{\left(2qn^{-1}b\gamma(2q)b^{1/2q}\right)^{2q}}\lesssim\exp(-q), \end{flalign} by Markov's inequality. Thus, taking $q=\lceil\log(dn)\rceil$, by combining the inequality ((ref)) and a union bound, we can conclude that \begin{equation} \bigg\|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{1}(\bm{x}^{(d)};\mathsf{s}_{\pi,l})\bigg\|_{\infty}\lesssim\gamma_{n,b}^{\star}\frac{b\log(dn)}{n} \end{equation} with probability greater than $1-Cn^{-1}$, where we recall that $\gamma_{n,b}^{\star}=\gamma(2\lceil\log(dn)\rceil)b^{1/\log(dn)}.$ To this end, observe that \begin{flalign} \|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{1}(x;\mathsf{s}_{\pi,l})\|_{2q} & \leq\|\sum_{i=1}^{b}\kappa(x,X_{i};D_{[b]})(m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])}))\|_{2q}\nonumber \\ & \leq\|\max_{i\in[b]}(m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])}))^{2}\|_{q}^{1/2}, \end{flalign} where the first inequality follows from Jensen's inequality and the second inequality follows from (ref), Part (ii). Lemma 2.2.2 of van1996weak, implies that, if $A_{1},\ldots,A_{n}$ are any collection of real-valued random variables, then \begin{equation} \|\max_{i\in[n]}A_{i}\|_{q}\lesssim n^{1/q}\max_{i\in[n]}\|A_{i}\|_{q}. \end{equation} Thus, we have that \begin{alignat*}{1} \|\max_{i\in[b]}(m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])}))^{2}\|_{q}^{1/2} & \leq b^{1/(2q)}\|m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\|_{2q}.\\ & \lesssim2qn^{-1}b\gamma(2q)b^{1/2q} \end{alignat*} where the second inequality follows from Assumption (ref). Hence, the condition ((ref)) holds, implying the high probability bound ((ref)). We now turn to the term ((ref)). Again, it will suffice to show that \begin{equation} \Big\|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{2}(x;\mathsf{s}_{\pi,l})\Big\|_{2q}\lesssim2qn^{-1}b\gamma(2q)b^{1/2q} \end{equation} as this will imply that \begin{equation} \|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{2}(\bm{x}^{(d)};\mathsf{s}_{\pi,l})\|_{\infty}\lesssim\gamma_{n,b}^{\star}\frac{b\log(dn)}{n} \end{equation} by the same steps used to show that ((ref)) implies ((ref)). To this end, let $\mathbf{D}_{n}^{\prime\prime}=(D_{i}^{\prime\prime})_{i=1}^{n}$ be another independent copy of $\mathbf{D}_{n}$. Observe that \begin{flalign} \|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{2}(x;\mathsf{s}_{\pi,l})\|_{2q} & \leq\|\sum_{i=1}^{b}A(x;\hat{g}_{n})-A(x;\hat{g}_{n}^{(-\mathsf{s})})\|_{2q}\nonumber \\ & \leq\|\sum_{i=1}^{b}\kappa(x,X_{i}^{\prime};D_{[b]}^{\prime})(m(D_{i}^{\prime},\hat{g}_{n})-m(D_{i}^{\prime},\hat{g}_{n}^{(-[b])}))\|_{2q}\nonumber \\ & \leq\|\max_{i\in[b]}(m(D_{i}^{\prime},\hat{g}_{n})-m(D_{i}^{\prime},\hat{g}_{n}^{(-[b])}))^{2}\|_{q}^{1/2},\nonumber \\ & \leq b^{1/(2q)}\|m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\|_{2q}.\nonumber \\ & \lesssim2qn^{-1}b\gamma(2q)b^{1/2q}, \end{flalign} where the first two inequalities follow from Jensen's inequality, the third inequality follows from (ref), Part (ii), the fourth inequality follows from ((ref)), and the final inequality follows from (ref). Hence, the condition ((ref)) holds, implying the high probability bound ((ref)). Finally, we consider the term (ref). The argument here is somewhat more involved. To simplify exposition, let $k=\lfloor n/b\rfloor$ and define the sets $\mathsf{s}_{l}=\left\{ (l-1)b+1,\ldots,bl\right\} $ for each $l$ in $1,\ldots,k$. It will suffice to show that \begin{equation} \|\frac{1}{k}\sum_{l=1}^{k}F_{3}(x;\mathsf{s}_{l})\|_{q}\lesssim\sqrt{\frac{1}{k}}\left(\left(\frac{1}{k}\right)^{1/4}\delta_{n,g}+\varepsilon_{b}^{1/2}+b^{1/q}\gamma(q)\right)q \end{equation} as Jensen's inequality implies that \begin{equation} \|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{3}(x;\mathsf{s}_{\pi,l})\|_{q}\leq\|\frac{1}{k}\sum_{l=1}^{k}F_{3}(x;\mathsf{s}_{l})\|_{q} \end{equation} and the conditions ((ref)) and ((ref)) imply that \begin{equation} \|\frac{1}{n!}\sum_{\pi\in\mathcal{P}_{n}}\bigg\lfloor\frac{n}{b}\bigg\rfloor^{-1}\sum_{l=1}^{\lfloor n/b\rfloor}F_{3}(\bm{x}^{(d)};\mathsf{s}_{\pi,l})\|_{\infty}\lesssim\sqrt{\frac{b}{n}}\left(\left(\frac{b}{n}\right)^{1/4}\delta_{n,g}+\varepsilon_{b}^{1/2}+\gamma_{n,b}^{\star}\right)\log(dn) \end{equation} with probability greater than $1-Cn^{-1}$, by the same steps used to show that ((ref)) implies ((ref)). To establish the bound ((ref)), we apply the following generalized Efron-Stein inequality, due to boucheron2005moment. See also Chapter 15.2 of boucheron2013concentration. \begin{lemma}[{boucheron2005moment}] Let $X=(X_{i})_{i=1}^{n}$ be a sequence of independent random variables and let $X^{\prime}=(X_{i}^{\prime})_{i=1}^{n}$ denote an independent copy of $X$. Let $X^{(-i)}$ be constructed by taking $X$ and replacing $X_{i}$ with $X_{i}^{\prime}$. Consider the random variable $f(X)$, where $f(\cdot)$ is any real-valued function. It holds that \begin{flalign*} \|f(X)-\mathbb{E}\left[f(X)\right]\|_{q} & \lesssim\sqrt{q}\|\sum_{i=1}^{n}(f(X)-f(X^{(-i)}))^{2}\|_{q/2}^{1/2} \end{flalign*} for any $q\geq2$. \end{lemma} To apply Theorem (ref), let $\hat{g}_{n}^{(-\mathsf{s}_{l},-\mathsf{s}_{r})}$ denote a version of the estimator $\hat{g}_{n}$ formed with all observations in $\mathbf{D}_{n}$, except that the observations $D_{\mathsf{s}_{l}}$ and $D_{\mathsf{s}_{r}}$ are replaced by the observations $D_{\mathsf{s}_{l}}^{\prime}$ and $D_{\mathsf{s}_{q}}^{\prime\prime}$, respectively. Additionally, let $\mathbf{D}_{n}^{\prime\prime\prime}$ be an additional, independent, copy of $\mathbf{D}_{n}$ and let $\tilde{g}_{n}^{(-\mathsf{s}_{l})}$ denote a version of the estimator $\hat{g}_{n}$ formed with all observations in $\mathbf{D}_{n}$, except that the observations $D_{\mathsf{s}_{l}}$ are replaced by the observations $D_{\mathsf{s}_{l}}^{\prime\prime\prime}$. Let the quantity $F_{3}^{(-r)}(x;\mathsf{s}_{l})$ be given by \begin{flalign*} F_{3}^{(-r)}(x;\mathsf{s}_{l}) & =\sum_{i\in\mathsf{s}_{l}}a(x;D_{i},D_{\mathsf{s}_{l}},\hat{g}_{n}^{(-\mathsf{s}_{l},-\mathsf{\mathsf{s}}_{r})})-a(x;D_{i},D_{\mathsf{s}_{l}},g_{0})\\ & \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-(A(x;\hat{g}_{n}^{(-\mathsf{s}_{l},-\mathsf{\mathsf{s}}_{r})})-A(x;g_{0})) \end{flalign*} if $r\neq l$ and by \begin{flalign*} F_{3}^{(-l)}(x;\mathsf{s}_{l}) & =\sum_{i\in\mathsf{s}_{l}}a(x;D_{i}^{\prime\prime},D_{\mathsf{s}_{l}}^{\prime\prime},\tilde{g}_{n}^{(-\mathsf{s}_{l})})-a(x;D_{i}^{\prime\prime},D_{\mathsf{s}_{l}}^{\prime\prime},g_{0})\\ & \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-(A(x;\tilde{g}_{n}^{(-\mathsf{s}_{l})})-A(x;g_{0})) \end{flalign*} otherwise. Theorem (ref) implies that \begin{flalign} & \|\frac{1}{k}\sum_{l=1}^{k}F_{3}(x;D_{\mathsf{s}_{l}})\|_{q}\nonumber \\ & \leq\sqrt{q}\bigg\|\sum_{l=1}^{k}\left(\frac{1}{k}\left(F_{3}(x;\mathsf{s}_{l})-F_{3}^{(-l)}(x;\mathsf{s}_{l})\right)+\frac{1}{k}\sum_{r\neq l}\left(F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\right)\right)^{2}\bigg\|_{q/2}^{1/2} . \end{flalign} We obtain a suitable bound for ((ref)) through the application of the following Lemma. \begin{lemma} Suppose that the kernel $\kappa(x,X_{i};D_{\mathsf{s}})$ satisfies (ref) and has shrinkage rate $\varepsilon_{b}$ and that the moment function $m(D_{i},g)$ satisfies (ref). Moreover, suppose that the nuisance parameter estimator satisfies the bound (ref) for some sequence $\delta_{n,g}$. If (ref) holds, then \begin{flalign} & \bigg\|\sum_{l=1}^{k}\left(\frac{1}{k}\left(F_{3}(x^{(j)};\mathsf{s}_{l})-F_{3}^{(-l)}(x^{(j)};\mathsf{s}_{l})\right)+\frac{1}{k}\sum_{r\neq l}\left(F_{3}(x^{(j)};\mathsf{s}_{r})-F_{3}^{(-l)}(x^{(j)};\mathsf{s}_{r})\right)\right)^{2}\bigg\|_{q/2}^{1/2}\nonumber \\ & \lesssim\sqrt{\frac{1}{k}}\left(\left(\frac{1}{k}\right)^{1/4}\delta_{n,g}+\varepsilon_{b}^{1/2}+b^{1/q}\gamma(q)\right)q \end{flalign} for each $j$ in $[d]$ and each even integer $q$. \end{lemma} Consequently, the bound ((ref)) and Lemma (ref) imply ((ref)), and thereby ((ref)). Putting the pieces together, by the decomposition ((ref)), the bounds ((ref)), ((ref)), and ((ref)) imply that \begin{align} \frac{1}{N_{b}}\sum_{\mathsf{s}\in\mathcal{S}_{n,b}}F(\bm{x}^{(d)};D_{\mathsf{s}},\hat{g}_{n})& \lesssim \sqrt{\frac{b}{n}}\left(\left(\frac{b}{n}\right)^{1/4}\delta_{n,g}+\varepsilon_{b}^{1/2}+\gamma_{n,b}^{\star}\right)\log(dn)+\frac{b}{n}\gamma_{n,b}^{\star}\log(dn) \nonumber \\ & \lesssim\sqrt{\frac{b}{n}}\left(\left(\frac{b}{n}\right)^{1/4}\delta_{n,g}+\gamma_{n,b}^{\star}\right)\log(dn)+ \varepsilon_{b} + \frac{b}{n}\gamma_{n,b}^{\star}\log^2(dn) \end{align} with probability greater than $1-Cn^{-1}$, as required.$\blacksquare$ \subsection{Proof of Lemma (ref)} Throughout, we let $x$ denote an arbitrary element of $\bm{x}^{(d)}$. Observe that \begin{flalign} & \bigg\|\sum_{l=1}^{k}\left(\frac{1}{k}\left(F_{3}(x;\mathsf{s}_{l})-F_{3}^{(-l)}(x^{(j)};\mathsf{s}_{l})\right)+\frac{1}{k}\sum_{r\neq l}\left(F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\right)\right)^{2}\bigg\|_{q/2}\\ & \leq\sum_{l=1}^{k}\bigg\|\left(\frac{1}{k}\left(F_{3}(x;\mathsf{s}_{l})-F_{3}^{(-l)}(x^{(j)};\mathsf{s}_{l})\right)+\frac{1}{k}\sum_{r\neq l}\left(F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\right)\right)^{2}\bigg\|_{q/2}\nonumber \\ & \lesssim\bigg\|\sum_{l=1}^{k}\frac{1}{k^{2}}\left(F_{3}(x;\mathsf{s}_{l})-F_{3}^{(-l)}(x^{(j)};\mathsf{s}_{l})\right)^{2}+\frac{1}{k}\sum_{l=1}^{k}\sum_{r\neq l}\left(F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\right)^{2}\|_{q/2}\nonumber \\ & \lesssim\frac{1}{k}\|F_{3}(x;\mathsf{s}_{l})\|_{q}^{2}\\ & +k\|F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\|_{q}^{2}, \end{flalign} where the first inequality follows from the triangle inequality, the second inequality follows from Cauchy-Schwarz and Jensen's inequality, and the final inequality follows from the triangle inequality. Consider the term ((ref)). Observe that \begin{flalign} & \mathbb{E}\left[\vert F_{3}(x;D_{\mathsf{s}_{1}})\vert^{q}\right]\nonumber \\ & =\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(x;D_{i},D_{\mathsf{s}},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(x;D_{i},D_{\mathsf{s}},g_{0})-(A(x;\hat{g}_{n}^{(-\mathsf{s}_{1})})-A(x;g_{0}))\bigg\vert^{q}\right]\nonumber \\ & \leq2^{q-1}\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(x;D_{i},D_{\mathsf{s}},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(x;D_{i},D_{\mathsf{s}},g_{0})\bigg\vert^{q}\right]\\ & +2^{q-1}\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}A(x;\hat{g}_{n}^{(-\mathsf{s}_{1})})-A(x;g_{0})\bigg\vert^{q}\right] \end{flalign} by the Binomial Theorem and Hölder's inequality. We begin by bounding the term ((ref)). Observe that \begin{flalign} & \mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(\bm{x}^{(d)};D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(\bm{x}^{(d)};D_{i},D_{\mathsf{s}_{1}},g_{0}))\bigg\vert^{q}\right]\nonumber \\ & =\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}\kappa(x,X_{i};D_{\mathsf{s}_{1}})(m(D_{i},\hat{g}_{n}^{(-\mathsf{s}_{1})})-m(D_{i},g_{0}))\bigg\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\sum_{i\in\mathsf{s}_{1}}\kappa(x,X_{i};D_{\mathsf{s}_{1}})\vert m(D_{i},\hat{g}_{n}^{(-\mathsf{s}_{1})})-m(D_{i},g_{0}))\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\mathbb{E}\left[\sum_{i\in\mathsf{s}_{1}}\kappa(x,X_{i};D_{\mathsf{s}_{1}})\mathbb{E}\left[\vert m(D_{i},\hat{g}_{n}^{(-\mathsf{s}_{1})})-m(D_{i},g_{0}))\vert^{q}\mid X_{i},\hat{g}_{n}^{(-\mathsf{s}_{1})}\right]\mid\hat{g}_{n}^{(-\mathsf{s}_{1})}\right]\right]\nonumber \\ & \leq\mathbb{E}\left[\mathbb{E}\left[\sum_{i\in\mathsf{s}_{1}}\kappa(x,X_{i};D_{\mathsf{s}_{1}})\mathbb{E}\left[\vert m(D_{i},\hat{g}_{n}^{(-\mathsf{s}_{1})})-m(D_{i},g_{0}))\vert^{q}\mid X_{i}=x\right]\mid\hat{g}_{n}^{(-\mathsf{s}_{1})}\right]\right]+\varepsilon_{b}^{q/2}\nonumber \\ & \lesssim\mathbb{E}\left[\|\hat{g}_{n}^{(-\mathsf{s}_{1})}-g_{0}\|_{q,\infty}^{q}\right]+\varepsilon_{b}^{q/2}, \end{flalign} where the first inequality follows from Jensen's inequality, the second inequality follows from Honesty, i.e., (ref), Part (i), the third inequality follows from the definition of the kernel shrinkage $\varepsilon_{b}$ and (ref), and the fourth inequality follows from (ref) and (ref), Part (ii). To handle the second, term ((ref)), let $\mathbf{D}_{n}^{\prime\prime}=(D_{i}^{\prime\prime})_{i=1}^{n}$ be another independent copy of $\mathbf{D}_{n}$. We have that \begin{flalign} & \mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}A(x;\hat{g}_{n}^{(-\mathsf{s}_{1})})-A(x;\hat{g}_{0})\bigg\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(\bm{x}^{(d)};D_{i}^{\prime\prime},D_{\mathsf{s}}^{\prime\prime},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(\bm{x}^{(d)};D_{i}^{\prime\prime},D_{\mathsf{s}}^{\prime\prime},g_{0})\bigg\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\|\hat{g}_{n}^{(-\mathsf{s}_{1})}-g_{0}\|_{q,\infty}^{q}\right]+\varepsilon_{b}^{q/2}, \end{flalign} where the first inequality follows by Jensen's inequality and the second inequality follows by the inequality ((ref)). Hence, the bounds ((ref)) and ((ref)) imply that \begin{flalign} \|F_{3}(x;D_{\mathsf{s}_{q}})\|_{q}^{2} & =\mathbb{E}\left[\vert F_{3}(x;D_{\mathsf{s}_{1}})\vert^{q}\right]^{2/q},\nonumber \\ & \lesssim\left(q^{q}\left(\frac{b}{n}\right)^{\frac{q}{4}}\delta_{n,g}^{q}+\varepsilon_{b}^{q/2}\right)^{2/q}\nonumber \\ & \lesssim q^{2}\left(\left(\frac{b}{n}\right)^{1/2}\delta_{n,g}^{2}+\varepsilon_{b}\right) \end{flalign} by the rate condition ((ref)), where we have used the fact that $\hat{g}_{n}^{(-\mathsf{s}_{1})}$ and $\hat{g}_{n}$ are identically distributed. Next, consider the term ((ref)). Observe that \begin{flalign} & \mathbb{E}\left[\bigg\vert F_{3}(x;D_{\mathsf{s}_{l}})-F_{3,-q}(x;D_{\mathsf{s}_{l}})\bigg\vert^{q}\right]\nonumber \\ & =\mathbb{E\Bigg[}\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(x;D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(x;D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\nonumber \\ & \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-(A(x;\hat{g}_{n}^{(-\mathsf{s}_{2})})-A(x;\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})}))\bigg\vert^{q}\Bigg]\nonumber \\ & \leq2^{q-1}\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(x;D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(x;D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\bigg\vert^{q}\right]\\ & +2^{q-1}\mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}A(x;\hat{g}_{n}^{(-\mathsf{s}_{2})})-A(x;\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\bigg\vert^{q}\right] \end{flalign} by the Binomial Theorem and Hölder's inequality. To bound the quantity ((ref)), observe that \begin{flalign} & \mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}a(x;D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1})})-a(x;D_{i},D_{\mathsf{s}_{1}},\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\bigg\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\sum_{i\in\mathsf{s}_{1}}\kappa(x,X_{i};D_{\mathsf{s}_{1}})\vert m(D_{i},\hat{g}_{n}^{(-\mathsf{s}_{1})})-m(D_{i},\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\max_{i\in[b]}\vert m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\vert^{q}\right].\nonumber \\ & \leq b\mathbb{E}\left[\vert m(D_{i},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\vert^{q}\right]\lesssim b\left(qn^{-1}b\gamma(q)\right)^{q}, \end{flalign} where the first inequality follows Jensen's inequality, the second inequality follows from (ref), Part (ii), the third inequality follows from ((ref)), and the final inequality follows from Assumption (ref). Similarly, to bound the quantity ((ref)), we have that \begin{flalign} & \mathbb{E}\left[\bigg\vert\sum_{i\in\mathsf{s}_{1}}A(x;\hat{g}_{n}^{(-\mathsf{s}_{2})})-A(x;\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\bigg\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\sum_{i\in\mathsf{s}_{1}}\kappa(x,X_{i};D_{\mathsf{s}_{1}}^{\prime\prime})\vert m(D_{i}^{\prime\prime},\hat{g}_{n}^{(-\mathsf{s}_{1})})-m(D_{i}^{\prime\prime},\hat{g}_{n}^{(-\mathsf{s}_{1},-\mathsf{s}_{2})})\vert^{q}\right]\nonumber \\ & \leq\mathbb{E}\left[\max_{i\in[b]}\vert m(D_{i}^{\prime\prime},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\vert^{q}\right].\nonumber \\ & \lesssim b\mathbb{E}\left[\vert m(D_{i}^{\prime\prime},\hat{g}_{n})-m(D_{i},\hat{g}_{n}^{(-[b])})\vert^{q}\right]\leq b\left(qn^{-1}b\gamma(q)\right)^{q}, \end{flalign} where the first inequality follows from Jensen's inequality and the other inequalities follow from the same steps used to obtain the bound ((ref)). Consequently, we have that \begin{flalign} \|F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\|_{q}^{2} & =\mathbb{E}\left[\left(F_{3}(x^{(j)};D_{\mathsf{s}_{l}})-F_{3,-q}(x;D_{\mathsf{s}_{l}})\right)^{q}\right]^{2/q}\nonumber \\ & \lesssim b^{2/q}\left(qn^{-1}b\gamma(q)\right)^{2}, \end{flalign} as desired. Putting the pieces together, we find that \begin{flalign*} & \bigg\|\sum_{l=1}^{k}\left(\frac{1}{k}\left(F_{3}(x;\mathsf{s}_{l})-F_{3}^{(-l)}(x^{(j)};\mathsf{s}_{l})\right)+\frac{1}{k}\sum_{r\neq l}\left(F_{3}(x;\mathsf{s}_{r})-F_{3}^{(-l)}(x;\mathsf{s}_{r})\right)\right)^{2}\bigg\|_{q/2}^{1/2}\\ & \lesssim\sqrt{\frac{1}{k}}\left(\left(\frac{1}{k}\right)^{1/4}\delta_{n,g}+\varepsilon_{b}^{1/2}+b^{1/q}\gamma(q)\right)q \end{flalign*} by applying the bounds ((ref)) and ((ref)) to the decomposition ((ref)), as required.$\blacksquare$ \subsection{Proof of (ref)} Consider the decomposition \begin{flalign*} & m(D_{i},\hat{g}_{n}(Z_{i}))-m(D_{i},\hat{g}_{n}^{(-[b])}(Z_{i}))\\ & =\left(\hat{\mu}_{n}(Z_{i},1)-\hat{\mu}_{n}^{(-[b])}(Z_{i},1)\right)-\left(\hat{\mu}_{n}(Z_{i},0)-\hat{\mu}_{n}^{(-[b])}(Z_{i},0))\right)\\ & +\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i})(\hat{\mu}_{n}(Z_{i},W_{i})-\hat{\mu}_{n}^{(-[b])}(Z_{i},W_{i}))-\left(Y_{i}-\hat{\mu}_{n}(Z_{i},W_{i})\right)\left(\hat{\beta}_{n}(W_{i},Z_{i})-\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i})\right). \end{flalign*} We have that \begin{flalign*} & \|m(D_{i},\hat{g}_{n}(Z_{i}))-m(D_{i},\hat{g}_{n}^{(-[b])}(Z_{i}))\|_{2p}\\ & \leq\|\hat{\mu}_{n}(Z_{i},1)-\hat{\mu}_{n}^{(-[b])}(Z_{i},1)\|_{2p}+\|\hat{\mu}_{n}(Z_{i},0)-\hat{\mu}_{n}^{(-[b])}(Z_{i},0))\|_{2p}\\ & +\|\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i})(\hat{\mu}_{n}(Z_{i},W_{i})-\hat{\mu}_{n}^{(-[b])}(Z_{i},W_{i}))\|_{2p}\\ &+\|\left(Y_{i}-\hat{\mu}_{n}(Z_{i},W_{i})\right)\left(\hat{\beta}_{n}(W_{i},Z_{i})-\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i})\right)\|_{2p}\\ & \lesssim\|\hat{\mu}_{n}(Z_{i},1)-\hat{\mu}_{n}^{(-[b])}(Z_{i},1)\|_{2p}+\|\hat{\mu}_{n}(Z_{i},0)-\hat{\mu}_{n}^{(-[b])}(Z_{i},0))\|_{2p}\\ &+\|\hat{\mu}_{n}(Z_{i},W_{i})-\hat{\mu}_{n}^{(-[b])}(Z_{i},W_{i})\|_{2p} +\|\hat{\beta}_{n}(W_{i},Z_{i})-\hat{\beta}_{n}^{(-[b])}(W_{i},Z_{i})\|_{2p} \end{flalign*} where the first inequality follows from the triangle inequality and the second inequality follows from the condition (ref). This verifies the condition ((ref)). An identical argument verifies the condition ((ref)).$\blacksquare$ \subsection{Proof of (ref)} To ease notation, we omit dependence on the evaluation point $z$. Recall the definition of the Hájek projection \[ u^{(1)}(D)=\mathbb{E}\left[u(D_{[b]})\mid D_{1}=D\right] \] where $D$ is an independent copy of $D_{1}$. Let \begin{flalign*} \nu^{2} & =\Var(u(D_{[b]}))\quad\text{and}\quad\sigma_{b^{\prime}}^{2}=\Var(u^{(1)}(D)) \end{flalign*} denote the kernel variance and Hájek projection variance, respectively. Consider the decomposition \begin{flalign} \hat{g}_{n}(\mathbf{D}_{n})-\hat{g}_{n}(\mathbf{D}_{n}^{(-[b])}) & =\frac{b^{\prime}}{n}\sum_{i=1}^{b}\left(u^{(1)}(D_{i})-u^{(1)}(D_{i}^{\prime})\right)\\ & +\left(\hat{g}_{n}(\mathbf{D}_{n})-\frac{b^{\prime}}{n}\sum_{i=1}^{n}u^{(1)}(D_{i})\right)\\ & +\left(\frac{b^{\prime}}{n}\sum_{i=1}^{b}u^{(1)}(D_{i}^{\prime})+\sum_{i=b+1}^{n}u^{(1)}(D_{i}^{\prime})-\hat{g}_{n}(\mathbf{D}_{n}^{(-[b])})\right). \end{flalign} The result is obtained by giving higher-order moment bounds for each of the terms ((ref)), ((ref)), and ((ref)). We begin by considering the terms ((ref)), and ((ref)). Observe that the bound (C.19) gives \begin{flalign} \|\hat{g}_{n}(\mathbf{D}_{n})-\frac{b^{\prime}}{n}\sum_{i=1}^{n}u^{(1)}(D_{i})\|_{q} & \leq\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\left(1 -q^{3/2}\frac{b^{\prime3/2}}{n^{1/2}}\right)\phi\log(n)\\ & \leq q^{3/2}\phi^{2}n\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\log(n)\nonumber \end{flalign} and \begin{flalign} \|\frac{b^{\prime}}{n}\sum_{i=1}^{b}u^{(1)}(D_{i}^{\prime})+\sum_{i=b+1}^{n}u^{(1)}(D_{i}^{\prime})-\hat{g}_{n}(\mathbf{D}_{n}^{(-[b])}))\|_{q} & \leq\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\left(1+q^{3/2}\frac{b^{\prime3/2}}{n^{1/2}}\right)\phi\log(n)\\ & \leq q^{3/2}\phi^{2}n\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\log(n)\nonumber \end{flalign} respectively. To handle the term ((ref)), we apply the following Rosenthal type inequality. \begin{lemma}[{boucheron2005moment}] Let $X_{1},\ldots,X_{b}$ be independent centered random variables. For any integer $q\geq2$, we have that \[ \|\frac{1}{n}\sum X_{i}\|_{1}\lesssim\sqrt{\frac{\Var(X_{i})q}{b}}+\frac{q}{b}\|\max_{i\in[b]}X_{i}\|_{q}. \] \end{lemma} In particular, Lemma (ref) implies that \begin{flalign} \|\frac{b^{\prime}}{n}\sum_{i=1}^{b}\left(u^{(1)}(D_{i})-u^{(1)}(D_{i}^{\prime})\right)\|_{q} & =\frac{b^{\prime}b}{n}\|\frac{1}{b}\sum_{i=1}^{b}\left(u^{(1)}(D_{i})-u^{(1)}(D_{i}^{\prime})\right)\|_{q}\nonumber \\ & \lesssim\frac{b^{\prime}b}{n}\left(\sqrt{\frac{\sigma_{b^{\prime}}^{2}q}{b}}+q\frac{\phi}{b}\right)\lesssim\frac{\sqrt{b^{\prime}b}}{n}q\phi, \end{flalign} where the second inequality follows from $\sigma_{b^{\prime}}^{2} \lesssim (b^\prime)^{-1}$, which was established in the proof of (ref). Putting the pieces together, we find that \[ \|\hat{g}_{n}(\mathbf{D}_{n})-\hat{g}_{n}(\mathbf{D}_{n}^{(-[b])})\|_{p}\lesssim\frac{\sqrt{b^{\prime}b}}{n}q\phi+q^{3/2}\phi^{2}n\left(C\frac{b^{\prime}q}{n}\right)^{b^{\prime}/2}\log(n) \] by combining the bounds ((ref)), ((ref)), and ((ref)), as required.$\blacksquare$ \subsection{Figures Introduced in (ref)} In this appendix, we display various figures introduced in (ref). (ref) displays binomial sample bootstrap upper and lower confidence bounds for the CATE (ref) on post-treatment assets. (ref) and (ref) gives versions of (ref) and (ref), constructed using the $2$-fold cross-split estimator (ref) and the $2$-fold cross-split bootstrap root (ref). (ref) displays results for the simulation presented in (ref), analogous to (ref), for the $2$-fold cross-split estimator (ref) and the $2$-fold cross-split bootstrap root (ref). (ref) displays the pseudo-true values of the CATE $\theta_0(x)$ used in the simulation. (ref) displays a scatterplot comparing the moments of the data from the banerjee2015multifaceted data to the data generated by our calibrated simulation. (ref) compares a scatter plot of the observed values of baseline consumption and baseline assets in the banerjee2015multifaceted data with a heat-map of the distribution of these covariates in our simulation. \begin{figure} \begin{centering} \caption{Binomial-Sample Lower Bound} \begin{tabular}{c} \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays heat maps giving binomial-sample lower confidence bound for the CATE of the intervention studied in banerjee2015multifaceted on post-treatment total assets. The axes and estimator are the same as in (ref).} \end{figure} \begin{figure}[t] \begin{centering} \caption{CATE Estimates, $2$-Fold Cross-Fitting} \begin{tabular}{c} \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays a heat map giving CATE estimates for the intervention studied in banerjee2015multifaceted on post-treatment assets, using the the $2$-fold cross-split estimator (ref). Other features of the display are analogous to (ref).} \end{figure} \begin{figure}[t] \begin{centering} \caption{Performance, $2$-Fold Cross-Fitting} \begin{tabular}{c} \end{tabular} \end{centering} \justifying {Notes: (ref) displays several measurements of the performance of the confidence intervals formulated in (ref), constructed using using the $2$-fold cross-split estimator (ref) and the $2$-fold cross-split bootstrap root (ref), in a simulation calibrated to the banerjee2015multifaceted data. Other features of the display are analogous to (ref).} \end{figure} \begin{figure}[t] \begin{centering} \caption{Half-Sample Confidence Region Lower Bound, $2$-Fold Cross-Fitting} \begin{tabular}{c} \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays heat maps giving half-sample lower confidence bound for the CATE of the intervention studied in banerjee2015multifaceted on post-treatment total assets. Other features of the display are analogous to (ref).} \end{figure} \begin{figure} \begin{centering} \caption{Calibrated CATEs} \begin{tabular}{c} \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays the “true” value of CATE used in our calibrated simulation.} \end{figure} \begin{figure} \caption{Validation} \begin{centering} \begin{tabular}{c} \tabularnewline \tabularnewline \tabularnewline \end{tabular} \end{centering} \justifying {Notes: (ref) displays scatterplots comparing the moments of the data from banerjee2015multifaceted to the GAN generated simulation data. Columns differentiate between different types of variables. Rows differentiate between different types of moments. The x-axis of each sub-panel measures the moments of the true data. The y-axis of each sub-panel measures the moments of the generated data. The x and y axes in the first two rows are displayed in log-scale. A forty-five degree line is displayed in all sub-panels. Blue and green dots denote moments conditioned on treatment being set to one and zero, respectively. Black dots denote unconditioned moments.} \end{figure} \begin{figure}[h] \begin{centering} \caption{Covariate Density} \begin{tabular}{c} \textit{Panel A: Observed Covariates}\tabularnewline \tabularnewline \textit{Panel B: Simulation Covariate Density}\tabularnewline \tabularnewline \end{tabular} \end{centering} \justifying {Notes: Panel A of (ref) displays a scatter plot of the observed values of baseline consumption and baseline assets in the banerjee2015multifaceted data. The horizontal and vertical axes display the baseline monthly consumption, normalized to 2014 dollars on a logarithmic scale base 10, and an index for baseline assets, respectively. Panel B displays a heat-map giving the density of the joint distribution of baseline consumption and baseline assets associated with our calibrated simulation.} \end{figure} \end{spacing}