The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
60,398 characters
Accuracy of Uniform Inference on Fine Grid Points
\baselineskip=18pt
\title{Accuracy of Uniform Inference on Fine Grid Points}
\author[1]{Shunsuke Imai \thanks{This research was supported by JSPS KAKENHI Grant Number 24KJ1472.
}}
\affil[1]{Graduate School of Economics, Kyoto University}
\maketitle
\begin{abstract}
Uniform confidence bands for functions are widely used in empirical analysis.
A variety of simple implementation methods (most notably multiplier bootstrap) have been proposed and theoretically justified.
However, an implementation over a literally continuous index set is generally computationally infeasible, and practitioners therefore compute the critical value by evaluating the statistic on a finite evaluation grid.
This paper quantifies how fine the evaluation grid must be for a multiplier bootstrap procedure over finite grid points to deliver valid uniform confidence bands.
We derive an explicit bound on the resulting coverage error that separates discretization effects from the intrinsic high-dimensional bootstrap approximation error on the grid.
The bound yields a transparent workflow for choosing the grid size in practice, and we illustrate the implementation through an example of kernel density estimation.
\vspace{2mm}
\noindent \textit{Keywords}:
bootstrap;
empirical process;
high-dimensional central limit theorem;
kernel smoothing;
nonparametric inference;
uniform confidence bands;
\end{abstract}
\newpage
\section{Introduction} \label{sec:introduction}
Uniform confidence bands are a useful device for quantifying uncertainty of estimates on an unknown function over a continuum of points.
A long line of work on uniform inference in nonparametric problems dates back at least to early contributions such as \citet{bickel1973some} and related papers.
Nowadays, by virtue of the seminal studies by \cite{chernozhukov2014anti,chernozhukov2014gaussian} (hereafter CCK) and their extensions like \cite{chernozhukov2016empirical,ChKa20}
, we can readily validate the uniform Gaussian approximation and the implementation of uniform critical value for a variety of possibly non-Donsker nonparametric estimators.
Specifically, CCK established approximation result between suprema of empirical processes and suprema of Gaussian processes under fairly general setting.
Therefore, using the quantile of supremum of approximating Gaussian process as the critical value results in the confidence band which attains asymptotically correct coverage.
More importantly for practical implementation, CCK also show that multiplier bootstrap-based critical values can be used as a feasible critical value and yields asymptotically correct coverage.
CCK theory has been widely used in applied econometrics and statistics to justify both analytic (Gaussian) critical values and bootstrap-based critical values for uniform inference, for example \cite{lee2017doubly,kato2018uniform,fan2022estimation,imai2025doubly} to name but a few.
However, in practice, neither supremum of Gaussian process nor supremum of bootstrap statistic over continuous interval is computationally feasible.
Accordingly, implementations replace supremum over continuous interval by maximum over a finite evaluation grid and compute critical values from the corresponding maximum of bootstrap statistics over finite grid points.
Then, the natural question is how fine the evaluation grid must be in order for the resulting procedure to remain theoretically valid while being computationally feasible.
In this study, we address the problem.
This paper provides a quantitative answer to this question.
More precisely, as \cref{thm:boot_disc_error}, we quantify how well the conditional $(1-\alpha)$-quantile of the grid maximum of the multiplier-bootstrap statistics (the critical value used in practice) approximates the infeasible $(1-\alpha)$-quantile of the supremum of the studentized statistic over continuous interval.
The key feature of \cref{thm:boot_disc_error} is that it separates the approximation error to two distinct sources of error.
One is a discretization error arising from replacing the continuum supremum of the studentized statistic by its maximum over the grid.
This discretization component has two elements.
First, the discretization error includes a term that captures the probability of the existence of sharp local spikes between grid points and the grid maximum can miss such peaks.
Second, even when no sharp between-grid spikes occur, a coarse grid can still miss the true peak simply because the between-grid space is too sparse.
The discretization error therefore includes an explicit mesh condition requiring the grid to be fine enough that this miss due to sparsity is negligible.
The other reflects how accurately the multiplier bootstrap approximates the distribution of the finite-dimensional vector of studentized statistics over fine grid points.
This term is controlled by Gaussian and multiplier-bootstrap approximation theory for high-dimensional maxima by \cite{CCKK22}.
This decomposition directly translates into a practical recipe for choosing the grid size; see \cref{rmk:implementation} for a detailed implementation workflow.
To illustrate how the bound in \cref{thm:boot_disc_error} can be used in practice, \cref{sec:KDE} works out a kernel density estimation example under standard regularity conditions.
The example translates the abstract ingredients of the bound into familiar quantities and yields a simple, implementable rule for choosing the grid size.
\paragraph{Organization:}
The remainder of the paper is organized as follows. \cref{sec:main} introduces the general framework and presents the main coverage error bound for grid-based multiplier-bootstrap uniform inference.
\cref{sec:KDE} develops and illustration with kernel density estimators.
\cref{sec:proof} contains the proofs.
\section{Main Results} \label{sec:main}
Suppose that we are interested in construction of uniform confidence band for some function $f$ over the compact region $\mathcal{X} \coloneqq[\underline{x}, \overline{x}] \subset \mathbb{R}$ and we have an estimator for $f$ with the sum of independent variables form $\hat{f}_{h_n}(x) \coloneqq n^{-1}\sum_{i=1}^n\psi_{h_n}(X_i,x) $ and
\begin{align}
\hat{f}_{h_n}(x) - \mathbb{E}[\hat f_{h_n}(x)] = \frac{1}{n}\sum_{i=1}^n \{\psi_{h_n}(X_i,x) - \mathbb{E}[\psi_{h_n}(X_i,x)]\}, \label{eq:linear}
\end{align}
where $\{X_i\}_{i=1}^n$ is a sample from i.i.d.~observations and $h_n$ denotes a tuning parameter possibly depending on $n$.
\begin{rmk}
To keep the paper concise and focused on the error caused by the discretization, we take $\mathbb{E}[\hat{f}_{h_n}(x)]$ as the baseline target.
However, in practice, the parameter of interest is typically $f(x)$ rather than the pseudo-true target $\mathbb{E}[\hat{f}_{h_n}(x)]$.
In that case, one may write
\begin{align*}
\hat{f}_{h_n}(x) - f(x) \coloneqq \frac{1}{n}\sum_{i=1}^n \{\psi_{h_n}(X_i,x) - \mathbb{E}[\psi_{h_n}(X_i,x)]\} + r_n(x),
\end{align*}
where $r_n(x)$ represents the bias term (or, more generally, a linearization remainder).
Nevertheless, the results below continue to hold provided $r_n(x)$ satisfies negligiblity condition uniformly over $\mathcal{X}$ relative to the stochastic term (cf. Lemma 1 in \citealp{chernozhukov2023high}).
\end{rmk}
Assume that $\sigma_n^2(x)\coloneqq n^{-1}\text{Var}[ \psi_{h_n}(X_i,x)]$ and let $\hat{\sigma}^2_n(x)$ be its estimator:
\begin{align}
\hat{\sigma}^2_n(x) \coloneqq \frac{1}{n} \left[ \frac{1}{n}\sum_{i=1}^n \psi_{h_n}^2(X_i,x) - \left( \frac{1}{n}\sum_{i=1}^n \psi_{h_n}(X_i,x)\right)^2 \right]. \label{eq:variance_estimator}
\end{align}
Define the standardized and studentized estimator as
\begin{align*}
T_n(x) \coloneqq \frac{\hat{f}_{h_n}(x) - \mathbb{E}[\hat f_{h_n}(x)]}{\sigma_n(x)}, \quad \hat{T}_n(x) \coloneqq \frac{\hat{f}_{h_n}(x) - \mathbb{E}[\hat f_{h_n}(x)]}{\hat\sigma_n(x)},
\end{align*}
respectively.
For each bootstrap iteration, we generate sets of i.i.d.~bootstrap weights $\{w_i^{\star}\}_{i=1}^n$ independently of the original data and compute
\begin{align*}
\hat T_n^{\star}(x) \coloneqq \frac{1}{n\hat\sigma_n(x)}\sum_{i=1}^n w_i^{\star} \{\psi_{h_n}(X_i,x) - \hat{f}_{h_n}(x)\}.
\end{align*}
For simplicity, we suppose that $\{w_i^{\star}\}_{i=1}^n$ follows the standard normal distribution.
Ideally, we would construct the critical value based on the conditional $(1-\alpha)$-quantile of the supremum of the bootstrap process over the continuum,
\begin{align*}
\hat{c}_{n,1-\alpha}^{\star, \texttt{infeasible}} \coloneqq \inf\left\{ t\in\mathbb{R} : \mathbb{P}^\star\left( \sup_{x\in\mathcal{X}}|\hat T_n^\star(x)| \le t \mid X_{1:n} \right) \ge 1-\alpha\right\},
\end{align*}
where $\mathbb{P}^\star(\cdot \mid X_{1:n})$ denotes probability with respect to the bootstrap weights conditional on the original sample.
However, evaluating $\sup_{x\in\mathcal{X}}|\hat T_n^\star(x)|$ is not computationally feasible in general.
Therefore, we approximate the continuum supremum by a maximum over a finite grid.
To proceed, we introduce some notation about discrete grids.
Let $\delta_n>0$ be a mesh size.
Set grid points $\{x_j\}_{j=1}^{p}\subset\mathcal X$ by
\[
x_j := \underline x + (j-1)\delta_n,\quad j=1,\dots,p-1,
~~~ x_{p} := \overline x, \text{ with } p := \left\lfloor \frac{\overline x-\underline x}{\delta_n}\right\rfloor+2.
\]
We write $\mathcal X_{\delta_n}:=\{x_1,\dots,x_{p}\}$.
Define the maximum gap as
\begin{align*}
\Delta_n := \max_{1\le j\le p-1}(x_{j+1}-x_j).
\end{align*}
Note that $x_1=\underline x$ and $x_{p}=\overline x$, and that the maximum gap satisfies $\Delta_n \le \delta_n$, since $x_{j+1}-x_j=\delta_n$ for $j\le p-2$ and $x_{p}-x_{p-1}\in(0,\delta_n]$.
In this study, we use the following feasible bootstrap critical value, which approximates the (infeasible) critical value based on $\sup_{x\in\mathcal{X}}|\hat T_n^\star(x)|$.
\begin{align*}
\hat{c}_{n,1-\alpha}^{\star} \coloneqq \inf\left\{ t\in\mathbb{R} : \mathbb{P}^\star\left( \max_{1\le j \le p}|\hat T_n^\star(x_j)| \le t \mid X_{1:n} \right) \ge 1-\alpha\right\},
\end{align*}
and investigate the quality of this approximation and provide conditions under which the resulting critical value delivers valid uniform inference.
To this end, we introduce the following assumptions.
\begin{assumption} \label{as:DGP_high-dim} \mbox{}
\begin{itemize}
\item[(i)] $\{X_i\}_{i=1}^n$ is a sample from i.i.d.~observations.
\item[(ii)] There exists some constant $B_n\ge 1$ such that $ \|Y_{i,h_n}(x_j)\|_{\psi_1} \le B_n$ and $ \mathbb{E}[Y_{i,h_n}^4(x_j)] \le B_n^2$ for all $1\le i\le n$ and $1\le j \le p$, with
\begin{align*}
Y_{i,h_n}(x) \coloneqq \frac{\psi_{h_n}(X_i,x) - \mathbb{E}[\psi_{h_n}(X_i,x)]}{\sqrt{\text{Var}[\psi_{h_n}(X_i,x)]}},
\end{align*}
and $\|\xi\|_{\psi_1} \coloneqq \inf \{C>0 : \mathbb{E}[\exp(|\xi|/C)] \le 2\}$.
\item[(iii)] The estimator $\hat{f}_{h_n}(x)$ has a linear form in \cref{eq:linear} uniformly over $\mathcal{X}_{\delta_n}$.
\item[(iv)] $\inf_{1\le j \le p}\text{Var}[\psi_{h_n}(X_i,x_j)] \ge c > 0$.
\item[(v)] There exists a deterministic sequence $L_n$ and $\varepsilon_n \downarrow 0$ such that
\begin{align*}
\mathbb{P}\left(\sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_n(x) - \hat{T}_n(y)|}{|x-y|} > L_n\right) \le \varepsilon_n.
\end{align*}
\end{itemize}
\end{assumption}
Two assumptions (ii) and (v) deserve further explanation.
\cref{as:DGP_high-dim}(ii) is imposed to control the high-dimensional Gaussian approximation and multiplier-bootstrap approximation errors using the results of \citet{CCKK22}.
See conditions E and M in \cite{CCKK22}.
\cref{as:DGP_high-dim}(v) is a high-level condition that controls the discretization error.
We discuss \cref{as:DGP_high-dim}(v) in detail later and provide primitive sufficient conditions for \cref{as:DGP_high-dim}(v) in the KDE example in \cref{subsec:KDE_theory}.
The following theorem provides an explicit bound on the approximation error that arises when the $(1-\alpha)$-quantile of the supremum statistic $\sup_{x\in\mathcal{X}} |\hat{T}_n(x)| $ is approximated by the feasible bootstrap critical value $\hat{c}^\star_{n,1-\alpha} $ computed from the grid-based bootstrap maximum $\max_{1\le j \le p}|\hat T_n^\star(x_j)|$.
\begin{theorem} \label{thm:boot_disc_error}
Under \cref{as:DGP_high-dim}, letting $r \coloneqq 2\left\{n^{-1}B_n^2 \log^3(np)\right\}^{1/4}$, it holds that
\begin{align*}
& \left|\mathbb{P}\left( \sup_{x\in\mathcal{X}} |\hat{T}_n(x)| \le \hat{c}^\star_{n,1-\alpha} \right) - (1-\alpha) \right| \\
& \quad \le 3\Big(\varepsilon_n + 1\{L_n\Delta_n/2 > r\} \Big) + O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
The proof of this theorem is provided in \cref{sec:proof_main}.
\end{theorem}
We first provide a detailed explanation on three terms in the right hand side of \cref{thm:boot_disc_error}, and then present a workflow for valid implementation (\cref{rmk:implementation}).
The first two terms on the right-hand side of \cref{thm:boot_disc_error} quantify the discretization error arising from replacing the infeasible continuum supremum by a maximum over the finite grid. The final term corresponds to the high-dimensional Gaussian approximation and multiplier bootstrap approximation error on the grid, whose rate follows from the results of \cite{CCKK22}.
The first term captures the probability that $\hat{T}_n(x)$ exhibits rapid local variation between nearby points, so that the grid maximum may miss a sharp peak occurring between grid points.
To see why this term appears, recall that the grid spacing is at most $\Delta_n$.
Hence, for any location $x\in\mathcal{X}$ there exists a grid point $x_j$ within distance of $\Delta_n$.
If $\hat{T}_n(x)$ does not change too much over such short distances, then the value of $\hat{T}_n(x)$ is well approximated by $\hat{T}_n(x_j)$ and consequently the continuum supremum $\sup_{x\in\mathcal{X}}|\hat{T}_n(x)|$ is well approximated by the grid maximum $\max_{1\le j\le p}|\hat{T}_n(x_j)|$.
More precisely, the quantity
\begin{align*}
\sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|}
\end{align*}
measures the largest change of $\hat{T}_n$ per unit change in $x$ over distances up to $\Delta_n$.
When this local slope is bounded by $L_n$ moving from any $x$ to a nearby grid point changes $\hat{T}_n$ by at most $L_n\Delta_n/2$.
Therefore, any peak that occurs between grid points cannot exceed the observed grid maximum by more than $L_n\Delta_n/2$; the first term is the probability that this favorable behavior fails.
The second term is a deterministic grid-design condition that complements the first term.
As explained above, when the local slope of $\hat{T}_n$ over distances up to $\Delta_n$ is bounded by $L_n$, moving from any location $x\in\mathcal{X}$ to a nearby grid point $x_j$ changes $\hat{T}_n$ by at most $L_n\Delta_n/2$.
Consequently, on this event, it holds that
\begin{align*}
\sup_{x\in\mathcal{X}}|\hat{T}_n(x)| \le \max_{1\le j\le p}|\hat{T}_n(x_j)| + L_n\Delta_n/2,
\end{align*}
so the discrepancy between the continuum supremum and the grid maximum is controlled by the worst-case between-grid fluctuation $L_n\Delta_n/2$.
The indicator $1\{L_n\Delta_n/2 > r\}$ checks whether this worst-case fluctuation is small enough relative to the tolerance level $r$, which stems from the high-dimensional Gaussian approximation error and an anti-concentration inequality used in the proof of high-dimensional Gaussian approximation and bootstrap approximation.
If $L_n\Delta_n/2 \le r$, then the discretization error contributed by replacing the continuum supremum with the grid maximum is guaranteed to be no larger than $r$ (up to the probability of the event captured by the first term).
In practice, this term provides a transparent guideline for choosing the grid: one should select the mesh fine enough so that $L_n\Delta_n/2$ is dominated by $r$.
The last term reflects how accurately the multiplier bootstrap based on the maximum over the grids approximates the distribution of the target statistic after reducing the continuum supremum to a finite-dimensional problem. The rate is inherited from the high-dimensional Gaussian approximation and multiplier bootstrap theory developed in \cite{CCKK22}.
In \cite[pp.~2375]{chernozhukov2023nearly}, they conjecture that the bound is ``near-optimal when the covariance is unrestricted".
A key feature of \cite{CCKK22} is that it does not require the covariance matrix of the high-dimensional vector $(\hat{T}_n(x_1), \dots, \hat{T}_n(x_p))$ to be uniformly invertible.
This robustness is particularly relevant in nonparametric applications, where the grid becomes dense, nearby evaluations of the estimator are typically strongly correlated, and the resulting finite-dimensional covariance matrix may approach singularity as $p$ increases.
For this reason, it is a reasonable choice to benchmark the bootstrap approximation error under the unrestricted covariance rate delivered by \cite{CCKK22}.
It should be desirable, for practical implementation, to characterize the constant in the Gaussian and multiplier bootstrap approximation error by \cite{CCKK22}.
Nevertheless, developing such a result appears technically demanding and we leave it for future work.
\begin{rmk}[Workflow for valid implementation]\label{rmk:implementation}
\cref{thm:boot_disc_error} provides a quantitative criterion for how fine the grid needs to be.
The following workflow summarizes how to choose the grid size in a transparent manner.
\begin{enumerate}
\item \textbf{Choose a tolerance level $\varepsilon_n$ for excursion probability of local variation:}
The quantity $\varepsilon_n$ bounds the probability of the event that the studentized statistic varies too abruptly over distances comparable to the grid gap, so that a narrow between-grid spike may be missed by the grid maximum.
If one only targets first-order validity, it suffices to take $\varepsilon_n=o(1)$.
If one aims to make discretization negligible relative to the intrinsic high-dimensional approximation error (the last term in \cref{thm:boot_disc_error}), a natural choice is to take $\varepsilon_n$ asymptotically smaller than that term.
\item \textbf{Determine the implied local variation bound $L_n$:}
Once $\varepsilon_n$ is determined, Assumption~\ref{as:DGP_high-dim}(v) specifies a corresponding bound $L_n$ on how rapidly the studentized statistic can change locally (over distances up to the grid gap).
In applications, $L_n$ can be obtained from properties of the estimator and the data generating process (DGP).
\item \textbf{Determine $B_n$ and the implied threshold level $r$:}
The quantity $B_n$ is determined by tail and moment properties of the normalized summands in Assumption~\ref{as:DGP_high-dim}(ii).
Once $B_n$ is determined, the threshold level $r = 2\{n^{-1}B_n^2\log^3(np)\}^{1/4}$ in \cref{thm:boot_disc_error} is determined, for a given number of grid points $p$.
\item \textbf{Choose the mesh size $\Delta_n$ to eliminate the indicator term:}
Given $L_n$ and $r$, the indicator term in \cref{thm:boot_disc_error} vanishes once the grid is fine enough so that the worst-case between-grid discrepancy implied by the local variation bound is below the threshold level, namely $L_n\Delta_n/2\le r$.
This observation yields an explicit rule: decrease $\Delta_n$ (or increase $p$) until the condition $L_n\Delta_n/2\le r$ is satisfied.
With the indicator term eliminated, the coverage error reduces to the excursion probability bound $\varepsilon_n$ plus the last term in \cref{thm:boot_disc_error}.
\item[-] \textbf{Caution (Do not take the grid unnecessarily dense):}
Although refining the grid reduces discretization error, it also increases $p$ and hence affects the high-dimensional bootstrap approximation error through $\log(np)$.
Therefore, the grid should be chosen just fine enough to satisfy the mesh condition $L_n\Delta_n/2\le r$ , while keeping $p$ within a range for which the high-dimensional approximation remains valid, that is $n^{-1}B_n^2\log^{5}(np)\to 0$.
\end{enumerate}
\end{rmk}
In the next section, we provide a concrete implementation discussion through a KDE example.
For first-order validity, see \cref{prop:KDE_grid} and Remarks~\ref{rem:plug-in}--\ref{rmk:simple} for an explicit grid rule and practical guidance.
Regarding the stronger goal of making discretization negligible relative to the high-dimensional approximation error, though a fully constant-level prescription is currently out of reach, we record one simple order-based implementation strategy in \cref{rmk:simple}.
\section{Illustration: Kernel Density Estimator} \label{sec:KDE}
As an illustration, we consider the construction of uniform confidence bands for the expectation of kernel density estimator.
\subsection{Theoretical Analysis} \label{subsec:KDE_theory}
In this section, we illustrate how the quantities $B_n$ and $L_n$ are determined in the case of kernel density estimator (KDE) and how they in turn guide an appropriate choice of the grid spacing $\Delta_n$.
KDE is defined as
\begin{align*}
\hat{f}_{h_n}(x) \coloneqq \frac{1}{n}\sum_{i=1}^n K_{i,h_n}(x), \text{ with } K_{i,h_n}(x) \coloneqq \frac{1}{h_n}K\left( \frac{X_i-x}{h_n}\right),
\end{align*}
where $K$ is a kernel function and $h_n>0$ is a bandwidth which satisfy assumptions introduced later.
For notational simplicity, define
\begin{align*}
Y_{i,h_n}(x) \coloneqq \frac{1}{\sqrt{\text{Var}[K_{i,h_n}(x)]}} \left\{ K_{i,h_n}(x) - \mathbb{E}[K_{i,h_n}(x)] \right\}.
\end{align*}
\begin{assumption}\label{as:KDE} \mbox{}
\begin{itemize}
\item[(i)] $\{X_i\}_{i=1}^n$ is an i.i.d.~sample from a distribution with density $f$ and the density $f$ is continuously differentiable, $\sup_{x\in\mathcal{X}}|f'(x)|<\infty$, and $0 < \inf_{x\in\mathcal{X}}f(x) < \sup_{x\in\mathcal{X}} f(x) < \infty$, where $f'(x) \coloneqq \partial f(x)/\partial x$.
\item[(ii)] $\sigma_n(x)$ is continuously differentiable on $\mathcal X$ for all $n\ge 1$, and
\begin{align*}
0< \inf_{x\in\mathcal X} (nh_n)^{1/2}\sigma_n(x) \le \sup_{x\in\mathcal X} (nh_n)^{1/2}\sigma_n(x) < \infty,
\qquad
\sup_{x\in\mathcal X}(nh_n)^{1/2}|\partial_x\sigma_n(x)|<\infty.
\end{align*}
\item[(iii)] The bandwidth $h_n>0$ satisfies $h_n \to 0$ and $nh_n^3 \to \infty$.
\item[(iv)] The kernel function $K:\mathbb{R}\to\mathbb{R}$ is twice continuously differentiable, $\int_{\mathbb{R}}K(u)du = 1$, $\sup_{u\in\mathbb{R}}|K^{(v)}(u)|<\infty$ for $v=0,1,2$ and
\begin{align*}
& \int_{\mathbb{R}}|K(u)|du < \infty, ~ \int_{\mathbb{R}}|u|K^2(u)du < \infty, ~ \int_{\mathbb{R}}K^4(u)du < \infty, \\
& \int_{\mathbb{R}}|K'(u)|du < \infty, ~ \int_{\mathbb{R}}\{K'(u)\}^2du < \infty, \int_{\mathbb{R}}|K''(u)|du < \infty.
\end{align*}
\end{itemize}
\end{assumption}
This kind of assumption is standard in the literature (cf. \citealp{wasserman2006all,li2007nonparametric,tsybakov2008nonparametric}, for example).
The condition $nh_n^3\to\infty$ in \cref{as:KDE}(iii) is stronger than the most standard condition $nh_n \to \infty$ for the analysis of KDE, but this restriction is not unduly strong for practical purposes, since commonly used bandwidth choices such as MISE-optimal rate ($h_n \asymp n^{-1/5}$) satisfy $nh_n^3\to\infty$.
The following lemma provides explicit quantities needed for the grid selection in the KDE example.
In particular, \cref{lem:KDE}(ii) provides a valid choice of $B_n$ in \cref{as:DGP_high-dim}(ii), while \cref{lem:KDE}(iii) verifies \cref{as:DGP_high-dim}(v) by constructing $L_n(\varepsilon_n)$.
We use \cref{lem:KDE}(i) in the discussion on an implementation issue later.
\begin{lemma} \label{lem:KDE} \mbox{}
Under \cref{as:KDE}, the following statements hold.
\begin{itemize}
\item[(i)] For all $x\in\mathcal{X}$, $ \text{Var}[K_{i,h_n}(x)] = h_n^{-1}f(x)\int K^2(u)du + o(h_n^{-1})$.
\item[(ii)] $\max_{1\le j \le p}\mathbb{E}[Y_{i,h_n}^4(x_j)]$ and $\max_{1\le j \le p}\| Y_{i,h_n}(x_j)\|_{\psi_1}$ are bounded as
\begin{align*}
& \max_{1\le j \le p}\mathbb{E}[Y_{i,h_n}^4(x_j)] \le \max_{1\le j \le p}\frac{16 \sup_{x\in\mathcal{X}}|f(x)|}{h_n^3 \text{Var}[K_{i,h_n}(x_j)]^2} \int_{\mathbb{R}}K^4(u)du, \\
& \max_{1\le j \le p}\| Y_{i,h_n}(x_j)\|_{\psi_1} \le \max_{1\le j \le p}\frac{1}{\log 2} \cdot \frac{2\sup_{u\in\mathbb{R}}|K(u)|}{h_n\sqrt{\text{Var}[K_{i,h_n}(x_j)]}}.
\end{align*}
\item[(iii)] For a given $\varepsilon_n\in(0,1)$, define $L_n(\varepsilon_n)$ as follows. Then, \cref{as:DGP_high-dim}(v) holds.
\begin{align*}
L_n(\varepsilon_n) \coloneqq 2\left(\sup_{x\in\mathcal{X}}\frac{1}{\sigma_n(x)} A_1\left( \frac{\varepsilon_n}{2} \right) + \sup_{x\in\mathcal{X}}\frac{\partial\sigma_n(x)}{\partial x}\frac{ 1}{\sigma^2_n(x)} A_0\left( \frac{\varepsilon_n}{2} \right) \right),
\end{align*}
with
\begin{align*}
& A_0(\varepsilon_n) \coloneqq\sqrt{\frac{2v_0\log(2m_{n}/\varepsilon_n)}{n}} + \frac{M_0\log(2m_{n}/\varepsilon_n)}{3n} + \frac{\sup_{u\in\mathbb{R}}|K'(u)|}{n^{1/2}h_n^{1/2}}, \\
& A_1(\varepsilon_n) \coloneqq \sqrt{\frac{2v_1\log(2m_{n}/\varepsilon_n)}{n}} + \frac{M_1\log(2m_{n}/\varepsilon_n)}{3n} + \frac{\sup_{u\in\mathbb{R}}|K''(u)|}{n^{1/2}h_n^{3/2}},
\end{align*}
and
\begin{align*}
& m_{n} \coloneqq \lfloor n^{1/2}h_n^{-3/2}(\overline{x} - \underline{x}) \rfloor + 2, \\
& M_0 \coloneqq 2h_n^{-1}\sup_{u\in\mathbb{R}}|K(u)|, ~~ v_0 \coloneqq h_n^{-1}\sup_{x\in\mathcal{X}}|f(x)|\int_{\mathbb{R}}\{K(u)\}^2du, \\
& M_1 \coloneqq 2h_n^{-2}\sup_{u\in\mathbb{R}}|K'(u)|, ~~ v_1 \coloneqq h_n^{-3}\sup_{x\in\mathcal{X}}|f(x)|\int_{\mathbb{R}}\{K'(u)\}^2du .
\end{align*}
\end{itemize}
The proof of this lemma is provided in \cref{sec:proof:lem:KDE}.
\end{lemma}
\begin{rmk} \label{rem:negligible_L}
Ignoring logarithmic factors, we can see that
\begin{align*}
& \sup_{x\in\mathcal{X}}\sigma_n^{-1}(x) \asymp (nh_n)^{1/2}, ~~A_1(\varepsilon_n)\asymp (nh_n^3)^{-1/2}, \\ & \sup_{x\in\mathcal{X}}\sigma_n^{-2}(x)\{\partial \sigma_n(x)/\partial x\} \asymp (nh_n)^{1/2}, ~~A_0(\varepsilon_n)\asymp (nh_n)^{-1/2}.
\end{align*}
This implies the second term in $L_n(\varepsilon_n)$ is negligible in comparison with the first term, in the case of KDE.
Therefore we can work with
\begin{align*}
\tilde{L}_n(\varepsilon_n) \coloneqq 2 \sup_{x\in\mathcal{X}}\sigma_n^{-1}(x) A_1(\varepsilon_n/2).
\end{align*}
While the second component is of smaller order in the KDE example (as shown above), its magnitude may depend on the estimator, and it need not be negligible in more general settings.
For this reason, we state the general bound with both terms.
\end{rmk}
Given \cref{lem:KDE} and \cref{rem:negligible_L}, we can proceed to determine the grid size.
For each mesh size $\delta\in(0,\overline{x}-\underline{x}]$, let $\mathcal X_\delta=\{x_1,\ldots,x_{p(\delta)}\}$
be the equispaced grid defined in Section~2.
In view of \cref{lem:KDE}(ii), we may take
\[
B_n=\sup_{x\in\mathcal{X}}\Big(B_{\psi,n}(x)\vee B_{4,n}(x)\Big),
\]
where
\[
B_{\psi,n}(x)=\frac{1}{\log 2}\cdot \frac{2\sup_{u\in\mathbb R}|K(u)|}{h_n\sqrt{\text{Var}[K_{i,h_n}(x)]}},
\qquad
B_{4,n}(x)=\sqrt{\frac{16\sup_{t\in\mathcal X}|f(t)|}{h_n^3\text{Var}[K_{i,h_n}(x)]^2}\int_{\mathbb R}K^4(u)\,du }.
\]
Inserting $p=p(\delta)$ and $B_n$ into the definition of $r$ in \cref{thm:boot_disc_error}
yields
\[
r(\delta)=2\left\{\frac{B_n^2\log^3\!\big(n\,p(\delta)\big)}{n}\right\}^{1/4}.
\]
The following proposition gives how to choose $\delta_n$ to ensure the first order validity of uniform inference.
\begin{proposition}\label{prop:KDE_grid}
Suppose that \cref{as:KDE} holds.
Let $\{\varepsilon_n\}_{n\ge 1}$ be a deterministic sequence with $\varepsilon_n\downarrow 0$, and set
$\tilde L_n(\varepsilon_n)$ as in \cref{rem:negligible_L}.
Choose
\[
\delta_n:=\sup\Big\{\delta\in(0,\overline{x}-\underline{x}]:\ \delta \le 2r(\delta)/\tilde L_n(\varepsilon_n)\Big\},
\]
and let $\Delta_n$ denote the maximum gap of the resulting grid $\mathcal X_{\delta_n}$ (as defined in \cref{sec:main}).
Then
\[
\frac{\tilde L_n(\varepsilon_n)\Delta_n}{2}\le r(\delta_n),
\]
and hence the indicator term $1\{\tilde L_n(\varepsilon_n)\Delta_n/2>r\}$ in \cref{thm:boot_disc_error} vanishes for this choice of grid.
\end{proposition}
\begin{proof}
Since the grid is equispaced, $\Delta_n\le \delta_n$, and $\tilde L_n(\varepsilon_n)\delta_n/2\le r(\delta_n)$ by construction.
Therefore $\tilde L_n(\varepsilon_n)\Delta_n/2\le \tilde L_n(\varepsilon_n)\delta_n/2\le r(\delta_n)$.
\end{proof}
We now discuss practical aspects of implementing Proposition~\ref{prop:KDE_grid}.
In particular, we highlight several bottlenecks that arise in applications in \cref{rem:plug-in} and \cref{rmk:implementation_delta}.
Also \cref{rmk:simple} provides a simple implementation procedure.
\begin{rmk}[Implementation Issue 1 : Unknown Quantities] \label{rem:plug-in}
The quantities in \cref{prop:KDE_grid} involves unknown distributional quantities such as $\sup_{x\in\mathcal X}f(x)$, $\sup_{x\in\mathcal{X}}\sigma_n^{-1}(x)$ and $\sup_{x\in\mathcal{X}}\{\text{Var}[K_{i,h_n}(x)]\}^{-1}$.
A feasible implementation can be obtained by replacing these unknown objects with plug-in estimators.
Note that Lemma~\ref{lem:KDE}(i) states that $\text{Var}[K_{i,h_n}(x)] = h_n^{-1}f(x)\int K^2(u)du + o(h_n^{-1})$.
Accordingly, we can approximate $\sup_{x\in\mathcal{X}}\{\text{Var}[K_{i,h_n}(x)]\}^{-1} \approx h_n \{\int_{\mathbb{R}} K^2(u)du\cdot\inf_{x\in\mathcal{X}}f(x)\}^{-1} $ .
Similarly, since $\sigma_n(x)^2=n^{-1}\text{Var}[K_{i,h_n}(x)]$, we have
$\sup_{x\in\mathcal X}\sigma_n(x)^{-1}\approx (nh_n \{\int_{\mathbb{R}} K^2(u)du\cdot\inf_{x\in\mathcal{X}}f(x)\}^{-1})^{1/2}$.
Therefore, it suffices to estimate $\sup_{x\in\mathcal X}f(x)$ and $\inf_{x\in\mathcal X}f(x)$.
Although there are many possible ways, one may use maximum and minimum of histogram estimators with a default binning rule provided by standard software packages.
\end{rmk}
\begin{rmk}[Implementation Issue 2 : Implementation of $\delta_n$] \label{rmk:implementation_delta}
The grid rule in \cref{prop:KDE_grid} is defined through the constraint
$\delta \le 2r(\delta)/\tilde L_n(\varepsilon_n)$, where $r(\delta)$ depends on $\delta$ only through the term
$\log\{np(\delta)\}$ with $p(\delta)=\lfloor(\overline x-\underline x)/\delta\rfloor+2$.
Because of the floor operator, $p(\delta)$ is a step function of $\delta$, and a closed-form solution for the largest admissible $\delta$ is generally inconvenient.
In practice, this issue is easily handled as follows.
Let $|\mathcal{X}|\coloneqq\overline x-\underline x$ and consider the continuous proxy $\tilde p(\delta)\coloneqq|\mathcal{X}|/\delta$ (or $|\mathcal{X}|/\delta+2$), which removes the floor operator.
Substituting $\tilde p(\delta)$ into $r(\delta)$ yields a smooth implicit one-dimensional equation in $\delta$, which can be solved numerically by, for example, the bisection method.
Let $\tilde \delta$ denote a numerical solution of the equality $\delta = 2\tilde r(\delta)/\tilde L_n(\varepsilon_n)$,
where $\tilde r(\delta)$ is defined as $r(\delta)$ with $p(\delta)$ replaced by $\tilde p(\delta)$.
We then define an integer number of grid points by
\[
\hat p:=\left\lceil \frac{|\mathcal{X}|}{\tilde \delta}\right\rceil+2
\qquad\text{and set}\qquad
\hat\delta:=\frac{|\mathcal{X}|}{\hat p-1}.
\]
This rounding produces a slightly finer grid (smaller mesh size) and is therefore conservative for the purpose of satisfying the original inequality.
\end{rmk}
\begin{rmk}[Implementation Issue 3 : Simple Implementation] \label{rmk:simple}
In this remark, we discuss a order-based grid choice.
While not specifying the constants is not ideal from the viewpoint of practical transparency, it greatly simplifies implementation by providing an easy-to-use rule.
Throughout, we ignore logarithmic factors (e.g., those involving $\log(2m_n/\varepsilon_n)$ and $\log(np_n)$) and focus on dominant polynomial rates in $n$ and $h_n$.
\begin{itemize}
\item[(i)] \textbf{Validity only.}
To ensure first-order validity, it suffices to choose the grid so that the indicator term in
\cref{thm:boot_disc_error} vanishes, i.e., $L_n(\varepsilon_n)\Delta_n/2\lesssim r$.
Observe that $L_n(\varepsilon_n) $ and $r(\delta)$ satisfy $L_n(\varepsilon_n)\asymp h_n^{-1}$ and
$r(\delta)\asymp (nh_n)^{-1/4}$ (since $B_n^2\asymp h_n^{-1}$).
Therefore a natural order-based choice is
\[
\Delta_n = \frac{2r}{L_n(\varepsilon_n)} c_{\Delta}n^{-\gamma} = c_\Delta n^{-1/4-\gamma}h_n^{3/4},
\]
or equivalently
\[
p_n = \left\lceil \frac{|\mathcal{X}|}{\Delta_n} \right\rceil = \left\lceil c_\Delta^{-1}|\mathcal{X}|n^{1/4 + \gamma}h_n^{-3/4} \right\rceil,
\]
with small constants $c_\Delta\in(0,1)$ and $\gamma>0$.
In practice, $\gamma$ should be very small.
\item[(ii)] \textbf{Discretization error smaller than the Gaussian approximation error.}
Under the same grid choice in (i), the indicator term in \cref{thm:boot_disc_error} eventually vanishes.
Moreover, choosing $\varepsilon_n=o(\rho_n)$ makes the rare-spike probability term negligible, where
\[
\rho_n:=\left(\frac{B_n^2\log^5(np_n)}{n}\right)^{1/4}.
\]
In the KDE example, $p_n$ in (i) grows at most polynomially in $n$, so $\log(np_n)=O(\log n)$ and $\rho_n\asymp (nh_n)^{-1/4}$ up to logarithmic factors.
Hence, once the grid is chosen for validity, the overall accuracy is driven by the high-dimensional approximation term rather than by discretization.
\end{itemize}
\end{rmk}
\section{Proofs} \label{sec:proof}
\subsection{Proof of \cref{thm:boot_disc_error}} \label{sec:proof_main}
First, observe that
\begin{align*}
\rho^\star_n \coloneqq& \sup_{t\in\mathbb{R}}\left|\mathbb{P}^\star\left(\max_{1\le j\le p}|\hat{T}_n^\star(x_j)| \le t\mid X_{1:n}\right) - \mathbb{P}\left(\sup_{x\in\mathcal{X}}|\hat{T}_{n}(x)| \le t\right)\right| \\
& \le \sup_{t\in\mathbb{R}}\left|\mathbb{P}^\star\left(\max_{1\le j\le p}|\hat{T}_n^\star(x_j)| \le t\mid X_{1:n}\right) - \mathbb{P}\left(\max_{1\le j \le p}|\hat{T}_{n}(x_j)| \le t\right)\right| \\
& \quad + \sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j \le p}|\hat{T}_{n}(x_j)| \le t\right) - \mathbb{P}\left(\sup_{x\in\mathcal{X}}|\hat{T}_{n}(x)| \le t\right)\right| \\
& \eqqcolon (\rho^\star. I) + (\rho^\star.II).
\end{align*}
Since $(\rho^\star.I) \le 1$, the bound $(\rho^\star.I) \le C\{n^{-1}B_n^2\log^5(np)\}^{1/4}$ is trivial when $n^{-1}B_n^2\log^5(np) > 1$, so we can assume $n^{-1}B_n^2\log^5(np) \le 1$ without loss of generality.
In the following evaluation, we often use this inequality.
First, in terms of $ (\rho^\star. I)$, it holds that
\begin{align*}
(\rho^\star. I) &\le \sup_{t\in\mathbb{R}}\left|\mathbb{P}^\star\left(\max_{1\le j\le p}|T_n^\star(x_j)| \le t\mid X_{1:n}\right) - \mathbb{P}\left(\max_{1\le j \le p}|T_n(x_j)| \le t\right)\right| \nonumber\\
& \quad + \sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j\le p}|\hat{T}_n(x_j)| \le t\right) - \mathbb{P}\left(\max_{1\le j\le p}|{T}_n(x_j)| \le t\right)\right| \nonumber\\
& \quad + \sup_{t\in\mathbb{R}}\left|\mathbb{P}^\star\left(\max_{1\le j\le p}|\hat{T}_n^\star(x_j)| \le t\mid X_{1:n}\right) - \mathbb{P}^\star\left(\max_{1\le j\le p}|T_n^\star(x_j)| \le t\mid X_{1:n}\right)\right| \\
& \coloneqq (\rho^\star. I.I) + (\rho^\star. I.II) + (\rho^\star. I.III),
\end{align*}
with $T_n^\star(x) \coloneqq n^{-1}\sigma_n^{-1}\sum_{i=1}^n w_i^\star\{\psi_{h_n}(X_i,x) - \hat{f}_{h_n}(x)\}$.
From Lemma 4.6 in \cite{CCKK22}, we can see that
\begin{align}
(\rho^\star. I.I) = O_p\left( \left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4} \right). \label{eq:high-dim-boot-error}
\end{align}
Also, from Lemma 15 in \cite[Chapter 2]{le1986asymptotic}, we have
\begin{align}
(\rho^\star. I.II) \le \sup_{t\in\mathbb{R}}\mathbb{P}\left(t< \max_{1\le j \le p}|T_n(x_j)| \le t + r \right) + \sup_{t\in\mathbb{R}} \mathbb{P}\left(\max_{1\le j\le p}\left|\hat{T}_n(x_j) - T_n(x_j) \right| > r\right). \label{eq:rho_star_1-2}
\end{align}
with $r \coloneqq 2\left( n^{-1}B_n^2 \log^3(pn)\right)^{1/4}$.
In terms of the first term in \cref{eq:rho_star_1-2}, observe that
\begin{align*}
& \sup_{t\in\mathbb{R}}\mathbb{P}\left(t < \max_{1\le j \le p}|T_n(x_j)| \le t + r \right) \\
& \le \sup_{t\in\mathbb{R}}\mathbb{P}\left(t < \max_{1\le j \le p}|G_n(x_j)| \le t + r \right) + \sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j\le p}|T_n(x_j)| \le t\right) - \mathbb{P}\left(\max_{1\le j \le p}|G_n(x_j)| \le t\right)\right|.
\end{align*}
Nazarov's inequality gives
\begin{align*}
& \sup_{t\in\mathbb{R}}\mathbb{P}\left(t < \max_{1\le j \le p}|G_n(x_j)| \le t + r \right) \le r(\sqrt{2\log p} + 2) = O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right) ,
\end{align*}
and Theorem 2.1 in \cite{CCKK22} states that there exists a universal constant $C$ such that
\begin{align}
\sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j\le p}|T_n(x_j)| \le t\right) - \mathbb{P}\left(\max_{1\le j \le p}|G_n(x_j)| \le t\right)\right| \le C\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}. \label{eq:hd-CLT}
\end{align}
Therefore
\begin{align}
\sup_{t\in\mathbb{R}}\mathbb{P}\left(t < \max_{1\le j \le p}|T_n(x_j)| \le t + r \right) = O\left( \left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4} \right). \label{eq:rho_star_1-2-1}
\end{align}
Next, we evaluate the second term in \cref{eq:rho_star_1-2}.
Since
\begin{align*}
\max_{1\le j\le p}\left|\hat{T}_n(x_j) - T_n(x_j) \right| \le \max_{1\le j\le p}|T_n(x_j)| \max_{1\le j\le p} \frac{|\hat{\sigma}^2_n(x_j) - \sigma_n^2(x_j)|}{\hat{\sigma}_n(x_j)|\hat{\sigma}_n(x_j) + \sigma_n(x_j)|},
\end{align*}
we can see that
\begin{align*}
& \mathbb{P}\left(\max_{1\le j\le p}\left|\hat{T}_n(x_j) - T_n(x_j) \right| > r\right) \\
& \le \mathbb{P}\left( \max_{1\le j\le p}|T_n(x_j)| \max_{1\le j\le p} \frac{|\hat{\sigma}^2_n(x_j) - \sigma_n^2(x_j)|}{\hat{\sigma}_n(x_j)|\hat{\sigma}_n(x_j) + \sigma_n(x_j)|} > r\right) \\
& \le \mathbb{P}\left( \max_{1\le j\le p}|T_n(x_j)| \cdot C\sqrt{\frac{B_n^2 \log(pn)}{n}} > r\right) + \mathbb{P}\left( \max_{1\le j\le p} \frac{|\hat{\sigma}^2_n(x_j) - \sigma_n^2(x_j)|}{\hat{\sigma}_n(x_j)|\hat{\sigma}_n(x_j) + \sigma_n(x_j)|} > C\sqrt{\frac{B_n^2 \log(pn)}{n}} \right).
\end{align*}
Since the variance estimator $\hat{\sigma}_n^2(x)$ is defined as \cref{eq:variance_estimator}, under \cref{as:DGP_high-dim}(ii), Lemma 4.2 in \cite{CCKK22} gives
\begin{align*}
& \mathbb{P}\left( \max_{1\le j\le p} \frac{|\hat{\sigma}^2_n(x_j) - \sigma_n^2(x_j)|}{\hat{\sigma}_n(x_j)|\hat{\sigma}_n(x_j) + \sigma_n(x_j)|} > C\sqrt{\frac{B_n^2 \log(pn)}{n}} \right)\\
& \quad \le \frac{1}{n} + 3\left( \frac{B_n^2 \log^3(pn)}{n}\right)^{1/2} = O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
Also, it holds that
\begin{align*}
& \mathbb{P}\left( \max_{1\le j\le p}|T_n(x_j)| \cdot C\sqrt{\frac{B_n^2 \log(pn)}{n}} > r\right) \\
& = 1- \mathbb{P}\left( \max_{1\le j\le p}|T_n(x_j)| \cdot C\sqrt{\frac{B_n^2 \log(pn)}{n}} \le r\right) \\
& \le \mathbb{P}\left( \max_{1\le j\le p}|G_n(x_j)| \cdot C\sqrt{\frac{B_n^2 \log(pn)}{n}} > r\right)+ \sup_{t\in\mathbb{R}}\left| \mathbb{P}\left(\max_{1\le j\le p}|T_n(x_j)| \le t \right) - \mathbb{P}\left(\max_{1\le j\le p}|G_n(x_j)| \le t \right)\right|.
\end{align*}
Since, from the union bound and the standard property of the Gaussian tail, we have
\begin{align*}
\mathbb{P}\left( \max_{1\le j\le p}|G_n(x_j)|\cdot C\sqrt{\frac{B_n^2 \log(pn)}{n}} > r\right) \le 2p\exp\left( - \frac{r^2}{2} \cdot \left( C\sqrt{\frac{B_n^2 \log(pn)}{n}} \right)^{-2} \right).
\end{align*}
From $r = 2\left( n^{-1}B_n^2 \log^3(pn)\right)^{1/4}$ and $B_n^2 \log^5(np)/ \le 1$ we have
\begin{align*}
& 2p\exp\left( - \frac{r^2}{2} \cdot \left( C\sqrt{\frac{B_n^2 \log(pn)}{n}} \right)^{-2} \right) = O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
In conjunction with \cref{eq:hd-CLT}, we have
\begin{align*}
\mathbb{P}\left( \max_{1\le j\le p}|T_n(x_j)| \cdot C\sqrt{\frac{B_n^2 \log(pn)}{n}} > r\right) = O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
Summing up
\begin{align}
\mathbb{P}\left(\max_{1\le j\le p}\left|\hat{T}_n(x_j) - T_n(x_j) \right| > r\right) = O\left( \left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4} \right). \label{eq:rho_star_1-2-2}
\end{align}
Inserting the evaluation \cref{eq:rho_star_1-2-1} and \cref{eq:rho_star_1-2-2} to \cref{eq:rho_star_1-2}, we have
\begin{align}
(\rho^\star.I.II)= O\left( \left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4} \right). \label{eq:rho_star_1-2_bound}
\end{align}
Similarly, we can see that
\begin{align}
(\rho^\star.I.III)= O_p\left( \left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4} \right). \label{eq:rho_star_1-3_bound}
\end{align}
From \cref{eq:high-dim-boot-error}, \cref{eq:rho_star_1-2_bound} and \cref{eq:rho_star_1-3_bound}, we have
\begin{align}
(\rho^\star. I) \le (\rho^\star. I.I) + (\rho^\star. I.II) + + (\rho^\star. I.III) =O_p\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right). \label{eq:rho_star_1_error}
\end{align}
In terms of $(\rho^\star.II)$, from Lemma 15 in \cite[Chapter 2]{le1986asymptotic}, we have
\begin{align*}
(\rho^\star.II) & \le \sup_{t\in\mathbb{R}} \mathbb{P}\left(t< \max_{1\le j \le p}|\hat{T}_{n}(x_j)| \le t + r\right) + \mathbb{P}\left(\sup_{x\in\mathcal{X}}|\hat{T}_{n}(x)| - \max_{1\le j \le p}|\hat{T}_{n}(x_j)| > r \right) \\
& \le \sup_{t\in\mathbb{R}}\mathbb{P}\left(t < \max_{1\le j \le p}|G_{n}(x_j)| \le t + r\right) \\
& \quad + \sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j\le p}|\hat{T}_n(x_j)| \le t\right) - \mathbb{P}\left(\max_{1\le j \le p}|G_n(x_j)| \le t\right)\right| \\
& \quad +\mathbb{P}\left(\sup_{x\in\mathcal{X}}|\hat{T}_{n}(x)| - \max_{1\le j \le p}|\hat{T}_{n}(x_j)| > r\right) \\
& \eqqcolon (\rho^\star.II.I) + (\rho^\star.II.II) + (\rho^\star.II.III).
\end{align*}
Nazarov's inequality gives
\begin{align*}
(\rho^\star.II.I) \le r(\sqrt{2\log p} + 2) =O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
Also, from \cref{eq:rho_star_1-2_bound} and \cref{eq:hd-CLT}, we have
\begin{align*}
(\rho^\star.II.II)
& = \sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j\le p}|\hat{T}_n(x_j)| \le t\right) - \mathbb{P}\left(\max_{1\le j \le p}|G_n(x_j)| \le t\right)\right| \\
& \le (\rho^\star.I.II)+ \sup_{t\in\mathbb{R}}\left|\mathbb{P}\left(\max_{1\le j\le p}|T_n(x_j)| \le t\right) - \mathbb{P}\left(\max_{1\le j \le p}|G_n(x_j)| \le t\right)\right| \\
& = O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
Finally, in terms of $ (\rho^\star.II.III)$, observe that
\begin{align*}
0 \le \sup_{x\in\mathcal{X}}|\hat{T}_{n}(x)| - \max_{1\le j \le p}|\hat{T}_{n}(x_j)| \le \sup_{\{(x,y):|x-y|\le \sup_{x\in\mathcal{X}}\min_{1\le j \le p} |x-x_j| \}} |\hat{T}_{n}(x) - \hat{T}_{n}(y)|.
\end{align*}
Recall that we define the maximum gap as $\Delta_n := \max_{1\le j\le p_n-1}(x_{j+1}-x_j)$.
Then, it holds that
\begin{align*}
& \sup_{\{(x,y):|x-y|\le \sup_{x\in\mathcal{X}}\min_{1\le j \le p} |x-x_j| \}} |\hat{T}_{n}(x) - \hat{T}_{n}(y)| \\
& \quad \le \sup_{x\in\mathcal{X}}\min_{1\le j \le p} |x-x_j| \cdot\sup_{\{(x,y):|x-y|\le \Delta_n\} }\frac{ |\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|}.
\end{align*}
This implies
\begin{align*}
(\rho^\star.II.III) & \le \mathbb{P}\left( \sup_{x\in\mathcal{X}}\min_{1\le j \le p} |x-x_j| \cdot\sup_{\{(x,y):|x-y|\le \Delta_n\} }\frac{ |\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > r\right).
\end{align*}
Under Assumption \cref{as:DGP_high-dim}, it holds that
\begin{align*}
\mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > L_n \right) \to 0.
\end{align*}
Therefore
\begin{align*}
& \mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n } |\hat{T}_{n}(x) - \hat{T}_{n}(y)| > r\right)\\
& \le \mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > L_n \right) + 1\{L_n\ \sup_{x\in\mathcal{X}}\min_{1\le j \le p} |x-x_j| > r\} \\
& \le \mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > L_n \right) + 1\{L_n \Delta_n/2 > r\},
\end{align*}
where the final inequality follows from $\sup_{x\in\mathcal{X}}\min_{1\le j \le p} |x-x_j|\le \Delta_n/2 $.
Summing up, we have
\begin{align}
& (\rho^\star.II) \nonumber\\
&\le \mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n }\frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > L_n \right) + 1\{L_n\Delta_n/2 > r\} + O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right). \label{eq:rho_star_2_bound}
\end{align}
From \cref{eq:rho_star_1_error} and \cref{eq:rho_star_2_bound}, we have
\begin{align*}
\rho^\star_n \le \mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n }\frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > L_n \right) + 1\{L_n\Delta_n /2> r\} + O_p\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
In the same way as Step 3 in the proof of Theorem 2 in \cite{kato2018uniform}, we have
\begin{align*}
& \left|\mathbb{P}\left( \sup_{x\in\mathcal{X}} |\hat{T}_n(x)| \le \hat{c}^\star_{n,1-\alpha} \right) - (1-\alpha) \right|\\
& \le 3\Bigg\{\mathbb{P}\left( \sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_{n}(x) - \hat{T}_{n}(y)|}{|x-y|} > L_n \right) + 1\{L_n\Delta_n/2 > r\}\Bigg\} + O\left(\left( \frac{B_n^2 \log^5(np)}{n}\right)^{1/4}\right).
\end{align*}
This completes the proof.
\subsection{Proof of \cref{lem:KDE}} \label{sec:proof:lem:KDE}
\begin{proof}[Proof of \cref{lem:KDE}(i)]
Since
\begin{align*}
\mathbb{E}[K^2_{i,h_n}(x)] = \frac{1}{h_n}\int K^2(u)f(x+uh_n)du, ~~~ \mathbb{E}[K_{i,h_n}(x)] = \int K(u)f(x+uh_n)du,
\end{align*}
, under \cref{as:KDE}, it holds that
\begin{align*}
& \sup_{x\in\mathcal{X}}\left| \text{Var}[K_{i,h_n}(x)] - \frac{f(x)}{h_n}\int K^2(u)du\right| \\
& = \sup_{x\in\mathcal{X}}\left| \mathbb{E}[K^2_{i,h_n}(x)] - \mathbb{E}[K_{i,h_n}(x)]^2 - \frac{1}{h_n}\int f(x)K^2(u)du\right| \\
& \le \sup_{x\in\mathcal{X}}\left| \frac{1}{h_n} \int K^2(u)\{f(x+uh_n) - f(x)\} du \right| + \sup_{x\in\mathcal{X}}\left| \left( \int K(u)f(x+uh_n)du \right)^2\right| \\
& \le \sup_{x\in\mathcal{X}}|f'(x)|\left| \int |u|K^2(u) du \right| + \sup_{x\in\mathcal{X}}\left| f(x)\right|^2 \left(\int |K(u)|du\right)^2 \\
& = o(h_n^{-1}).
\end{align*}
\end{proof}
\begin{proof}[Proof of \cref{lem:KDE}(ii)]
First, from $c_r$ inequality and Jensen's inequality, we have
\begin{align*}
\mathbb{E}[Y_{i,h_n}^4(x)]
& \le \frac{16}{h_n^4\text{Var}[K_{i,h_n}(x)]^2} \int_{\mathcal{X}} K^4\left( \frac{t-x}{h_n} \right)f(t)dt \le \frac{16 \sup_{x\in\mathcal{X}}|f(x)|}{h_n^3 \text{Var}[K_{i,h_n}(x)]^2} \int_{\mathbb{R}}K^4(u)du.
\end{align*}
Since $\sup_{u\in\mathbb{R}}|K(u)| < \infty$, the triangle inequality gives
\begin{align*}
|Y_{i,h_n}(x)|= \frac{|K_{i,h_n}(x) - \mathbb{E}[K_{i,h_n}(x)]|}{\sqrt{\text{Var}[K_{i,h_n}(x)]}} \le \frac{2\sup_{u\in\mathbb{R}}|K(u)|}{h_n\sqrt{\text{Var}[K_{i,h_n}(x)]}}.
\end{align*}
Therefore, by definition of $\psi_1$-norm, we have
\begin{align*}
\|Y_{i,h_n}(x)\|_{\psi_1} \le \frac{1}{\log 2} \cdot \frac{2\sup_{u\in\mathbb{R}}|K(u)|}{h_n\sqrt{\text{Var}[K_{i,h_n}(x)]}}.
\end{align*}
\end{proof}
Before the proof of \cref{lem:KDE}(iii), we provide additional auxiliary results.
The proofs of these Lemmas are provided in \cref{subsec_proof:lem:dominated_convergence_theorem,subsec_proof:lem:variance_consistency,subsec_proof:lem:maximal_with_constant}.
\begin{lemma}\label{lem:dominated_convergence_theorem} \mbox{}
Under \cref{as:KDE}, it holds that
\begin{align*}
\frac{\partial}{\partial x}\mathbb{E}[\hat{f}_{h_n}(x)] = \mathbb{E}\left[\frac{\partial}{\partial x}\hat{f}_{h_n}(x) \right], \quad \frac{\partial}{\partial x}\mathbb{E}\left[\frac{\partial}{\partial x}\hat{f}_{h_n}(x) \right] = \mathbb{E}\left[\frac{\partial^2}{\partial^2 x}\hat{f}_{h_n}(x) \right].
\end{align*}
\end{lemma}
\begin{lemma} \label{lem:variance_consistency} \mbox{}
Under \cref{as:KDE},
\begin{itemize}
\item[(i)] $\sup_{x\in\mathcal{X}}\left|\frac{\hat\sigma_n(x)}{\sigma_n(x)} -1\right| = o_p(1)$.
\item[(ii)] $\sup_{x\in\mathcal{X}}\left|\frac{\partial}{\partial x}\hat\sigma_n(x) - \frac{\partial}{\partial x}\sigma_n(x)\right| = o_p(1)$.
\end{itemize}
\end{lemma}
\begin{lemma} \label{lem:maximal_with_constant} \mbox{}
Under \cref{as:KDE},
\begin{itemize}
\item[(i)]
for any $\varepsilon\in(0,1)$, it holds that
\begin{align*}
&\mathbb{P}\Bigg( \sup_{x\in\mathcal{X}} \left|\hat{f}_{h_n}(x) - \mathbb{E}\left[\hat f_{h_n}(x)\right]\right| > A_0(\varepsilon) \Bigg)\le \varepsilon,
\end{align*}
\item[(ii)]
for any $\varepsilon\in(0,1)$, it holds that
\begin{align*}
&\mathbb{P}\Bigg( \sup_{x\in\mathcal{X}} \left|\frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right]\right| > A_1(\varepsilon)\Bigg) \le \varepsilon.
\end{align*}
\end{itemize}
\end{lemma}
\begin{proof}[Proof of \cref{lem:KDE}(iii)]
From the mean-value theorem, we can see that
\begin{align*}
\sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_n(x) - \hat{T}_n(y)|}{|x-y|} \le \sup_{x\in\mathcal{X}}\left|\frac{\partial}{\partial x}\hat{T}_n(x)\right| .
\end{align*}
This implies
\begin{align*}
\mathbb{P}\left(\sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_n(x) - \hat{T}_n(y)|}{|x-y|} > L_n\right) \le \mathbb{P}\left(\sup_{x\in\mathcal{X}}\left|\frac{\partial}{\partial x}\hat{T}_n(x)\right| > L_n\right).
\end{align*}
Observe that
\begin{align*}
\frac{\partial}{\partial x}\hat{T}_n(x) \coloneqq \frac{1}{\hat\sigma_n(x)}\left( \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \frac{\partial}{\partial x}\mathbb{E}[\hat f_{h_n}(x)] \right) - \frac{\hat{f}_{h_n}(x) - \mathbb{E}[\hat f_{h_n}(x)]}{\hat\sigma^2_n(x)} \partial_x\hat{\sigma}_n(x).
\end{align*}
From Lemma \cref{lem:dominated_convergence_theorem}, we can work with
\begin{align*}
\frac{\partial}{\partial x}\hat{T}_n(x) = \frac{1}{\hat\sigma_n(x)}\left( \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right] \right) - \frac{\hat{f}_{h_n}(x) - \mathbb{E}[\hat f_{h_n}(x)]}{\hat\sigma^2_n(x)} \partial_x\hat{\sigma}_n(x).
\end{align*}
From \cref{lem:variance_consistency}, it holds that
\begin{align*}
& \sup_{x\in\mathcal{X}}\left| \frac{\partial}{\partial x}\hat{T}_n(x) \right| \\
&\le \{1+o_p(1)\}\sup_{x\in\mathcal{X}}\frac{1}{\sigma_n(x)} \left| \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right]\right|
\\
& \quad\quad\quad+ \{1+o_p(1)\}\sup_{x\in\mathcal{X}}\frac{\partial\sigma_n(x)}{\partial x}\frac{1}{\sigma^2_n(x)} \left| \hat{f}_{h_n}(x) - \mathbb{E}\left[\hat f_{h_n}(x)\right]\right| \\
& \le 2\left( \sup_{x\in\mathcal{X}}\frac{1}{\sigma_n(x)} \left| \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right]\right| + \sup_{x\in\mathcal{X}}\frac{\partial\sigma_n(x)}{\partial x}\frac{1}{\sigma^2_n(x)} \left| \hat{f}_{h_n}(x) - \mathbb{E}\left[\hat f_{h_n}(x)\right]\right| \right),
\end{align*}
with probability approaching to $1$.
Taking
\begin{align*}
L_n = 2\left(\sup_{x\in\mathcal{X}}\frac{1}{\sigma_n(x)} A_1\left( \frac{\varepsilon}{2} \right) + \sup_{x\in\mathcal{X}}\frac{\partial\sigma_n(x)}{\partial x}\frac{1}{\sigma^2_n(x)} A_0\left( \frac{\varepsilon}{2} \right) \right),
\end{align*}
then it holds that
\begin{align*}
& \Bigg\{ \sup_{x\in\mathcal{X}} \left| \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right]\right| \le A_1\left( \frac{\varepsilon}{2}\right), \\
& \quad\quad\sup_{x\in\mathcal{X}} \left| \hat{f}_{h_n}(x) - \mathbb{E}\left[\hat f_{h_n}(x)\right]\right| \le A_0\left( \frac{\varepsilon}{2} \right)\Bigg\} \implies \sup_{x\in\mathcal{X}}\left| \frac{\partial}{\partial x}\hat{T}_n(x) \right| \le L_n,
\end{align*}
with probability approaching to $1$.
Therefore, the union bound and \cref{lem:maximal_with_constant} give
\begin{align*}
& \mathbb{P}\left(\sup_{x\in\mathcal{X}}\left|\frac{\partial}{\partial x}\hat{T}_n(x)\right| > L_n\right) \\
& \le \mathbb{P}\left( \sup_{x\in\mathcal{X}} \left| \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right]\right| \le A_1\left( \frac{\varepsilon}{2}\right)\right) \\
& \quad + \mathbb{P}\left(\sup_{x\in\mathcal{X}} \left| \hat{f}_{h_n}(x) - \mathbb{E}\left[\hat f_{h_n}(x)\right]\right| \le A_0\left( \frac{\varepsilon}{2} \right) \right) + o(1) \le \varepsilon + o(1).
\end{align*}
This implies that we can take
\begin{align*}
L_n = 2\left(\sup_{x\in\mathcal{X}}\frac{1}{\sigma_n(x)} A_1\left( \frac{\varepsilon}{2} \right) + \sup_{x\in\mathcal{X}}\frac{\partial\sigma_n(x)}{\partial x}\frac{1}{\sigma^2_n(x)} A_0\left( \frac{\varepsilon}{2} \right) \right),
\end{align*}
so that
\begin{align*}
\mathbb{P}\left(\sup_{\{(x,y):|x-y|\le \Delta_n } \frac{|\hat{T}_n(x) - \hat{T}_n(y)|}{|x-y|} > L_n\right) \le \varepsilon + o(1).
\end{align*}
\end{proof}
\subsubsection{Proof of \cref{lem:dominated_convergence_theorem}} \label{subsec_proof:lem:dominated_convergence_theorem}
\begin{proof}[Proof of \cref{lem:dominated_convergence_theorem}(i)]
Observe that
\begin{align*}
\mathbb{E}[\hat{f}_{h_n}(x)] = \int \frac{1}{h_n}K\left( \frac{t-x}{h_n}\right)f(t)dt,
\end{align*}
and the derivative of the integrant is given by
\begin{align*}
\frac{\partial}{\partial x}\frac{1}{h_n}K\left( \frac{t-x}{h_n}\right)f(t) = -\frac{1}{h_n^2}K'\left( \frac{t-x}{h_n}\right)f(t).
\end{align*}
Then, under \cref{as:KDE}, it holds that
\begin{align*}
& \int \left| \frac{\partial}{\partial x}\frac{1}{h_n}K\left( \frac{t-x}{h_n}\right)f(t)\right| dt \le \frac{\sup_{x\in\mathcal{X}}|f(x)|}{h_n} \int \left|K'\left( u\right)\right|du <\infty.
\end{align*}
Therefore, the dominated convergence theorem completes the proof.
\end{proof}
\begin{proof}[Proof of \cref{lem:dominated_convergence_theorem}(ii)]
Similarly to the proof of \cref{lem:dominated_convergence_theorem}(i),
\begin{align*}
\mathbb{E}\left[ \frac{\partial}{\partial x} \hat{f}_{h_n}(x) \right] = \int -\frac{1}{h_n^2}K'\left( \frac{t-x}{h_n}\right)f(t),
\end{align*}
and the derivative of the integrant is given by
\begin{align*}
\frac{\partial}{\partial x}\left\{-\frac{1}{h_n^2}K'\left( \frac{t-x}{h_n}\right)f(t)\right\} = \frac{2}{h_n^3}K''\left( \frac{t-x}{h_n}\right)f(t).
\end{align*}
Then, under \cref{as:KDE}, it holds that
\begin{align*}
\int \left| \frac{\partial}{\partial x}\left\{-\frac{1}{h_n^2}K'\left( \frac{t-x}{h_n}\right)f(t)\right\} \right|dt \le \frac{\sup_{x\in\mathcal{X}}|f(x)|}{h_n^2} \int \left|K''\left( u\right)\right|du <\infty.
\end{align*}
Therefore, the dominated convergence theorem completes the proof.
\end{proof}
\subsubsection{Proof of \cref{lem:variance_consistency}} \label{subsec_proof:lem:variance_consistency}
\begin{proof}[Proof of \cref{lem:variance_consistency}]
Likewise the proof of Lemma 6 and 7 in the supplementary material of \cite{imai2025doubly}, under \cref{as:KDE}, we can show that $\sup_{x\in\mathcal{X}}\left|\frac{\hat\sigma_n(x)}{\sigma_n(x)} -1\right|$ and $\sup_{x\in\mathcal{X}}\left|\frac{\partial}{\partial x}\hat\sigma_n(x) - \frac{\partial}{\partial x}\sigma_n(x)\right|$ are suprema of empirical process index by the VC-type class using Lemmas 2.6.15, 2.6.16 and 2.6.18 of \cite{van1996weak} (See Definition 2.1 of \citealp{chernozhukov2014gaussian} for the definition of VC-type class).
Then, we can show that the statements hold using Corollary 5.1 in \cite{chernozhukov2014gaussian}.
\end{proof}
\subsubsection{Proof of \cref{lem:maximal_with_constant}} \label{subsec_proof:lem:maximal_with_constant}
First we show \cref{lem:maximal_with_constant}(ii), then show \cref{lem:maximal_with_constant}(i).
\begin{proof}[Proof of \cref{lem:maximal_with_constant}(ii)]
Define $D_n(x) \coloneqq \frac{\partial}{\partial x}\hat{f}_{h_n}(x) - \mathbb{E}\left[\frac{\partial}{\partial x}\hat f_{h_n}(x)\right]$.
Then
\begin{align*}
D_n(x) = \frac{1}{n}\sum_{i=1}^n \xi_i(x), \text{ with } \xi_i(x)\coloneqq - \frac{1}{h_n^2}K'_{i,h_n}(x) - \mathbb{E}\left[ - \frac{1}{h_n^2}K'_{i,h_n}(x) \right].
\end{align*}
For each $x\in\mathcal{X}$, we can see that $|\xi_i(x)| \le 2h_n^{-2}\sup_{u\in\mathbb{R}}|K'(u)| \eqqcolon M_1$ and
\begin{align*}
\text{Var}[\xi_i(x)]
&\le \mathbb{E}\left[ \left\{ \frac{1}{h_n^2} K'_{i,h_n}(x)\right\}^2 \right] \\
& = \frac{1}{h_n^4} \int (K')^2\left( \frac{t-x}{h_n} \right)f(t)dt \le \frac{\sup_{x\in\mathcal{X}}|f(x)|}{h_n^3} \int_{\mathbb{R}} \{K'(u)\}^2 du \eqqcolon v_1.
\end{align*}
Then, for any $t>0$, Bernstein's inequality gives,
\begin{align}
\mathbb{P}\left( |D_n(x)| > \sqrt{\frac{2v_1t}{n}} + \frac{M_1t}{3n}\right) \le 2\exp(-t). \label{eq:uni_bernstein}
\end{align}
Define $\eta_n \coloneqq n^{-1/2}h_n^{3/2}$ and the grid points $\{x_k\}_{k=1}^{m_{1,n}}$ over $\mathcal{X}$ as
\begin{align*}
x_k \coloneqq \underline{x} + (k-1)\eta_n, ~~ k =1,\dots, m_{1,n}-1 ~~ x_{m_{1,n}} \coloneqq \overline{x}, ~~ m_{1,n} \coloneqq \left\lfloor \frac{|\overline{x} - \underline{x}|}{\eta_n} \right\rfloor +2.
\end{align*}
Then the union bound and \cref{eq:uni_bernstein} give
\begin{align*}
\mathbb{P}\left( \max_{1\le k \le m_{1,n}}|D_n(x_k)| > \sqrt{\frac{2v_1t}{n}} + \frac{M_1t}{3n}\right) \le 2m_{1,n}\exp(-t).
\end{align*}
Taking $t_{n}(\varepsilon) \coloneqq \log(2m_{1,n}/\varepsilon)$ then, since $2m_{1,n}\exp(-t_{n}(\varepsilon)) = \varepsilon$,
\begin{align}
\mathbb{P}\left( \max_{1\le k \le m_{1,n}}|D_n(x_k)| > \sqrt{\frac{2v_1t_n(\varepsilon)}{n}} + \frac{M_1t_n(\varepsilon)}{3n}\right) \le \varepsilon. \label{eq:D-bernstein}
\end{align}
For any $x\in\mathcal{X}$, there exists $k \in \{1,\dots, m_{1,n}\}$ such that $|x - x_{k}| \le \eta_n/2 = n^{-1/2}h_n^{3/2}/2$.
Using Lemma \cref{lem:dominated_convergence_theorem}, we can see that $|\partial D_n(x)/\partial x| \le 2h_n^{-3}\sup_{u\in\mathbb{R}}|K''(u)|$.
Therefore mean value theorem, we have
\begin{align*}
& |D_n(x) - D_n(x_k)| \\
& \le |x-x_k| \sup_{x\in\mathcal{X}}\left| \frac{\partial}{\partial x}D_n(x)\right| \le \frac{h_n^{3/2}}{2n^{1/2}} \cdot \frac{2\sup_{u\in\mathbb{R}}|K''(u)|}{h_n^3} = \frac{\sup_{u\in\mathbb{R}}|K''(u)|}{n^{1/2}h_n^{3/2}}.
\end{align*}
Therefore
\begin{align}
\sup_{x\in\mathcal{X}}|D_n(x)| \le \max_{1\le k \le m_{1,n}} |D_n(x_k)| + \frac{\sup_{u\in\mathbb{R}}|K''(u)|}{n^{1/2}h_n^{3/2}}. \label{eq:D-mean-value}
\end{align}
From \cref{eq:D-bernstein} and \cref{eq:D-mean-value}, we have
\begin{align*}
& \mathbb{P}\left( \sup_{x\in\mathcal{X}}|D_n(x)| > \sqrt{\frac{2v_1\log(2m_{1,n}/\varepsilon)}{n}} + \frac{M_1\log(2m_{1,n}/\varepsilon)}{3n} + \frac{\sup_{u\in\mathbb{R}}|K''(u)|}{n^{1/2}h_n^{3/2}} \right) \le \varepsilon.
\end{align*}
\end{proof}
\begin{proof}[Proof of \cref{lem:maximal_with_constant}(i)]
The proof is identical to that of part (ii) by replacing $K''$ with $K'$, $K'$ with $K$ and adjusting the scaling accordingly.
Specifically, define $D_{0,n}(x) \coloneqq \hat{f}_{h_n}(x) - \mathbb{E}[\hat{f}_{h_n}(x)] = n^{-1}\sum_{i=1}^n \xi_{0,i}(x)$ with $\xi_{0,i}(x) \coloneqq K_{i,h_n}(x) - \mathbb{E}[K_{i,h_n}(x)]$.
Then $|\xi_{0,i}(x)| \le M_0$ and $\text{Var}[\xi_{0,i}(x)] \le v_0$.
Applying Bernstein’s inequality on the auxiliary grid with mesh size $\eta_{n,0}\coloneqq n^{-1/2}h_n^{3/2}$ and using the mean value theorem with the bound $\sup_{x\in\mathcal{X}}|\partial D_{n,0}(x)/\partial x| \le 2h_n^{-2}\sup_{u\in\mathbb{R}}|K'(u)|$ gives the claim.
\end{proof}
\bibliography{ref_FINE}
\bibliographystyle{apalike}