The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
18,757 characters
Bias correction and uniform inference for the quantile density function
\maketitle
\begin{abstract}
For the kernel estimator of the quantile density function (the derivative of the quantile function), I show how to perform the boundary bias correction, establish the rate of strong uniform consistency of the bias-corrected estimator, and construct the confidence bands that are asymptotically exact uniformly over the entire domain $[0,1]$. The proposed procedures rely on the pivotality of the studentized bias-corrected estimator and known anti-concentration properties of the Gaussian approximation for its supremum.
\end{abstract}
\section{Introduction}
The derivative of the quantile function, the \emph{quantile density} (QD), has been long recognized as an important object in statistical inference.\footnote{This function is sometimes also called the \emph{sparsity function} \citep{tukey1965part}.} In particular, it arises as a factor in the asymptotically linear expansion for the quantile function \citep{bahadur1966note,kiefer1967bahadur}, and hence may be used for asymptotically valid inference on quantiles \citep{csorgo1981strong,csorgo1981two,koenker_2005}.
Given its importance, several estimators of the QD have been proposed in the literature. The most widely used estimator is the \emph{kernel quantile density} (KQD), originally developed by \citet{siddiqui1960distribution} and \citet{bloch1968simple} for the case of rectangular kernel, and generalized to arbitrary kernels by \citet{falk1986estimation}, \citet{welsh1988asymptotically}, \citet{csorgHo1991estimating}, and \cite{jones1992estimating}. This estimator is simply a smoothed derivative of the empirical quantile function, where smoothing is performed via convolution with a kernel function.
Similarly to the classical case of kernel density estimation, the KQD suffers from bias close to the boundary points $\{0,1\}$ of its domain $[0,1]$, rendering the estimator inconsistent. To the best of my knowledge, no bias correction procedures have been developed for the QD.
In this paper, I show how to perform correction for the boundary bias, recovering strong uniform consistency for the resulting bias-corrected KQD (BC-KQD) estimator. The bias correction is computationally cheap and is based on the fact that the bias of the KQD is approximately equal to the integral of the localized kernel function, a quantity that only depends on the chosen kernel and bandwidth. I also develop an algorithm for construction of the uniform confidence bands around the QD on its entire domain $[0,1]$. This procedure relies on the fact that the studentized BC-KQD exhibits an influence function that is \emph{pivotal}. This makes it possible to calculate the critical values by simulating from either the known influence function or the studentized BC-KQD under an alternative (pseudo) distribution of the data.
The rest of the paper is organized as follows. \cref{sec:KQD} outlines the framework and defines the KQD estimator. \cref{sec:BC} introduces the BC-KQD estimator and establishes its Bahadur-Kiefer expansion. \cref{sec:CB} develops the uniform confidence bands based on the BC-KQD. \cref{sec:MC} illustrates the performance of the confidence bands in a set of Monte Carlo simulations. \cref{sec:conclusion} concludes. Proofs of theoretical results are given in the Appendix.
\section{Setup and kernel quantile density estimator}\label{sec:KQD}
The data consist of independent identically distributed draws $X_1,\dots,X_n$ from a distribution on $\mathbb{R}$ with a cumulative distribution function (CDF) $F$ satisfying the following assumption.
\begin{assumption}[Data generating process]\label{ass:dgp} The distribution $F$ has compact support $[\underline x, \bar x]$ and admits a density $f=F'$ that is continuously differentiable and bounded away from zero and infinity on $[\underline x, \bar x]$.
\end{assumption}
Assumption \ref{ass:dgp} implies that the quantile density
\begin{align}
q(u) \coloneqq \frac{d F^{-1}(u)}{du} = \frac{1}{f(F^{-1}(u))}
\end{align}
is continuously differentiable and bounded away from zero and infinity on the support $[\underline x, \bar x]$.
Let $X_{(1)}\le \cdots \le X_{(n)}$ be the order statistics of the sample $X_1,\dots,X_n$, and let $\hat Q$ denote the empirical quantile function,
\begin{align}
\hat Q(u) \coloneqq
\begin{cases}
X_{(\left\lfloor{nu}\right\rfloor +1)}, \quad u\in [0,1),\\
X_{(n)}, \quad u=1,
\end{cases}
\end{align}
The KQD estimator is defined as
\begin{align}
\hat q_h(u) &\coloneqq \int_{0}^{1} K_h(u-z) \, d\hat Q(z) = \sum_{i=1}^{n-1} K_h\left(u-\frac{i}{n}\right) \left( X_{(i+1)}-X_{(i)} \right), \quad u\in [0,1],
\end{align}
where $K$ is a kernel function, $K_h(z) \coloneqq h^{-1}K\left(h^{-1}z\right)$, and $h>0$ is bandwidth \citep[see, e.g.,][]{csorgHo1991estimating}. We impose the following assumptions on the kernel and bandwidth.
\begin{assumption}[Kernel function]\label{ass:kernel}
The kernel $K$ is a nonnegative function of bounded variation that is supported on $[-1/2,1/2]$, symmetric around $0$, and satisfies
\begin{align}
\int_\mathbb{R} K(x) \, dx = 1, \quad \int_\mathbb{R} K^2(x) \, dx < \infty.
\end{align}
\end{assumption}
\begin{assumption}[Bandwidth, estimation]\label{ass:bandwidth-for-consistency}
The bandwidth $h=h_n$ is such that $h_n \to 0$ and
\begin{enumerate}
\item \label{ass:band-large-cons} $h_n^{-1} = o\left(n^{1/2} (\log n)^{-1} (\log\log n)^{-1/2}\log h^{-1} \right)$,
\item \label{ass:band-undersmooth-cons} $h_n=o\left(n^{-1/3}(\log h^{-1})^{-1/3}\right)$.
\end{enumerate}
\end{assumption}
\begin{assumption}[Bandwidth, inference]\label{ass:bandwidth}
The bandwidth $h=h_n$ is such that $h_n \to 0$ and
\begin{enumerate}
\item \label{ass:band-large} $h_n^{-1} = o\left(n^{1/2} (\log n)^{-2} (\log\log n)^{-1/2} \right)$,
\item \label{ass:band-undersmooth} $h_n=o\left(n^{-1/3}(\log n)^{-1}\right)$.
\end{enumerate}
\end{assumption}
Assumption \ref{ass:kernel} is standard; boundedness of the total variation of $K$ ensures that the class
\begin{align}
\mathcal{F} \coloneqq \left\{K \left(\frac{u-\cdot}{h}\right), \,\,u \in [0,1], h >0 \right\}
\end{align}
is a bounded VC class of measurable functions, see, e.g., \citet{nolan1987u}.
Assumptions \ref{ass:bandwidth-for-consistency} and \ref{ass:bandwidth} are essentially the same, up to the log terms in the bandwidth rates, with Assumption \ref{ass:bandwidth-for-consistency} being slightly weaker. Assumption \ref{ass:bandwidth}.\ref{ass:band-large} states that the bandwidth rate is large enough (slightly larger than $n^{-1/2}$) to guarantee that the smoothed remainder of the classical Bahadur-Kiefer expansion vanishes asymptotically, see the proof of \cref{cor:conv-rate} below. Assumption \ref{ass:bandwidth}.\ref{ass:band-undersmooth} imposes the \emph{undersmoothing} bandwidth rate (slightly smaller than $n^{-1/3}$), which ensures that the smoothing bias disappears fast enough for the confidence bands to be valid, see the proof of \cref{thm:CB} below.
\section{Bias correction and Bahadur-Kiefer expansion}\label{sec:BC}
In this section, I introduce the bias-corrected estimator and develop its asymptotically linear expansion with an explicit a.s. uniform rate of the remainder (the Bahadur-Kiefer expansion).
To see the necessity of bias correction, note that, for $u$ close to the boundary, the kernel weights $K_h(u-i/n)$, $i=1,\dots,n-1$, do not approximately sum up to one, rendering the KQD $\hat q_h(u)$ inconsistent. Therefore, dividing the KQD by the sum of the kernel weights (or the corresponding integral of the kernel function) may eliminate the boundary bias. To this end, define
\begin{align}
\psi_h(u) &\coloneqq \int_0^1 K_h(u-z) \, dz = \int_{\max(u-h/2,0)}^{\min(u+h/2,1)} K_h(u-z) \, dz, \quad u\in[0,1].
\end{align}
For computational purposes, note that $\psi_h$ is symmetric around $1/2$ (i.e. $\psi_h(u) = \psi_h(1-u)$ for all $u \in [0,1]$), $\psi_h\in[1/2,1]$ and $\psi_h(u) = 1$ for $u \in [h/2,1-h/2]$.
The bias-corrected KQD (BC-KQD) is then defined as
\begin{align}
\hat q_h^{bc}(u) \coloneqq \frac{\hat q_h(u)}{\psi_h(u)} = \frac{\sum_{i=1}^{n-1} K_h\left(u-\frac{i}{n}\right) \left( X_{(i+1)}-X_{(i)} \right)}{\int_0^1 K_h(u-z) \, dz}, \quad u\in [0,1].
\end{align}
The following theorem establishes that the studentized BC-KQD is approximately equal to the centered kernel density estimator with an approximation error that converges to zero a.s. at an explicit uniform rate. Since this result resembles (and relies on) the classical asymptotically linear expansion for the quantile function \citep{bahadur1966note,kiefer1967bahadur}, we call it the \emph{Bahadur-Kiefer expansion for the BC-KQD}. Denote $U_i=F(X_i)$, $i=1,\dots,n.$
\begin{thm}[Bahadur-Kiefer expansion for the BC-KQD]\label{Thm:BK-expansion}
Suppose Assumptions \ref{ass:dgp} and \ref{ass:kernel} are satisfied and $h_n \to 0$. Then the following representation holds uniformly in $u \in [0,1]$,
\begin{align}
Z_n^{bc}(u) = -\mathbb{G}_n(u) + O_{a.s.}\left( n^{1/2} h^{3/2} + h \log h^{-1} + h^{-1/2}n^{-1/4}(\log n)^{1/2}(\log \log n)^{1/4} \right),
\end{align}
where
\begin{align}
Z_n^{bc}(u) &\coloneqq \frac{\sqrt{nh}\left( \hat q_h^{bc}(u)-q(u) \right)}{q(u) / \psi_h(u)},\\
\mathbb{G}_n(u) &\coloneqq \frac{1}{\sqrt{nh}}\sum_{i=1}^n\left[K\left( \frac{U_i-u}{h} \right)-\mathbf{E} K\left( \frac{U_i-u}{h} \right) \right] \\
&= \sqrt{nh} \cdot \frac{1}{n}\sum_{i=1}^n \left[K_h(U_i-u) - \psi_h(u)\right].
\end{align}
\end{thm}
This representation allows us to establish the exact rate of strong uniform consistency of the BC-KQD under a bandwidth that achieves undersmoothing (Assumption \ref{ass:bandwidth-for-consistency}.\ref{ass:band-undersmooth-cons}).
\begin{cor}[Strong uniform consistency of BC-KQD]\label{cor:conv-rate}
Suppose Assumptions \ref{ass:dgp}, \ref{ass:kernel}, and \ref{ass:bandwidth-for-consistency} hold. Then
\begin{align}
\lim_{n \to \infty} \sqrt{\frac{nh_n}{2\log h_n^{-1}}} \sup_{u \in [0,1]} \left| \hat q_h^{bc}(u) - q(u) \right| = \left(\int_\mathbb{R} K^2(x) \, dx \right)^{1/2} \text{ a.s.}
\end{align}
\end{cor}
One of the convenient features of the KQD (and BC-KQD) estimator is that its bandwidth has a natural scale $[0,1]$ which is independent of the data generating process. Hence, I put aside the choice of constant $c$ in the bandwidth $h=c n^{-\eta}$ and suggest setting $c=1$.
Regarding the choice of the rate $\eta$, ignoring the log terms, it is easy to establish the rate-optimal bandwidth, which is achieved whenever the rate of the smoothing bias $n^{1/2}h^{3/2}$ matches that of the remainder in the original Bahadur-Kiefer expansion $n^{-1/4}h^{-1/2}$. It follows that the nearly-optimal bandwidth is
\begin{align}
h_n^{opt} = O\left(n^{-3/8}\right).
\end{align}
Under this bandwidth, the exact rate of strong uniform convergence is
\begin{align}
O\left(\frac{\log n}{n^{5/16}}\right),
\end{align}
which is just slightly worse than the familiar ``cube-root'' rate \citep{kim1990cube}.
\section{Uniform confidence bands}\label{sec:CB}
Suppose we had access to valid approximations $c_{n,\tau}$, $c_{n,\tau}^{abs}$ to the $\tau$-quantiles of the random variables
\begin{align}
W_n^{bc} &= \sup_{u\in[0,1]} Z_n^{bc}(u),\\
W_n^{bc,abs} &= \sup_{u\in[0,1]}\left| Z_n^{bc}(u) \right|,
\end{align}
respectively, in the sense that
\begin{align}
\mathbb{P}(W_n^{bc} \le c_{n,\tau}) &= \tau + o(1), \label{eq:valid-cv-abstract}\\
\mathbb{P}(W_n^{bc,abs} \le c_{n,\tau}^{abs}) &= \tau + o(1).
\end{align}
Then the following confidence bands for $q(\cdot)$ would be asymptotically valid at the confidence level $1-\alpha$:
\begin{enumerate}
\item the one-sided CB
\begin{align}
\left[ \frac{\hat q_h^{bc}(u)}{1 + \frac{c_{n,1-\alpha}}{\psi_h(u)\sqrt{nh}}}, \, +\infty\right), \quad u \in [0,1], \label{eq:CB-1}
\end{align}
\item the one-sided CB
\begin{align}
\left(-\infty, \, \frac{\hat q_h^{bc}(u)}{1 - \frac{c_{n,1-\alpha}}{\psi_h(u)\sqrt{nh}}} \right], \quad u \in [0,1], \label{eq:CB-2}
\end{align}
\item the two-sided CB
\begin{align}
q(u) \in \left[\frac{\hat q_h^{bc}(u)}{1 + \frac{c_{n,1-\alpha/2}^{abs}}{\psi_h(u)\sqrt{nh}}}, \,\, \frac{\hat q_h^{bc}(u)}{1 - \frac{c_{n,1-\alpha/2}^{abs}}{\psi_h(u)\sqrt{nh}}} \right], \quad u \in [0,1]. \label{eq:CB-3}
\end{align}
\end{enumerate}
I propose two ways of obtaining such approximate critical values, both making use of the pivotality of the studentized bias-corrected KQD $Z_n^{bc}(u)$, see \cref{Thm:BK-expansion}. I focus on the one-sided critical value $c_{n,\tau}$ for simplicity; the proofs for the two-sided critical value are analogous.
The first approach is to let $c_{n,\tau}$ be the $\tau$-quantile of the random variable
\begin{align}
W_n^\mathbb{G} = \sup_{u \in [0,1]} \mathbb{G}_n(u).
\end{align}
Since $\mathbb{G}_n$ is a known process, $c_{n,\tau}$ can be obtained easily by simulation. In principle, $c_{n,\tau}$ can be tabulated for different choices of the kernel $K$ and values of the sample size $n$ and the bandwidth $h$.
The other approach is to let $c_{n,\tau}$ be the $\tau$-quantile of the random variable
\begin{align}
W_n^{U[0,1]} \coloneqq \sup_{u\in[0,1]} Z_n^{bc,U[0,1]}(u),
\end{align}
where $Z_n^{bc,U[0,1]}(u)$ is equal to $Z_n^{bc}(u)$ evaluated at a pseudo-sample $\tilde X_1,\dots,\tilde X_n \sim U[0,1]$ in place of the original sample. For the uniform distribution, $q \equiv 1$, and hence
\begin{align}
Z_n^{bc,U[0,1]}(u) \coloneqq \frac{\sqrt{nh}(\tilde q_h^{bc}(u)-q(u))}{q(u)/\psi_h(u)} = \sqrt{nh}(\tilde q_h(u)-\psi_h(u)),
\end{align}
where $\tilde q_n(u)$ is the (non-bias-corrected) KQD calculated using the pseudo-sample, i.e.
\begin{align}
\tilde q_n(u) = \sum_{i=1}^{n-1} K_h\left(u-\frac{i}{n}\right) \left( \tilde X_{(i+1)}-\tilde X_{(i)} \right), \quad u\in [0,1].
\end{align}
The following theorem establishes that the two aforementioned approximations to the critical values are valid, implying the asymptotic validity of the confidence bands. These confidence bands are centered at an AMSE-suboptimal estimator $\hat q_h^{bc}$ and are expected to shrink at a rate slightly slower than the minimax optimal rate, as noted by \citet[p.1795]{chernozhukov2014anti}. This is compensated for by the confidence bands exhibiting the coverage that is asymptotically \emph{exact}.
\begin{thm}[Exactness of confidence bands]\label{thm:CB}
Suppose Assumptions \ref{ass:dgp}, \ref{ass:kernel}, and \ref{ass:bandwidth} hold. Then
\begin{align}
&\lim_{n \to \infty} \sup_{t\in\mathbb{R}}\left| \mathbb{P}\left(W_n^{bc} \le t \right) - \mathbb{P}\left(W_n^{\mathbb{G}} \le t \right) \right| = 0,\\
&\lim_{n \to \infty} \sup_{t\in\mathbb{R}}\left| \mathbb{P}\left(W_n^{bc} \le t \right) - \mathbb{P}\left(W_n^{U[0,1]} \le t \right) \right| = 0,
\end{align}
and hence the confidence bands \eqref{eq:CB-1}, \eqref{eq:CB-2}, and \eqref{eq:CB-3} are asymptotically exact.
\end{thm}
\section{Monte Carlo study}\label{sec:MC}
In this section I study the finite-sample behavior of the proposed confidence bands in a set of Monte Carlo simulations.
I consider the following distributions of the data, all supported on the interval $[0,1]$: (i) uniform[0,1] distribution (ii) the distribution $N(1/2,1)$ truncated to $[0,1]$ (iii) the linear distribution with the PDF $f(x) = x+1/2$, $x\in[0,1]$. I set the nominal confidence level to be $1-\alpha \in \{0.8,0.9,0.95,0.99\}$ and the sample size $n \in \{100,500,1000,5000\}$. The critical values are obtained by simulating $\mathbb{G}_n(u)$ and calculating the quantiles of its supremum on the grid $u \in \{0.005,0.015,0.02,\dots,0.995\}$, with the number of simulations set to $20000$ (simulation results for the critical values based on $Z_n^{bc,U[0,1]}(u)$ are very similar, so I do not report them here). I use the kernel corresponding to the standard normal distribution truncated to $[-1/2,1/2]$ and the nearly-optimal bandwidth $h=cn^{-3/8}$, where I set $c=1$ since the scale of the bandwidth is $[0,1]$, see \cref{sec:BC}.
In \cref{fig:CB}, included for illustration, I plot 100 independent realizations of the $90\%$ confidence bands for the linear distribution, along with the true quantile density (in blue). \cref{tab:coverage} contains simulated coverage values for the two-sided confidence bands. The coverage is almost invariant to the distribution of the data, but the size distortion tends to be smaller for higher nominal confidence levels.
\begin{figure}[t!]
\centering
\includegraphics[scale=0.5]{bc_kqd_power_dist_a_01_n_5000.png}
\caption{$90\%$ confidence bands for the quantile density (in blue) of the linear distribution with the PDF $f=x+0.5$, $x\in [0,1]$. Number of independent realizations of the bands $S=100$, sample size $n=5000$.}
\label{fig:CB}
\end{figure}
\begin{table}[t!]
\centering
\begin{tabular}{l c c c c}
Confidence level & $0.8$ & $0.9$ & $0.95$ & $0.99$ \\
\hline
& \multicolumn{4}{c}{Uniform distribution} \\
\hline
$n=100$& 0.891& 0.936& 0.962& 0.986 \\
$n=500$& 0.881& 0.943& 0.966& 0.990 \\
$n=1000$& 0.898& 0.947& 0.970& 0.993 \\
$n=5000$& 0.907& 0.949& 0.976& 0.996 \\
\hline
& \multicolumn{4}{c}{Linear distribution} \\
\hline
$n=100$& 0.891& 0.929& 0.956& 0.987 \\
$n=500$& 0.878& 0.936& 0.961& 0.989 \\
$n=1000$& 0.890& 0.944& 0.970& 0.991 \\
$n=5000$& 0.914& 0.949& 0.976& 0.996 \\
\hline
& \multicolumn{4}{c}{Truncated normal distribution} \\
\hline
$n=100$& 0.898& 0.942& 0.964& 0.988 \\
$n=500$& 0.887& 0.944& 0.967& 0.992 \\
$n=1000$& 0.905& 0.950& 0.972& 0.993 \\
$n=5000$& 0.911& 0.952& 0.978& 0.997 \\
\end{tabular}
\caption{Simulated coverage of the two-sided confidence bands}
\label{tab:coverage}
\end{table}
\section{Conclusion}\label{sec:conclusion}
To the best of my knowledge, no boundary bias correction or uniform inference procedures have been developed for the quantile density (sparsity) function.
In this paper, I develop such procedures, establish their validity and show in a set of Monte Carlo simulations that they perform reasonably well in finite samples. I hope that, even when the quantile density itself is not the main inference target, these results may be employed for improving the quality of inference for other statistical objects, including the quantile function.
\bibliographystyle{ecta}
\bibliography{main}
\newpage
\part*{Appendix}