Kernel-weighted specification testing under general distributions

Kernel-weighted specification testing under general distributions

Introduction

Kernel-weighted statistics are widely used for inference on the functional form of a density, conditional distribution and conditional mean. In testing for a parametric specification of a conditional mean, kernel-based tests have been used in the traditional regression context Zhang,mammen1,horowitz2001. Those types of statistics are also employed in various extensions such as regression quantiles zheng1998consistent,mammen2, semi-parametric models chen2009goodness, propensity score ShaikhVytlacil, panel data lin, non-stationary regression gao,phillips and survival analysis muller2019goodness. If $F_{X}$ represents the law of the regressors, it can always be expressed in its Lebesgue decomposition:

where $F_{X}^d$ is a discrete measure, $F_{X}^{a.c.}$ is absolutely continuous to the Lebesgue measure and $F_{X}^s$ is a singular continuous measure. While kernel-weighted statistics have been investigated in several distinct applications, not much is known about the statistical properties of procedures based on these statistics when the distribution is “contaminated” with non-trivial singular components. To the best of our knowledge, all the available analyses in the literature have assumed that the Lebesgue decomposition of $F_{X}$ does not admit any singular components, usually with additional smoothness regularity conditions imposed on the density function.

In this paper, we study a class of kernel-weighted U-statistics that frequently arise in the analysis of goodness-of-fit testing. A general limit theory is provided that allows for the Lebesgue decomposition of $F_{X}$ to contain singular components. Our generalization of the standard limit theory is partially motivated by a desire to understand the finite sample properties of these statistics when the distribution is absolutely continuous with a density that is either non-smooth or possesses large derivatives. Indeed, in finite samples, singular distributions tend to exhibit characteristics similar to such measures. For example, the claw and its variants in Marron are Gaussian mixtures with a density that exhibits sharp continuous “spikes” at several points in the support. While these distributions are absolutely continuous with a smooth density, the derivatives may be so large as to make them resemble distributions with singular components in finite samples.

As illustrated in Figure (ref), the estimated density derivative of covariates may be quite large: within the same data set, the budget shares for alcohol, travel and leisure were significantly more volatile and exhibited estimated derivatives in the thousands. While the standard limit theory can account for this phenomenon by imposing an arbitrarily large upper bound on the density and/or its derivative, we find it more appropriate to model these situations with the possibility of a non-trivial singular component.

The main results of this paper develop the limit theory for kernel-based inference statistics when the distribution of the conditioning variables admits a Lebesgue decomposition with singular components. The class of distributions that our theory covers include absolutely continuous measures with a bounded density, measures supported on lower dimensional subspaces, continuous measures that contain discrete marginals (e.g. mixed variables and discrete regressors), normalized Hausdorff measures on self-similar fractals and any Ahlfors-David regular measure. All our main results also apply to countable mixtures generated by such measures. Even in the special case where $F_{X}$ is absolutely continuous, our analysis only requires a bounded density and avoids imposing any further regularity conditions.

Denote by $\iota$ a vector of ones and the $F_{X}$ probability of a cube of radius $h > 0 $ and centred at $x$ by $F_X(x-h \iota , x +h \iota) = \mathbb{P}(\| X- x\| _{\infty} \leq h ) $. Our approach provides interpretable conditions that are based on local features of the underlying distribution through the expected small ball probability:

If $F_{X}$ is a probability measure on $\mathbb{R}^q$ with a bounded density, $r(h) \asymp h^{q}$ as $h \downarrow 0 $ but the rate is always slower when the Lebesgue decomposition admits singular components. The idea behind our results is to utilize a version of integration by parts to extract local features of $F_{X}$ that depend on $r(h)$ when the density does not exist (at the cost of using sufficiently differentiable kernels). This approach is of independent interest and may be useful in other applications that utilize kernel-based statistics.

Theorem (ref) establishes the general result for asymptotic normality of a degenerate kernel-weighted U-statistic. For some mixture subclasses that include singular distributions of reduced Hausdorff dimension $sq$ for some $s \in (0,1)$, exact rates of convergence for the U-statistic are obtained. These results form the basis for the study of self-normalizing statistics that arise in goodness-of-fit tests of a parametric regression function. In Theorem (ref), we establish the limit distribution for such statistics under the null hypothesis. In Theorem (ref), we develop the local power analysis of the test statistic under a Pitman sequence of local alternatives

where $\gamma_n \downarrow 0$ and $\delta(.)$ is a fixed drift function that determines the direction of approach to the null model. We characterize the fastest possible rate $\gamma_n$ at which alternatives can approach and yet remain distinguishable from the null. If the Lebesgue decomposition of $F_{X}$ contains singular components, we show that it is possible for the alternatives to approach the null at a rate faster than in the fully absolutely continuous case. The novel feature of the result is the interplay between the rate of approach $\gamma_n$, the direction of approach $\delta(.)$ to the null model and the singular components of the distribution. In particular, power does not exist at the fastest possible rate if the support of $\delta(.)$ does not sufficiently “touch” areas where the local singularity of the measure (the rate at which $ h \rightarrow \mathbb{P}(\| X- x\| _{\infty} \leq h ) $ decays) coincides with $r(h)$. In Theorem (ref), we provide further details on the mechanism through which this interplay can influence the local power of the test statistic.

The paper is organized as follows. Section (ref) provides the model framework and assumptions. Section (ref) develops the main limit results. Section (ref) provides simulation evidence on the sensitivity of the kernel test statistic to the distribution of the conditioning variables. Section (ref) concludes. The supplemental file sidvsup contains additional proofs and technical results that were omitted in the main text.

Framework and assumptions

Consider the nonlinear regression model

where $X\in\mathbb{R}^{q}$ is a vector of regressors, $Y$ is a scalar dependent variable and $u$ is an unobserved error. Given a family of parametric regression functions $\{ g(x,\beta) : \beta \in \Theta \}$ indexed by a finite dimensional parameter $\beta \in \Theta \subseteq \mathbb{R}^p$, we address the common problem of testing the null hypothesis $$ H_0 : \mathbb{P}( \bar{g}(X) = g(X,\beta_0)) = 1 \; \; \; \text{for some} \;\; \beta_0 \in \Theta .$$ Denote by $\hat{u}$ the vector of estimated residuals $\hat{u} _{i}=Y_{i}- g(X_i,\hat{\beta})$ (e.g. using non-linear least squares or maximum likelihood). To test $H_0$, we make use of the kernel smoothed statistic

where $K(.)$ is a non-negative symmetric kernel function and $h_n \downarrow 0$ is a deterministic bandwidth sequence. The usual goodness-of-fit statistic (see e.g. gao,phillips,Zhang) uses a self-normalized form

where $ n^{-2} \hat{\sigma}_n^2$ is an estimator of the variance of $ \hat{I}_n$. In the literature, this test statistic falls under the class of smoothing-based tests (see gonz2013 for a comprehensive review). Let $U_n$ denote the version of $\hat{I}_n$ obtained by replacing $\hat{u}_i$ with the unobserved error:

Notation

Denote by $\iota$ a vector of ones and $c \iota$ a vector of $c \in \mathbb{R}$. For any vector $x \in \mathbb{R}^q$, denote the coordinate components by $x=(x_1,\dots,x_q)$. Let $\| .\|_{2}, \|. \|_{\infty} $ and $\| . \|_{\text{op}}$ denote the Euclidean, infinity and operator norm, respectively. For positive sequences $(a_n,b_n)$, we use $a_n \lessapprox b_n$ to denote $\limsup_{n \rightarrow \infty} a_n / b_n < \infty$ and $a_n \asymp b_n$ to denote $a_n \lessapprox b_n \lessapprox a_n$. Let $L^p(X)$ denote the usual equivalence class of $p$ integrable (with respect to $F_X$) functions that are measurable with respect to the $\sigma$ algebra generated by $X$. Denote the absolute continuity of a measure $\nu$ with respect to a measure $\lambda$ by $\nu \ll \lambda$. Let $\mathbb{E}$ and $\mathbb{P}$ denote the usual expectation and probability operators. In the special case where $F_{X}$ can be expressed as a mixture that includes component $F_{t}$, the notation $\underset{X \stackrel{}{\sim} F_{t} }{\mathbb{P}} $ and $\underset{X \stackrel{}{\sim} F_{t} }{\mathbb{E}}$ will be used to indicate that the operators are defined with respect to $X\sim F_{t}$. We use $\underset{d}{\rightarrow}$ to denote convergence in distribution.

Assumptions

Assumption (ref) could be generalized but is made here to facilitate the focus on the distribution of the conditioning variables. Assumptions (ref) and (ref)(i) imply that the statistic $U_{n}$ in $\left( \ref{U(u)}\right) $ is degenerate. Assumptions (ref)(ii) and (ref)(iii) are made for convenience and could be weakened further. Of course, Assumption (ref)(ii-iii) applies under homoscedasticity.

Assumptions (ref)(i-iii) are standard and satisfied by e.g. the Epanechnikov kernel $k_{E}(t) = \frac{3}{4} (1-t^2) \mathbbm{1} \{ |t| \leq 1 \} $ and Quartic kernel $ k_{Q}(t) = \frac{15}{16} (1-t^2)^2 \mathbbm{1} \{ |t| \leq 1 \} $. The support assumption on $k(\,\cdot\,)$ could be modified to allow for any compact interval without changing any of our main results. This could be generalized even further to admit a wider class of kernel functions (e.g. Gaussian kernels) where the assumption of compact support is replaced with a rate of decay. However, such an analysis will typically involve some interplay between the decay rate of the kernel and the tails of the distribution. Here, we simplify to highlight the impact of the distribution. Assumption (ref)(iii) is satisfied if $k'(\,\cdot\,), \: k''(\,\cdot\,)$ exist and are uniformly continuous on $ \Delta^\mathrm{o} = (-1,1)$. It ensures that in the event a density does not exist, one can use (as we explain below) integration by parts to find the local behavior of moments of the U-statistic.

Main results

In Section (ref), we provide results about the moments of kernel smoothed statistics. In Section (ref) and (ref), a class of distributions is defined over which asymptotic normality for the U-statistics in ((ref), (ref)) is subsequently established. Section (ref) develops the limit theory and local power analysis within the context of specification testing.

Derivations and bounds for moments

From the seminal work of Hall Hall84, hallhyde, it is known that limit theory for $U_n$ ((ref)) can be established by appealing to a version of the martingale central limit theorem. Indeed, by defining

it is shown in Hall84 that $ n U_n / \sqrt{2 \mathbb{E}(H_n^2)}\underset{d}{\rightarrow} N(0,1) $, provided that the moments satisfy

As noted in the literature (e.g. koltch), Condition ((ref)) (or its variants in other applications) is typically difficult to interpret as it depends non-trivially on the underlying distribution of the regressors. It is shown in Zhang (see also mammen1,mammen2,ShaikhVytlacil for related applications) that when the distribution is absolutely continuous and certain smoothness regularity conditions hold on the density, Condition ((ref)) reduces to the usual restriction on the bandwidth: $h_n \downarrow 0$ and $n h_n^q \uparrow \infty$.

Our starting point in generalizing beyond absolutely continuous measures is to derive the distributional restrictions that are implicitly imposed through Condition ((ref)). This requires us to express and bound the moments that appear in ((ref), (ref)) in terms of interpretable functionals of $F_{X}$. This will be the subject of several subsequent Lemmas that appear below.

Given $f \in L^1(X)$, a straightforward application of Fubini's theorem shows that integration (with respect to $f dF_{X} $) of a compactly supported sufficiently differentiable function admits a representation as a standard integral with respect to the Lebesgue measure. The integrand in this case is the $\partial_{t} g(.)$-weighted $f dF_{X} $ measure of a Euclidean cube.

The next Lemma aims to interpret the moments appearing in Condition ((ref)) through repeated applications of Lemma (ref). We begin by introducing some convenient notation. Let $\mu_{2}\left( t\right) ,\mu_{4}\left( t\right) $ be as in Assumption (ref). Given $x,s,t \in \mathbb{R}_{}^q$, we define the cube centered at $x$ with directions $(s,t)$ to be \[ B\left( x-s,x+t\right) =\left\{ y \in \mathbb{R}^q :x_{i}-s_{i}\leq y_{i}\leq x_{i}+t_{i} \; \: \forall \; i=1,...,q\right\} . \] Define

The following Lemma expresses the moments in terms of functionals of $\Omega_{l}$.

Given the form of the integrand that defines $\mathbb{E}(H_n^2)$ in Lemma (ref), it is expected (by Lebesgue's differentiation theorem) that $\mathbb{E}(H_n^2) \asymp h_n^q$ whenever $F_{X} \ll $ Lebesgue measure. If the Lebesgue decomposition of $F_{X}$ admits singular components, the following Corollary shows that $h_n^q$ may at least be used as a conservative lower bound on the rate.

In particular, Corollary (ref) extends the standard result (see e.g. Zhang) for the limiting behavior of $\mathbb{E}(H_n^2)$ when the Lebesgue density exists (although here we do not assume that it is continuous) and demonstrates that $h_n^{-q} \mathbb{E}(H_n^2)$ diverges when there are singular components. The next Lemma provides bounds on the moments that will be instrumental in verifying Condition ((ref)) and as a consequence the limit behavior of the U-statistic in $\left( \ref{U(u)}\right)$.

The bounds on the moments derived in this section are in terms of the expected small ball probability $\mathbb{E} \big[ F_{X} (X -h \iota , X +h \iota) \big]$. We next turn to defining classes of distributions where these bounds can be used to provide limit properties of the statistic.

Classes of distributions

We begin by making an assumption that delineates a class of distributions for which the asymptotic normality of the kernel statistic will be established. As we show below, this class encompasses some well-known distributions. We will refer to $F_{X}$ as a continuous measure if $ F_X(x - h \iota , x + h \iota) = \mathbb{P} \big( \| X - x \|_{\infty} \leq h \big) \downarrow 0 $ as $h \downarrow 0$ for every $x $ in the support of $F_{X}$.

A stronger pointwise version of Assumption (ref)(i) is commonly known as the “doubling” condition in the literature (see e.g. dbl1,ussr). Specifically, doubling measures are exactly those that satisfy

for some universal constant $ C <\infty $ and all sufficiently small $h > 0$.

A stronger condition which implies both parts of Assumption (ref) is that $F_{X}$ be Ahlfors-David regular (see e.g. ahl1) on the support of $F_X$. These are precisely the measures where there exists a $s \in (0,1] $ for which

holds for some universal constants $C,D < \infty $ and all sufficiently small $h > 0 $.

We note that while conditions ((ref), (ref)) are sufficient for Assumption (ref) to hold, they are not necessary. In particular, the existence of a universal $C > 0 $ that satisfies ((ref), (ref)) may be excessively restrictive when $X$ is not compactly supported. Nonetheless, Assumption (ref) may still hold in this case as the assumption only depends on expectations of the distribution. To expand on this point, we introduce a rich class of distributions that extends beyond the absolutely continuous and Ahlfors-David regular subclass. Define $$ \underline{F_{X}} (x,s) = \liminf_{h \downarrow 0} \frac{F_X(x - h \iota , x + h \iota)}{(2h)^{sq}} . $$

The following four examples demonstrate that the class $\mathcal{D}$ includes a wide range of distributions that may underlie various cases of interest in economics, finance and natural sciences.

In particular, Example (ref) allows for absolutely continuous measures that admit a discontinuous density function.

Fractal measures are useful for modelling data that describe processes that are similar at different scales, used frequently in the natural sciences and the analysis of spatial data (see e.g. burrough1981fractal,davies1999fractal).

Example (ref) can be generalized further to allow for a general $r$-rectifiable (see e.g. preiss,de2006lecture,moore1950density) measure.\footnote{A measure $\nu(.)$ on $\mathbb{R}^q$ is $r$-rectifiable if there exists a Borel measurable function $f(.)$ and a countable collection of $r$-dimensional $C^1$ submanifolds $\{M_i \}_{i=1}^{\infty}$ such that $ \nu(A) = \sum_{i=1}^{\infty} \int _{M_i \cap A} f(x) d \text{Vol}^r(x)$ for every Borel set $A$, where $\text{Vol}^r$ is the natural $r$-dimensional volume measure that a $C^1$ submanifold inherits as a subset of $\mathbb{R}^q$.} These are low dimensional measures in the sense that there exists a countable collection of $r$-dimensional $C^1$ submanifolds $\{ M_i \}_{i=1}^{\infty}$ such that $ \mathbb{P}\big( X \in \bigcup_{i=1}^{\infty} M_i \big) = 1$.

In Example (ref), $Z$ may be fully discrete (the support of $Z$ is a countably infinite set $\mathcal{Z} \subset \mathbb{R}^{q-r}$) or $Z = (Z_1,Z_2)$ where $Z_1$ is fully discrete and $Z_2$ is a mixture of a continuous and discrete variable. Example (ref) can be generalized in a straightforward way to allow for the continuous measure $F_{W}$ to contain singular components (the argument is identical if we insist that $ F_{W} $ and $W|Z=z^{\star}$ are elements of $\mathcal{D}(t)$ for some $t \in (0,1]$). To the best of our knowledge, Example (ref) extends the known results in the literature (e.g. hsiao2007) to allow for discrete regressors with countably infinite support, mixed regressors and continuous regressors whose joint law may have singular components . In this case, our approach also highlights that there are possible advantages to viewing the joint distribution of mixed data containing discrete and continuous variables as a continuous singular measure (which can be analyzed directly).

The next result shows that every $ F_X \in \mathcal{D} $ satisfies Assumption (ref). Moreover, $\mathcal{D}$ is closed under mixtures, so that any mixture combination of its elements (such as the distributions in Example (ref)-(ref)) satisfies Assumption (ref) as well.

In particular, by mixing absolutely continuous measures with elements of $ \bigcup_{s \in (0,1)} \mathcal{D}(s)$, we obtain a large class of distributions that admit a non-trivial Lebesgue decomposition and satisfy Assumption (ref). Finally, we remark that it is not necessary to argue for Assumption (ref) through membership in $\mathcal{D}$. In more complicated setups, a measure may not charge cubes in a way that satisfies Definition (ref). In such cases, Assumption (ref) should be verified directly.

Limit theory

In the next result, we establish asymptotic normality for the U-statistic $U_{n}$ in ((ref)) for every distribution $F_X $ that satisfies Assumption (ref). For the remainder of Section (ref), we assume that the bandwidth sequence $(h_n)_{n=1}^{\infty}$ satisfies $h_n \downarrow 0$ and $n h_n^q \uparrow \infty$.

Specification Testing

Consider the feasible statistic $\hat{I}_{n}$ in ((ref)) that differs from $U_{n}$ in that the residuals $\hat{u}_{i} = Y_i - g(X_i, \hat{\beta})$ replace the true errors $u_{i}$. To construct the goodness-of-fit test statistic $\hat{\tau}_{n}$ in ((ref)), we will also require a feasible analog of $2 \mathbb{E}(H_n^2)$. To that end, define

Let $\nabla_{\beta} g(x, \beta)$ and $\nabla_{\beta}^2 g(x, \beta)$ denote the Gradient and Hessian of $g(x,\beta)$ (with respect to $\beta$), respectively. To derive the limit theory under $H_0$, we impose the following regularity conditions on the model and parameter estimator.

Assumption (ref) is commonly imposed in the literature (e.g. Zhang, phillips). For distributions in $\mathcal{D}$, all our results in this section also hold under the weaker requirement that $G(X) \in L^1(X)$. The theorem below establishes the limiting distribution for the self-normalized goodness-of-fit statistic $\hat{\tau}_{n}$ in ((ref)) under $H_0$.

Thus, the test statistic converges weakly to a standard Gaussian and does not depend on any nuisance parameters. When singular components exist, the rate at which the distribution of the statistic approaches the limit Gaussian could be quite slow, in particular for $\mathbb{E}(H_n^2)$ converging to zero slowly (e.g. for small $s$ in $\mathcal{D}\left( s\right) )$. If knowledge of the singular components is assumed, one can in principle choose a bandwidth sequence that approaches zero at a faster rate than usual (see Remark (ref)) to improve on the rates.

To investigate the asymptotic power of the test, we consider the sequence of local alternative models

where $\gamma_n \downarrow 0$ is a deterministic sequence of constants and $\delta(.) $ is a real-valued drift function that determines the direction of approach to the null model.

We will continue to assume (as is standard) that Assumption (ref)(i) holds under $H_1$. Indeed, when the residuals are computed using non-linear least squares (NLS) and the usual regularity conditions to ensure consistency under $H_0$ hold, the estimator continues to admit (under $H_1$) the asymptotic linear expansion

To facilitate the derivation we make an assumption on the moments additional to Assumption (ref).

This assumption is similar to Assumption (ref)(ii). By arguing as in Lemma (ref), it is straightforward to deduce that Assumption (ref) holds for every distribution in the class $\mathcal{D}$.

The next theorem provides the local power analysis of the specification test. In general, it depends on the interplay between $(i)$ the rate $\gamma_n$ at which the local alternatives approach the null, $(ii)$ the drift function $\delta(.)$ that determines the direction of approach to the null model and $(iii)$ the distribution $F_{X}$.

Thus, under a sequence of local alternatives, the fastest rate at which asymptotic power may exist is $ n^{-1/2} \{ \mathbb{E}[ F_{X}(X - h_n \iota , X + h_n \iota) ] \}^{-1/4} $. Clearly ((ref)) is satisfied whenever $\delta^2(X)$ is bounded away from zero (up to a $F_{X}$ null set). If $F_{X}$ is absolutely continuous with density $f_X \in L^{\infty}(X)$, we have $\mathbb{E}[ F_X(X -h \iota , X + h \iota) ] \asymp h^q$ and ((ref)) reduces (by Lebesgue differentiation) to the usual condition $$ \lim_{h \downarrow 0} \frac{ \mathbb{E} \big[ \delta^2(X) F_X(X - h \iota , X +h \iota) \big]}{(2h)^q} = \mathbb{E}[\delta^2(X) f_X(X)] > 0 . $$

The implications of Theorem (ref) are more complex for distributions with singular components when $\delta^2(X)$ is not bounded away from zero. Loosely speaking, ((ref)) says that the support of $\delta(.)$ must intersect nontrivially with a subset of the support of the distribution where the local singularity is maximized \big(support points $x$ where the decay rate of $ F_{X}(x - h_n \iota , x + h_n \iota)$ coincides with the rate for $\mathbb{E}[ F_{X}(X - h_n \iota , X + h_n \iota)]$ \big). The following example illustrates the essential idea of how the direction of approach may influence the local power properties of the test statistic.

We now examine the situation where ((ref)) holds. In this case, there is no power for alternatives that approach the null model in the $\delta(.)$ direction with rate

A natural question then is what is the minimum rate $\gamma_n > \gamma_n^*$ for which power exists? In general $\gamma_n$ depends non-trivially on the interaction between the support of $\delta(.)$ and the local singularity of the measure. To illustrate the essential idea, we consider the case where $F_{X} $ can be represented as a finite mixture of distributions from $\mathcal{D}$. That is, there exists a finite index set $T$ such that

In this case, $\mathbb{E}\big[ F_X(X-h_n \iota, X+ h_n \iota) \big] \asymp h_n^{ sq} $ where $s = \min_{t \in T} s_t$. We consider the case where $\delta(.)$ concentrates away from the support of the components $\{ F_t : s_t =s \}$ or equivalently, the support of $\delta(.)$ does not interact with areas where the local singularity of the measure is at its maximum.

Given a distribution $F_{t}$, its support can always be expressed as

Let $\mathcal{S}_{\delta} = \{ x \in \mathbb{R}^q : \delta(x) \neq 0 \}$ and denote the common support of $(F_t,\delta)$ by $ \mathcal{S}_{t,\delta} = \mathcal{S}_{t} \cap \mathcal{S}_{\delta} $. For the mixture distribution in ((ref)), denote by $R = R(\delta)$ a subset of the index set $T$ for which

holds for all sufficiently small $h > 0 $. Condition ((ref)) restricts the support of $\delta(.)$ to lie outside of $ \bigcup_{t \in T \setminus R} \mathcal{S}_t $. Condition ((ref)) essentially states that $ \bigcup_{t \in R} \mathcal{S}_{t, \delta} $ is well-separated from $\mathcal{S}_z$ for every $z \in T \setminus R$ that has a sharper singularity than that of $R$. In particular, if there exists $\epsilon > 0 $ such that $ \inf_{x \in \mathcal{S}_{t , \delta} \, , \, y \in \mathcal{S}_z } \| x-y \|_{\infty} \geq \epsilon $ for every $t \in R$ and $z \in T \setminus R$, then ((ref)) holds for every $h < \epsilon$ (for a one dimensional illustration, see Example (ref)).

Denote the restricted $\delta$-singularity coefficient of $F_{X}$ by $s_{\delta} = \min_{t \in R} s_{t}$ and the probability measure induced from the restriction by $F_{R} = (\sum_{t \in R } \alpha_t)^{-1} \sum_{t \in R } \alpha_t F_t $. The following theorem illustrates that for directions $\delta(.)$ that satisfy ((ref)-(ref)) with no power at rate $\gamma_n^*$, there may be power when the alternatives approach at a slower rate $\gamma_n > \gamma_n^*$, provided that the support of $\delta(.)$ intersects nontrivially with some subset of the mixture components support.

In the special case where $ R = T $, we have $s_{\delta} = \min_{t \in T} s_t = s $. By Lemma (ref), $F_{X} \in \mathcal{D}(s)$ and $ \mathbb{E}[ F_X(X - h_n \iota , X + h_n \iota) ] \asymp h_n^{sq}$. Therefore, the hypothesis and conclusion of Theorem (ref) is identical to Theorem (ref) in this setting. Moreover, when $F_{X} \in \mathcal{D}(s)$, boundedness of $\delta(.)$ can be replaced with $L^2(X)$ integrability and the existence of sufficient moments for the envelope function $M(.)$ appearing in Assumption (ref). Other variations on these assumptions are possible as well.

When $ R \subsetneqq T $ and $s_{\delta} > s $, Condition ((ref)) of Theorem (ref) will fail as the ratio has rate $ h^{q(s_{\delta}-s)} $. Condition ((ref)) corrects for this by using only the mixture components where $\delta(.)$ has support and the appropriate rate $ \mathbb{E}_{X \sim F_R} \big[ F_R(X - h \iota , X + h \iota) \big] \asymp h_n^{s_{\delta}q}$ for the denominator. If ((ref)) holds, there will be power for alternatives that approach in the $\delta(.)$ direction at rate $$ \gamma_n = n^{-1/2} \{ \mathbb{E}[ F_X(X - h_n \iota , X + h_n \iota) ] \}^{1/4} h_n^{-s_{\delta}q/2} \asymp \gamma_n^* h_n^{-(s_{\delta}-s)q/2} > \gamma_n^* .$$ As an application, let $\delta(.)$ be as in Example (ref). We take $R$ to be the subcomponent that contains only the $N(0,1)$ distribution. Then ((ref)) holds whenever $ \mathbb{E} _{X \sim N(0,1)}[ \delta^2(X) ] > 0 $. In this case, Theorem (ref) implies that power exists at the rate $$ \gamma_n = n^{-1/2} h_n^{- \log_{3}(2) /4} h_n^{-( 1 - \log_{3}(2) )/2} \approx n^{-1/2} h_n^{-0.3423} .$$ Thus, under the null, the goodness-of-fit statistic $\hat{\tau}_n$ in ((ref)) converges weakly to a standard Gaussian for a large class of continuous measures. However, the usual local power analysis is complicated by possible singularities and their interplay with the direction of approach to the null model.

Simulations

In this section, we provide simulation evidence on the asymptotic normality of the goodness-of-fit test statistic under the null hypothesis. Additionally, we illustrate the sensitivity of the test to the distribution of the conditioning variables. We use the Epanechnikov kernel and the number of replications in all cases is $5000$ (population quantities defined through $\mathbb{P}$ use the empirical analog from the simulation draws).

In the interest of working with a distribution that admits a non-trivial singular component, we make use of the fact that there exists a well developed theory (see e.g. ergodic) for approximating a random sample from the normalized Hausdorff measure of any self-similar fractal. We focus on a two dimensional regressor with mixture distribution

where $F_{U}$ is the uniform distribution on $[-2,2]^2$ and $F_{S}$ is the $\log_{2}(3) \approx 1.585 $ normalized Hausdorff measure on $T=$ Sierpi\'{n}ski's Triangle (with vertices at $\{ (-2,0),(2,0),(0,2) \}$).

A random sample from $F_{S}$ can be approximated using the Markov chain generated by the chaos game (see e.g. ergodic) on the iterated function system (the family of contraction maps in Example (ref)) associated with $T$. In all cases, the initial $100$ draws are discarded as a Markov chain burn-in and the remaining draws are used as the observed sample of size $n$.

Denote the coordinates of $X$ by $X=(V_1,V_2)$. We consider the null model

The data is generated with $(\beta_0,\beta_1,\beta_2) = (1,1,1)$. We use the feasible goodness-of-fit statistic $ \hat{\tau}_n = n \hat{I}_n / \sqrt{ \hat{\sigma}_n^2} $ where $ \hat{I}_n $ and $ \hat{\sigma}_n^2$ are as in ((ref)) and ((ref)), respectively. The estimated residuals $\hat{u}_i$ are computed using ordinary least squares on the regression in ((ref)).

Figure (ref) illustrates the approximate normality of the test statistic under the null. As noted in the literature (e.g. mammen1), even with absolutely continuous regressors, the finite sample distribution of the test statistic typically places more mass on the negative axis. In particular, a test computed with the usual one-tailed Gaussian critical value may be slightly undersized in finite samples.

Next, we examine the sensitivity of the test statistic’s power as the level of singular contamination increases. This is incorporated into the data generating process through the mixture coefficient $\alpha_2$ that appears in ((ref)). We consider alternative models

As Tables (ref) and (ref) illustrate, the power exhibits a relevant dependence on the bandwidth. This is true even in the fully absolutely continuous case (see e.g. Table 2-4 in Zhang). Denote by $\delta(.)$, the trigonometric functions appearing in the alternative models $(i)$ and $(ii)$ above. From expanding $\hat{I}_n$ (as in the proof of Theorem (ref)), the term that provides the positive bias under the alternative hypothesis is given by

Intuitively, as $h_n \rightarrow 0$, the terms in this sum are nonzero only when $\| X_i - X_j \|_{\infty} $ is small and by uniform continuity this implies $\delta(X_i) \delta(X_j) \approx \delta^2(X_i)$.

At all bandwidth levels, the test exhibits higher power as the weight on the singular component increases. The expected small ball probability increases as the weight on the singular component increases. The interpretation of this in finite samples is that, given any observation $X_i$, there is a higher frequency of $X_j$ for which $\| X_i - X_j \|_{\infty} < h_n $ (and hence $K(h_n^{-1} [ X_i - X_j ] ) \neq 0 $) as the weight on the singular component increases. A larger bandwidth has a similar effect on the small ball probability, although it also influences the statistic through other factors, such as its interaction with the uniform continuity of $\delta(.)$.

Finally, we note that the situation in small to moderate samples also depends on the choice of $\delta(.)$ used to construct the alternative. The trigonometric functions have comparable magnitude everywhere, thereby allowing us to focus more closely on the effects of singular contamination. By contrast, using a drift function $\delta(.)$ that is large in magnitude away from the support of the singular component could result in a power loss as contamination increases (although, by Remark (ref), this effect is expected to vanish in large samples, provided that the support of $\delta(.)$ intersects the support of the singular component).

Discussion

This paper develops the limit theory for a class of kernel-weighted statistics when the underlying distribution of the conditioning variables admits a non-trivial Lebesgue decomposition. The limit theory for these statistics centers around the behavior of the expected small ball probability. Under the null, the usual kernel smoothed goodness-of-fit statistic converges weakly to a standard Gaussian for a large class of continuous measures. However, in contrast to the absolutely continuous case, the usual local power analysis of these statistics depends non-trivially on the direction of approach to the null model. We expect that our analysis has similar implications for the more complicated setups that make use of kernel smoothed statistics.

The results could be extended in future work in a number of directions. In this paper, we focus primarily on the goodness-of-fit statistic proposed in Zhang. The results could be extended to other applications (e.g. phillips,gao,ShaikhVytlacil,lin,zheng1998consistent) that make use of an identical form of the statistic. As discussed in gonz2013, this statistic is motivated by the moment condition $ \mathbb{E} \big( u \mathbb{E}(u|X) f_X(X) \big) = 0 $. By contrast, the kernel-based goodness-of-fit statistics in mammen1 and dette1999consistent are motivated by the moment conditions $\mathbb{E} \big( \mathbb{E}^2[ u |X] \big) = 0$ and $ \mathbb{E} \big[ u^2 - \big( u - \mathbb{E}[u|X] \big)^2 \big] = 0$, respectively. In all cases, the test statistic is asymptotically equivalent to a quadratic form, and so an appropriately debiased version of the statistic has similar asymptotics to that of a degenerate U-statistic. We expect that our analysis could be extended to these cases. A more ambitious avenue would be to expand the investigation to accommodate the more complicated setups beyond goodness-of-fit testing that make use of kernel smoothed statistics.

Acknowledgments

The authors are grateful to Donald Andrews, Xiaohong Chen, Yuichi Kitamura, Renaud Raqu\'epas, Michael R Sullivan and Edward Vytlacil for their suggestions and constructive comments.

\nocite{fanLi} \nocite{gao2008} \nocite{meilan2020goodness} \nocite{santana} \nocite{sen2014} \nocite{shah2018} \nocite{verzelen2010} \nocite{dette1999consistent} \nocite{Bierens2}

\setcounter{equation}{0} \setcounter{figure}{0} \setcounter{table}{0} \setcounter{page}{1} \makeatletter

This supplemental file contains additional proofs and technical details omitted in the main text. For convenience, we first list some of the notation that was introduced in the main text and is frequently encountered in the proofs. Section (ref) details a number of technical auxiliary lemmas that are used for the proofs of the main results. Section (ref) contains the proofs of the statements appearing in the main text.

Given $x,s,t \in \mathbb{R}_{}^q$, we define the cube centered at $x$ with directions $(s,t)$ to be \[ B\left( x-s,x+t\right) =\left\{ y \in \mathbb{R}^q :x_{i}-s_{i}\leq y_{i}\leq x_{i}+t_{i} \; \: \forall \; i=1,...,q\right\} . \] Given any $f(X) \in L^1(X)$, define

If $\mu_{2}(X) $ and $ \mu_{4}(X)$ are defined as in Assumption $2(ii)$, we denote the special case of $f(X) = \mu_{2}(X), \mu_{4}(X)$ by $\Omega_{2}(.)$ and $\Omega_{4}(.)$, respectively. Additionally, we denote the case $f(X)=1$ by

Given a sufficiently differentiable function $g$ (in the sense of Definition $3.1$), we define $$ \partial_{x} g(x) = \frac{\partial^q g(x)}{ \partial x_{1} \dots \partial x_{q}} .$$

The sequence of local alternatives used in the local power analysis is denoted by

The support of $\delta(.)$ is denoted by $\mathcal{S}_{\delta} = \{ x \in \mathbb{R}^q : \delta(x) \neq 0 \}$. When $F_{X}$ can be expressed as a mixture that includes component $F_{t}$, the notation $\underset{X \stackrel{}{\sim} F_{t} }{\mathbb{P}} $ and $\underset{X \stackrel{}{\sim} F_{t} }{\mathbb{E}}$ is used to indicate that the operators are defined with respect to $X\sim F_{t}$.

Auxiliary Lemmas

Proofs

Lemma 3.2

Lemma 3.3

Corollary 3.4

Lemma 3.5

Lemma 3.7

Theorem 3.9

Theorem 3.10

Theorem 3.11