Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
33,822 characters · 7 sections · 37 citation commands
Flatness-Robust Critical Bandwidth
\global\long \global\long \global\long \global\long \global\long \global\long
Critical bandwidth (CB), introduced by silverman_unimodal, is used to test the multimodality\footnote{When discussing the number of modes (equivalently, “peaks”), I am referring to the weak concept. For example, when discussing monotonicity I am referring to weak monotonicity; and I refer to a function $f$ as unimodal if it is weakly unimodal, i.e., there exists a value $m$ for which $f$ is monotonically increasing for $x\le m$ and monotonically decreasing for $x\ge m$.} of densities and regression functions, to detect mixture distributions, and as a component of clustering methods. CB is discussed in monographs on the bootstrap as an innovative way to enforce the null hypothesis,\footnote{See, e.g., Section 16.5, “Testing multimodality of a population,” of efron_bootstrap; hallbootstrap; and davison1997bootstrap.} and has applications across many fields. Examples of the diverse applications of CB include identifying the number of growth spurts in height harezlak_heckman, exploring the multimodality of labor productivity henderson2008_parmeter, identifying determinants of the U-shape of life satisfaction over the life cycle kostyshak_2017, and testing the ecological niche separation of species with similar dietary requirements cumming2017_niche_separation.\footnote{Although CB can be applied to multi-dimensional functions of interest, in this paper I focus on univariate functions because they are the most common applications of CB.}
Despite the variety of null hypotheses that CB tests are used for, the robustness of results based on CB tests has been limited because they are generally inconsistent if the true function of interest is constant (“flat”) over an interval. With finite sample sizes, this flatness problem can arise in additional situations, even when the true regression function or density does not contain a perfectly flat region: If the derivative is close to zero in absolute value, the CB test can suffer from low power and incorrect size, as shown in the simulations in Section (ref). Although flatness exclusion is a reasonable assumption in some situations, many real-world relationships have regions over which the function of interest is constant or has a small derivative. For example, the height of humans is increasing in the early ages, flattens out, and then eventually decreases at older ages.
Flatness exclusion was assumed in the original proof of CB consistency silverman1983,\footnote{silverman1983 assumes bounded support of the density, and that the derivative has “no multiple zeros.”} and has been noted by mammen1992, cheng1998, and hallHeckman_mono_dip.\footnote{The widespread recognition of the inconsistency is reflected by the multiple terms used to refer to it: the “spurious mode problem,” the “flatness problem,” and the “boundary problem.”} Further, new techniques that propose improvements to standard CB also assume flatness exclusion (e.g., ameijeiras2016_multimode). With no solution available, the flatness problem has motivated the development of non-CB methods that do not suffer from the same inconsistency (e.g., cheng1998, hallHeckman_mono_dip, and gijbels2000). Such methods are solutions for specific situations in which CB has been applied (e.g., regression monotonicity), but no solution has been proposed that applies to all situations for which the flexible CB framework can be used.
This paper proposes a flatness-robust critical bandwidth (FRCB) test that is valid without the assumption of flatness exclusion. Since flat regions do not contradict the null hypothesis about the number of peaks or valleys of the function of interest, a consistent test can be constructed by identifying and excluding such regions. Formally, consider a parameter space of functions $\Theta$. Let $\hat{f}_{f}$ be a semi-parametric estimator of $f\in\Theta$, where the subscript emphasizes that the distribution of the estimator depends on the true parameter $f$. Suppose that $\hat{f}_{f}$ is consistent for all $f\in\Theta$. That is, suppose that for the domain of interest $\mathcal{X}\subset\mathbb{R}$, $\sup_{x\in\mathcal{X}}\left|\hat{f}_{f}(x)-f(x)\right|\overset{p}{\rightarrow}0$. Let the pair $(\Theta^{N},\Theta^{A})$ partition $\Theta$ into the sets corresponding to the null and alternative hypotheses. Consider the subspace $\Theta^{F}\subset\Theta$ of functions with flat regions that has non-null intersections with both $\Theta^{N}$ and $\Theta^{A}$.\footnote{For examples of elements in the null and alternative sets with flat or near-flat regions, see Figure \vref{reg_fns_graph}.} Standard CB techniques exclude $\Theta^{F}$ from the parameter space by assumption. That is, even if for all $f\in\Theta^{F}$, $\hat{f}_{f}$ converges in probability to $f$, standard CB tests are still not consistent for this parameter subspace. In this paper, I allow $\Theta^{F}$ to be part of the parameter space and provide a data-driven transformation $\mathbf{T}$ such that for all $f\in\Theta^{N}$, $\mathbf{T}\hat{f}_{f}\overset{p}{\rightarrow}\tilde{f}_{f}$ for some $\tilde{f}_{f}\in\Theta^{N}\setminus\Theta^{F}$; and (for the same $\mathbf{T}$) for all $f\in\Theta^{A}$, $\mathbf{T}\hat{f}_{f}\overset{p}{\rightarrow}\tilde{f}_{f}$ for some $\tilde{f}_{f}\in\Theta^{A}\setminus\Theta^{F}$. In other words, for any element in $\Theta$, after the transformation standard CB tests are asymptotically valid, since the probability limit of $\mathbf{T}\hat{f}_{f}$ is not in $\Theta^{F}$. the transformation mechanism thus exploits that it is unnecessary, and in fact undesired (because of the flatness problem), that $\mathbf{T}\hat{f}_{f}\overset{p}{\rightarrow}f_{f}$ for the class of multimodal hypothesis tests.
The transformation discussed above, $\mathbf{T}$, relies on a uniform confidence band for $f'$ over $\mathcal{X}$ that essentially filters out potentially flat regions as a first step before applying standard CB. Asymptotically, as the confidence band shrinks, non-flat regions are included while flat regions are not. Recent work in statistics has developed results for confidence bands for $f'$ in different settings. For non-parametric regression, belloni2015_uniform provides results for uniform confidence bands of linear functionals (which include derivatives) of the regression function. Similarly, chen2018optimal_uniform provides results for non-parametric instrumental variables regression. Any uniform confidence band can be improved by applying the post-estimation procedures in chen2021shape. For example, a researcher may use FRCB to test monotonicity of a regression function, and could gain efficiency by imposing convexity. FRCB thus builds on the long-standing CB literature, and the more recent literature on uniform confidence bands of derivatives, to provide a test that removes a previously-needed assumption.
The rest of the paper is organized as follows. Section (ref) reviews the standard CB test and the flatness problem, which causes inconsistency, low power, and incorrect size. Section (ref) introduces the FRCB test, which forms the core of the paper, and provides sufficient conditions for consistency of the test. Section (ref) compares the performance of FRCB to CB in simulations of a test of regression monotonicity. Section (ref) illustrates the usefulness of FRCB with an empirical analysis of the monotonicity of the conditional mean function of radiocarbon age with respect to calendar age. Section (ref) concludes. Proofs of the theorems are in the Appendix.
In this section, I define the class of CB test statistics and give specific examples that fit in the framework. Let $D$ be a random matrix of data with support $\mathscr{D}$, $\mathcal{F}$ the parameter space, $f\in\mathcal{F}$ the function of interest, $G\subseteq\mathbb{R}$ the grid over which $f$ is estimated,\footnote{Every CB implementation uses a grid to check that a non-parametric estimate is of a certain shape. The grid can be as dense as desired. In the simulations in Section (ref), a grid of 100 and a grid of 500 yield the same results for the hypothesis test.} and $\mathsf{H}$ the space of smoothing parameters (e.g., the space of valid bandwidths in kernel regression). Let $\hat{f}:\mathscr{D}\times\mathsf{H}\times\mathbb{R}\rightarrow\mathcal{F}$ be the semi-parametric estimator that maps data and smoothing parameters to the parameter space, evaluated on a grid. This setup covers many situations in applied statistics. $D$ often contains the explanatory variable of interest, covariates, and the outcome variable. For example, even though we assume for simplicity that $f$ is univariate, often covariates are controlled for additively (e.g., in partial linear models or generalized additive models).
The class of null hypotheses for which CB applies can be characterized by a property of the underlying estimator for functions in a null set. Let $\mathcal{H}_{0}\subseteq\mathcal{F}$, $D\subseteq\mathscr{D}$, $\{h,h'\}\subseteq\mathsf{H}$, and suppose the following property holds:\footnote{For cases in which this property is not satisfied exactly, simulations suggest that contradictions of the property rarely occur. For example, bowman_monotonicity find that “out of the total of 90,000 simulations only two cases were discovered where the estimated regression curve was monotonic at one bandwidth and nonmonotonic at a higher one. This behavior is therefore extremely rare and its effect on the test procedure will be negligible.”} \[ \hat{f}(D,h,G)\in\mathcal{H}_{0},\,\,h'>h\implies\hat{f}(D,h',G)\in\mathcal{H}_{0}. \] For example, if $\mathcal{H}_{0}$ is the class of monotone functions, satisfying this property requires that if a smoothing parameter value yields an estimate that is monotone, increasing the smoothing parameter from that value also yields an estimate that is monotone. The CB test statistic is then defined as \[ h^{CB}(D,G)\coloneqq\min\left\{ h\in\mathsf{H}\mid\hat{f}(D,h,G)\in\mathcal{H}_{0}\right\} . \] The null hypothesis is rejected if $h^{CB}(D,G)$ is too large, which occurs when only a large smoothing parameter can force the estimator into being consistent with the null hypothesis. Critical values are determined from a bootstrap.\footnote{To create the bootstrapped data sets, a sample (with replacement) is taken from the residuals and added to $\hat{f}(D,h^{CB},G)$. For more details, see bowman_monotonicity and kostyshak_2017.}
\paragraph{Examples of CB tests}
(1) The seminal CB test of silverman_unimodal corresponds to $\mathcal{F}$ as the class of densities, $\mathcal{H}_{0}$ as the class of densities with less than a specified number of modes, $\hat{f}$ as a kernel density estimator, and $\mathsf{H}$ as the set of kernel bandwidths. (2) The test of bowman_monotonicity corresponds to $\mathcal{F}$ as the class of regression functions, $\mathcal{H}_{0}$ as the collection of monotone regression functions, and $\hat{f}$ as a non-parametric regression estimator. (3) harezlak_heckman extends $\mathcal{H}_{0}$ to regression functions of an arbitrary number of modes, and $\hat{f}$ to any estimator with a smoothing parameter. (4) kostyshak_2017 extends $\mathcal{H}_{0}$ to quasi-convex and quasi-concave regression functions, and $\hat{f}$ to estimators of generalized additive models.
For standard CB tests to have good statistical properties, flatness exclusion is required, which is formally defined as follows:
To gain intuition for why the presence of a flat region invalidates the properties of standard CB tests, consider a simple example in the context of regression. Suppose that \[ y=f(x)+\epsilon, \] where $\mathrm{E}(\epsilon|x)=0$ and $f$ is univariate. The standard CB test is inherently flawed if there exist ${f_{1}\in\mathcal{H}_{0}},{f_{2}\in\mathcal{H}_{0}^{C}}$ that induce indistinguishable test statistics, even in arbitrarily large samples. This flaw can occur even if $\left\Vert f_{1}-f_{2}\right\Vert _{\infty}$ is large. For an example of how this situation arises, suppose that $\mathcal{H}_{0}$ is the class of monotone functions and that $f_{1}$ is monotone and contains an interval, say $[a,b]$, over which it is constant. Suppose that $f_{2}$ is the same as $f_{1}$, except that it has a dip (violation of monotonicity) in some interval $[c,d]$. Since the derivative of $f_{1}$ is zero over $[a,b]$, for small $h$, $\hat{f}_{1}\in\mathcal{H}_{0}^{C}$ with high probability because even a moderate amount of variation in $\hat{f}_{1}$ leads to estimates of the derivative on both sides of zero. As $h$ increases, the variation in $\hat{f}_{1}$ decreases, and eventually $\hat{f}_{1}\in\mathcal{H}_{0}$ with probability increasing toward 1.\footnote{For most semi-parametric estimators, as $h$ tends to infinity, $\hat{f}_{1}$ becomes linear.} Define $h_{stat}^{1}$ to be the CB test statistic associated with $\hat{f}_{1}$. The identification issue described above occurs if $\hat{f}_{2}(D,h_{stat}^{1},G)$ is monotone over $[c,d]$: In this case, the violation of monotonicity of $f_{2}$ over $[c,d]$ was smoothed away as a result of the flat interval over $[a,b]$, and $\hat{f}_{2}$ has a similar distribution as $\hat{f}_{1}$.
A similar manifestation of the flatness problem can distort the size of the CB test and is not driven inherently by the value of the test statistic, as above, but rather by the bias of the bootstrap. The validity of the CB bootstrap depends on $\hat{f}(D,h^{CB},G)$ being close (in probability) to $f$ under the null, because the bootstrapped data sets are constructed based on $\hat{f}(D,h^{CB},G)$. However, in the presence of flat regions, $\hat{f}(D,h^{CB},G)$ is smoother than $f$. The function $\hat{f}(D,h^{CB},G)$ has an almost surely non-zero derivative over the flat interval, and thus the bootstrapped test statistics are smaller, on average, than the original test statistic, yielding a low p\nobreakdash-value. Even in regions outside the flat region, the over-smoothed $\hat{f}(D,h^{CB},G)$ could be substantially different from $f$, and thus the bootstrapped test statistic can be further biased. Nominal size is thus smaller than the actual size.
If $f'(x)$ does not exactly equal zero but is small in absolute value over an interval, although the CB test is consistent, it can have poor finite-sample properties. Simulations in Section (ref) include such a function under the alternative (see $m_{4}$ in Figure (ref)), as well as functions that suffer from the flatness problem under the null (see $m_{1}$ and $flat_{1}$). The next section proposes a solution to the flatness problem.
In this section I introduce flatness-robust critical bandwidth (FRCB), which transforms standard CB into a test that is robust to the presence of flat regions in the function of interest.
We now specify sufficient conditions for consistency of FRCB.
Theorems in this section demonstrate that FRCB extends consistency of standard CB to hold without flatness exclusion. Sufficient conditions for a consistent CB test under flatness exclusion depend on the function of interest (e.g., density or regression function), as well as the specific estimation method employed. For proofs of consistency of CB under flatness exclusion in specific settings, see silverman1983 for the case of densities and kostyshak_2017 for the case of regression functions.
The following two theorems are the main result of this paper and show consistency of FRCB without requiring the flatness exclusion assumption.
The convergence rate and asymptotic distribution of CB test statistics depend on the specific situation (e.g., underlying semi-parametric estimator of the test), but the following theorem shows that when $f$ does not have flat regions, the test statistics of FRCB and standard CB have the same asymptotic distribution.
\paragraph{Software implementation}
FRCB is implemented in the R\nocite{rcore} package frcbstats kostyshak2020_frcbstats, which makes it easy for practitioners to apply FRCB to the testing of monotonicity, quasi-convexity, and quasi-concavity of regression functions. The bootstraps used in standard CB and in step (ref) of Algorithm (ref) are easily parallelized. Compared with other semi-parametric methods, the algorithms are fast and the test is practical when working with millions of observations.
This section discusses the results of simulations of a test of regression monotonicity, and compares rejection rates of the standard CB test with those of the FRCB test for the regression functions shown in Figure (ref). The regressors are uniformly distributed and the additive error term is independent and normally distributed with standard deviation 0.25. The tests use local polynomial regression for the underlying estimator of $f$ and thin plate regression splines for simultaneous confidence bands of the derivative.\footnote{This combination of estimators was chosen because local polynomial regression is well known and thin plate regression splines have nice properties for estimating the derivative and are the default in R's mgcv package wood_mgcv. For more information, see wood_book. The smoothing parameter for the thin plate regression is chosen by generalized cross-validation.} 1,000 bootstrap replications are used for the core CB algorithm and for step (ref) of Algorithm (ref), with an $\alpha_{n}^{flat}$ sequence of $n^{-1/2}$. The same estimation method is used for the application in the next section.
Table (ref) shows rejection rates at the nominal 5% level of CB and FRCB tests for the regression functions graphed in Figure (ref). Rows above the solid horizontal line are cases in which the null hypothesis is true. In the case of a flat line ($m_{1}$), the CB test rejects the null hypothesis of monotonicity more than 10% of the time (twice the nominal level) across all sample sizes. A worse scenario for CB is $flat_{1}$, for which the test rejects in almost half of the samples with a sample size of 100, and the CB rejection proportion tends to 1 in probability as the sample size increases. For both $m_{1}$ and $flat_{1}$, the FRCB test rejects no more than 6% of the time across all sample sizes.
CB tests are known to be conservative, and FRCB tests inherit this property. Even with the correction to conservativeness of CB tests proposed by hallYork2001_calibration, which is used in the simulations, the test is still conservative: Although the test was run at the nominal 5% level, the rejection rate of the FRCB test is closer to 0% than 5% for $m_{1}$. More complex calibrations of the level could be explored but are beyond the scope of this paper.
The rows in the table below the solid line are cases in which the null hypothesis is false. Both CB and FRCB asymptotically reject, but for the case of $m_{4}$, the power of the CB test converges more slowly to 1 than the FRCB test because of the flatness problem that is triggered by the small derivative of the second half of the function. For $m_{3}$, in contrast to $m_{4}$, there is no large region with a derivative close to zero. The rejection rates of CB and FRCB thus become similar for large sample sizes for this regression function. The intuition for this similarity is captured in Theorem (ref).
Two competing effects determine whether the power of FRCB is larger than CB for small sample sizes. Lack of precision when identifying flat regions can cause non-flat regions—which might contain evidence against the null—to be excluded from the $\hat{G}^{NF}$ grid (see step (ref) of Algorithm (ref)) that is passed to the CB test. This effect explains why CB performs better than FRCB in the case of $m_{3}$ for sample sizes 50 and 100: The region just after 0.5 is not estimated precisely enough to determine whether it is non-flat, and thus the grid on which $h^{CB}$ is calculated might not contain a strong violation of monotonicity. On the other hand, a derivative that is relatively small in absolute value results in low power for the same reason that flatness results in asymptotic inconsistency. This second effect explains why, for the case of $m_{4}$, FRCB has higher power than CB for sample sizes of 100 and larger. Which of the two effects dominates depends on the shape of the regression function and the precision of the underlying non-parametric estimator, which is influenced by factors such as the sample size and the distribution of the error term.
The results of the simulations show how the flatness problem can arise under both the null and the alternative, and when there are no flat parts but there are parts with a small first derivative. It is not surprising that FRCB has lower power than CB when the sample size is 50: Typically, relying on fewer assumptions has a cost of lower power. That FRCB has higher power for most sample sizes in the simulation is a bonus feature, and highlights its usefulness in situations in which the derivative is not exactly zero but is small enough to trigger symptoms of the flatness problem.
We examine a subset of radiocarbon data from Irish oak trees, published by pearson1993high and previously analyzed in the context of regression monotonicity using CB by bowman_monotonicity. The dataset consists of the radiocarbon age and the calendar age.\footnote{A measure of precision of the radiocarbon dates is also included in the dataset, but for simplicity the variable is ignored in this paper since bowman_monotonicity found that incorporating the measure did not substantively change their results.} Calibration of this relationship is important, as the radiocarbon age is often used to predict the calendar age when the calendar age is unknown. The raw data are shown in Figure (ref), along with a non-parametric fit and corresponding simultaneous confidence band at the 95% confidence level. The relationship is known to be non-monotone,\footnote{In the radiocarbon dating literature, these fluctuations are sometimes referred to as “wiggles” or “de Vries effects.” For more information, see taylor2014radiocarbon.} and thus is an interesting case study for whether the non-monotonicity can be picked up with a small sample size. A visual inspection of the simultaneous confidence band does not suggest strong evidence against monotonicity. However, visual inspection of the data points themselves does hint at non-monotone fluctuations in the underlying regression function.
A standard CB test, using the same non-parametric estimation method as the simulations, results in a p-value of 0.002. Fitted values corresponding to three selected smoothing parameters, including the test statistic value, are shown in Figure (ref). Consider the fitted curve corresponding to a smoothing parameter of 37.8, which is considerably smaller than the test statistic value of 56.3. The 37.8 fit is monotone, except for the section with calendar ages less than 5,250 years. This shows that the flatness problem could be binding, i.e., both monotone and non-monotone regression functions exist that would yield the same distribution of the test statistic. Another indication that flatness could be a problem for this application is that the three fitted curves have different signs of the derivative over considerable portions of the segment before 5,250. In summary, the small CB p-value could be driven by the 12 data points with values less than 5,250, and it is not clear whether those data points do indeed carry enough information to suggest such a strong rejection of monotonicity.
Given the concerns regarding the flatness problem described above, it is interesting to consider using FRCB to examine whether there exists strong evidence of non-monotonicity when regions with a small derivative (e.g., the region before 5,250) are detected and excluded. The FRCB test statistic is 38.3, which is smaller than the CB statistic (56.3). The FRCB test statistic is usually smaller than the CB test statistic because the set of FRCB inequality constraints is a subset of the CB inequality constraints. In this particular application the difference is large, which is not surprising given the flatness concerns. The fitted value curves corresponding to smoothing parameters from the FRCB test are shown in Figure (ref). The gaps between the line segments are a result of the filtering of the CB grid from step (ref) of Algorithm (ref), and are useful for understanding where the derivative can be interpreted away from zero with precision. The FRCB p-value is 0.013, and thus the non-monotone fluctuations in the sample provide some evidence of non-monotonicity in the population.
This paper provides an asymptotic solution to the fundamental problem that in some situations, even with a large sample size, CB does not provide correct inference. Further, simulations show that FRCB performs better than CB in some finite-sample situations, even when no regions are exactly flat. The application to radiocarbon data provides insight on the flatness problem in an applied situation, and shows how FRCB can be used to provide inference that is robust to flat regions.
By removing an assumption that in practice might be violated, FRCB offers a robust improvement, and, in turn, opportunities to apply critical bandwidth methods to the large class of potential applications.
\printbibliography[heading=bibintoc]