Nonparametric Tests for Treatment Effect Heterogeneity with Duration Outcomes

Nonparametric Tests for Treatment Effect Heterogeneity with Duration Outcomes

Introduction

Assessing whether a policy or treatment has any effect on an outcome of interest has been one of the main concerns in economics and statistics. As summarized by Imbens2009, the focus of policy evaluation literature has been mainly confined to identifying and estimating unconditional treatment effect (TE) measures such as the average, distribution and quantile treatment effects. However, one important aspect of policy evaluations is that treatment effects tend to vary across different subpopulations, and focusing on unconditional\ TE measures can mask important heterogeneity in policy interventions. For instance, a labor market program that does not affect the unemployment duration for the overall population might still be effective for a subgroup of individuals with specific observable characteristics. Assessing if this is the case is particularly important for researchers and policymakers interested in generalizing some findings across time, places and populations, which the literature calls \textquotedblleft external validity\textquotedblright ; see e.g. Hotz2005, Bitler2006, Bitler2008, Bitler2014, Crump2008, and Ding2015. Treatment effect heterogeneity also plays an important role in designing statistical treatment rules, see e.g. Manski2004.

In this article, we propose a unified approach to construct tests for different forms of treatment effect heterogeneity, paying particular attention to situations in which the outcome of interest, typically a duration variable, may be subject to right censoring. We develop tests for both average and distribution treatment effects conditional on covariate values. In particular, we consider nonparametric tests to assess whether $ (a) $ there is any particular subpopulation defined by covariates for which a policy intervention has a nonzero distribution (or average) effect, and $ (b)$ the average treatment effect vary across different subgroups. All proposed tests can be applied under unconfounded treatment assignments, see e.g. Rosenbaum1983, but also when selection into treatment is endogenous and a binary instrumental variable is available to the researcher, see e.g. Imbens1994 and Angrist1996. Finally, we emphasize that all proposed tests are also directly applicable to assess TE heterogeneity with respect to a subset of the available covariate and not only with respect to the entire vector of observed characteristics; see Remark (ref).

The proposed methodology relies on three main components. First, the tests are based on inverse probability weighted (IPW) estimators, in which the propensity score is estimated by nonparametric methods. We focus on the Series Logit Estimator proposed by Hirano2003. Second, as we are interested in TE heterogeneity across subgroups defined by covariates, the tests are based on conditional moment restrictions. To avoid the use of smooth estimates and the \textquotedblleft curse of dimensionality,\textquotedblright\ we adopt an integrated moment approach, see e.g. Bierens1982, Bierens1997, Stute1997, and Escanciano2006a. Finally, to tackle the potential censoring problem inherited in duration outcomes, we characterize the integrated moments as Kaplan-Meier (KM) integrals, see e.g. Stute1993a, Stute1993, Stute1995a, Stute1996a, Chen1997, Sellero2005, and SantAnna2016. It is important to emphasize that such an approach is suitable for both censored and uncensored data.

Combining the aforementioned ingredients, we propose different tests for TE heterogeneity. Our test statistics are suitable functionals of empirical processes whose limiting distribution under the null can be estimated using a multiplicative-type bootstrap. Our proposed tests are of the omnibus type, i.e., they are consistent against any nonparametric fixed alternative. Furthermore, they can detect nonparametric local alternatives converging to the null at the parametric $n^{-1/2}$-rate, $n$ being the sample size. To the best of our knowledge, no other nonparametric test for TE heterogeneity share these properties, even when censoring is not an issue.

The closest papers to ours are Abadie2002, Crump2008, and Lee2009d. In a context without censoring and covariates, Abadie2002 proposes tests for the null hypotheses of zero distribution (local) treatment effect and first-order stochastic dominance between treatment and control groups when selection into treatment may be endogenous. Our proposal generalizes Abadie2002 by accommodating both covariates (and, therefore, treatment effect heterogeneity) and randomly censored outcomes. Crump2008 propose smoothed-based tests for the null of hypotheses of zero and constant conditional average treatment effects under the unconfoundedness assumption. Our proposal generalizes Crump2008 by considering tests for treatment effects heterogeneity beyond the conditional mean, and by allowing endogenous treatment allocations and censored outcomes. Finally, Lee2009d proposes a Mann--Whitney test for the null hypothesis of zero conditional distribution treatment effect (like $ \left( a\right) $ above) for randomly censored outcomes. Nonetheless, it is not clear how one can generalize the proposal in Lee2009d to settings with endogeneity, or how one can use his approach to test other hypotheses related to treatment effect heterogeneity like $(b)$. Delgado2013, Chang2015, Hsu2013, and Lee2017 propose alternative tests for treatment effect heterogeneity but do not allow for right-censored outcomes.

In summary, we contribute to the literature on different fronts. This paper is the first to propose a family of nonparametric tests for TE heterogeneity that $\left( i\right) $ can easily accommodate a variety of research designs and (random) censoring and $\left( ii\right) $ are able to detect local alternatives converging to null at the parametric rate. In addition, $\left( iii\right) $ this paper is one of the first to introduce Kaplan-Meier integrals to the program evaluation literature; see also SantAnna2016 for two-step Kaplan-Meier estimators of different unconditional treatment effect measures.

The remainder of the paper is organized as follows. To gain intuition, we first describe the basic setup in which selection into treatment is exogenous and concentrate on testing the null of zero conditional distribution treatment effect. In Section 3, we derive the asymptotic distribution for the baseline tests and introduce a bootstrap method to approximate their critical values. In Section 4, we present extensions of our basic setup. We consider the null of zero conditional average treatment effect and the null of constant average treatment effect across subpopulations. Furthermore, we show how one can modify the aforementioned tests to accommodate endogenous treatment allocation. A Monte Carlo study in Section 5 investigates the finite sample properties of the tests. In Section 6, \ we use data from the Illinois Reemployment Bonus Experiment and apply the proposed policy evaluation tools to assess the effect of unemployment insurance bonus on unemployment duration. All mathematical proofs are gathered in the Appendix. Finally, all tests discussed in this article can be implemented via the open-source R package kmte, which is freely available from GitHub (\url{https://github.com/pedrohcgs/kmte}).

Notation:\ Let $1\left\{ A\right\} $ be the indicator function, that is, $1\left\{ A\right\} $ is equal to one if $A$ is true and equal to zero otherwise. When $A$ is a vector, such function is taken coordinatewise. For any generic function $J,$ let $J\left( y-\right) =\lim_{a\uparrow y}J\left( a\right) $, $J\left\{ y\right\} =J\left( y\right) -J\left( y-\right) $, and denote the continuous part of $J\left( \cdot \right) $ by $ J^{c}\left( \cdot \right) $. Let $i=\sqrt{-1}$ be the imaginary number. Denote the support of a generic random variable $Z$ by $\mathcal{X}_{Z}$. For a set $\mathcal{W}$, let $l^{\infty }\left( \mathcal{W}\right) $ be the Banach space of all uniformly bounded real functions on $\mathcal{W}$ equipped with the uniform metric $\left\Vert f\right\Vert _{\mathcal{W} }\equiv \sup_{z\in \mathcal{W}}\left\vert f\left( z\right) \right\vert $. We use the notation $\left\Vert \cdot \right\Vert _{\infty }$ to denote the supremum norm. The symbol $\Rightarrow $ denotes weak convergence in $\left( l^{\infty }\left( \mathcal{W}\right) ,\mathcal{W}_{\infty }\right) $ in the sense of J. Hoffmann-J$\phi $rgensen, where $\mathcal{W}_{\infty }$ denotes the corresponding Borel $\sigma $-algebra, and $\overset{p}{\rightarrow }$ denotes convergence in (outer) probability, see e.g. VanderVaart1996. Throughout the paper, all random variables are defined on a common probability space $\left( \Omega ,\mathcal{A},\mathbb{P}\right) .$

Testing for zero conditional distribution treatment effect with censored outcomes

Statistical Framework

We consider a set of individuals flowing into a state of interest, and the time these individuals spend in that state is the outcome of interest, $Y$. Upon inflow, an individual participates in the program or not, i.e., he/she either receives treatment or not. Let $Y\left( 0\right) $ be the potential outcome if no treatment were received and let $Y\left( 1\right) $ be the potential outcome if treatment were received. Define $T$ as the treatment indicator, i.e., $T=1$ if the unit is treated and $T=0$ otherwise. The realized outcome is $Y=(1-T)Y\left( 0\right) +TY\left( 1\right) $. The realized outcome, however, is not always observed because of the censoring mechanism. Let $C\left( 0\right) $ and $C\left( 1\right) $ be potential censoring random variables under the control and treatment groups, respectively, and $C=(1-T)C\left( 0\right) +TC\left( 1\right) $ be the realized censored variable, beyond which $Y$ is not observed. For example, $ C $ may be the time from treatment assignment until the end of a follow-up. The observed outcome is $Q=(1-T)Q\left( 0\right) +TQ\left( 1\right) $ where $ Q\left( t\right) =\min \left( Y\left( t\right) ,C\left( t\right) \right) $, $ t\in \left\{ 0,1\right\} $. On top of $Q$, the non-censoring indicator $ \delta =(1-T)\delta \left( 0\right) +T\delta \left( 1\right) ,$ $\delta \left( t\right) =1\left\{ Y\left( t\right) \leq C\left( t\right) \right\} ,~t\in \left\{ 0,1\right\} $, and a vector of pre-treatment variables $ \mathbf{X}$ are also observed. We consider $\left\{ \left( Q_{i},\delta _{i},T_{i},\mathbf{X}_{i}\right) \right\} _{i=1}^{n}$ as independent and identically distributed ($iid$) random variables.

Denote the conditional distribution of potential outcomes $Y\left( 0\right) $ and $Y\left( 1\right) $ by $F_{Y\left( 0\right) |\mathbf{X}}\left( y|\mathbf{ \cdot }\right) $ and $F_{Y\left( 1\right) |\mathbf{X}}\left( y|\mathbf{\cdot }\right) $, respectively, and let the conditional distribution treatment effect be defined as $\Upsilon \left( y|\cdot \right) \equiv F_{Y\left( 1\right) |\mathbf{X}}\left( y|\cdot \right) -F_{Y\left( 0\right) |\mathbf{X} }\left( y|\cdot \right) .$ To gain intuition, we first focus on testing the hypothesis that the distribution treatment effect (DTE) is equal to zero for every subpopulation defined by covariates, that is,

where $\mathcal{W}_{Y}$ $\subset $ $\mathcal{X}_{Y}$. The alternative hypothesis $H_{1}$ is the negation of $H_{0}$.

A crucial step towards testing ((ref)) is to show that $\Upsilon \left( y|\mathbf{\cdot }\right) $ can be identified from the data. To this end, we make the following assumptions.

We will use the shortcut notation $p_{0}\left( \mathbf{x}\right) \equiv \mathbb{P}(T=1|\mathbf{X=x})$ and refer to $p_{0}\left( \mathbf{x}\right) $ as the (true) propensity score. Assumption (ref) is standard in the treatment effects literature. Assumption (ref)$\left( i\right) $ states that, conditional on observables, treatment assignment is independent of potential outcomes and censoring. Assumption (ref)$ \left( ii\right) $ states that there is overlap in the covariate distributions.

In the absence of censoring, Rosenbaum1983 show that Assumption (ref) suffices to identify different treatment effect measures, in particular $\Upsilon \left( y|\mathbf{\cdot }\right) $. Nonetheless, in our setup censoring introduces an additional identification challenge because the probability of being censored is related to potential outcomes, that is, censoring occurs only if $Y\left( t\right) >C\left( t\right) ,$ $t\in \left\{ 0,1\right\} $. Ignoring the censoring problem or analyzing only the uncensored outcomes would, therefore, introduce another source of confounding. To circumvent this problem, Assumption (ref) imposes additional structure on the censoring mechanism.

Assumption (ref) states that, conditionally on the treatment status, the potential outcomes are independent of the potential censoring random variables, and that, given the underlying potential outcome\ $Y\left( t\right) $, $t\in \left\{ 0,1\right\} $, and the treatment status $T$, the covariates do not provide any further information on whether censoring will take place. A particular case in which Assumption (ref) is satisfied is when $C$ is independent of $ \left( Y,\mathbf{X},T\right) $, as assumed by Bang2000, Anstrom2001, Honore2002, Lee2005, Blundell2007, among many others. One must bear in mind that Assumption (ref) is more general than this particular case: it does not impose any restriction on how $\left( Y\left( 1\right) ,Y\left( 0\right) \right) $ and $\left( C\left( 1\right) ,C\left( 0\right) \right) $ depend on $T$, it allows some dependence between $C\left( 1\right) $, $ C\left( 0\right) $, $T$ and $\mathbf{X}$, and allows the occurrence of censoring to depend on $\mathbf{X}$.

In the following, we establish that, given Assumptions (ref)-(ref), a variety of TE measures are identified from $ \left( Q,\delta ,T,\mathbf{X}\right) .$ In particular, we show that the joint distribution of potential outcome $Y\left( t\right) $ and the vector of covariates $\mathbf{X}$, denoted by $F_{Y\left( t\right) ,\mathbf{X} }\left( y,\mathbf{x}\right) =\mathbb{P}(Y\left( t\right) \leq y,\mathbf{X} \leq \mathbf{x})$, $t\in \left\{ 0,1\right\} $ is identified. Once $ F_{Y\left( t\right) ,\mathbf{X}}\left( y,\mathbf{x}\right) $ is identified, $ F_{Y\left( t\right) |\mathbf{X}}\left( y|\mathbf{\cdot }\right) $ can be recovered by taking the appropriate Radon-Nikodym derivative.

We\ now introduce the multivariate Kaplan-Meier joint distribution, which is the key piece to characterize our Kaplan-Meier integrals. Toward this end, let $H_{Q,\mathbf{X}|T}(y,\mathbf{x}|t)=\mathbb{P}(Q\leq y,\mathbf{X}\leq \mathbf{x}|T=t)$, $H_{Q,\mathbf{X}|T}^{1}(y,\mathbf{x}|t)=\mathbb{P}(Q\leq y, \mathbf{X}\leq \mathbf{x},\delta =1|T=t)$ and

For $t\in \left\{ 0,1\right\} $, let $\tau _{\left( t\right) }:=\min \left( \tau _{Y\left( t\right) },\tau _{C\left( t\right) }\right) $, where $\tau _{Y\left( t\right) }=\inf $ $\left\{ y:\mathbb{P}\left( Y\left( t\right) \leq y\right) =1\right\} ,$ and $\tau _{C\left( t\right) }=\inf $ $\left\{ y: \mathbb{P}\left( C\left( t\right) \leq y\right) =1\right\} $ are the least upper bound of the support of $Y\left( t\right) $ and $C\left( t\right) $. For simplicity, assume that $\tau _{C\left( 1\right) }=\tau _{C\left( 0\right) }=\tau _{C}$, $\tau _{Y\left( 1\right) }=\tau _{Y\left( 0\right) }=\tau _{Y}$, implying that $\tau _{\left( 1\right) }=\tau _{\left( 0\right) }=\tau $.\ For $t\in \left\{ 0,1\right\} $, denote by $A\left( t\right) $ the (possibly empty) set of atoms of the cumulative distribution function of $Q\left( t\right) $, and let

where $\Lambda \left( \left\{ \bar{y}\right\} ,\mathbf{x}|t\right) \equiv \Lambda \left( \bar{y},\mathbf{x}|t\right) -\Lambda \left( \bar{y}-,\mathbf{x }|t\right) $ denotes the jump size (mass) of $\Lambda \left( y,\mathbf{x} |t\right) $ at $\bar{y}$. In the case where $Q$ is absolutely continuous, $ \Lambda \left( \left\{ \bar{y}\right\} ,\mathbf{x}|t\right) \equiv 0$ for almost all $\bar{y},\mathbf{x}$ and $t$. In the other extreme case where $Q$ is purely discrete with support points $\left\{ y_{1},y_{2},\dots ,y_{s}\right\} $, $\Lambda ^{c}\left( y,\mathbf{x}|t\right) \equiv 0~$for almost all $y,\mathbf{x}$ and $t$, and

Note that ((ref)) allows $Q$ to have both discrete and absolutely continuous components.

Finally, for any measurable function $g\left( \cdot \right) ,${

}

Lemma (ref) is based on SantAnna2016, and extends Rosenbaum1983 identification results to setups with censored outcomes. It relies on replacing $F_{Y,\mathbf{X}|T}$ by the multivariate Kaplan-Meier $ F_{Q,\mathbf{X}|T}^{km}$ in ((ref)). Note that $F_{Q,\mathbf{X} |T}^{km} $ only depends on $\left( Q,\delta ,T,\mathbf{X}\right) $ and, therefore, is self-adjusted to the censoring problem. Furthermore, in the absence of censoring, $F_{Q,\mathbf{X}|T}^{km}=F_{Y,\mathbf{X}|T}~a.s.$ Shorack1986, implying that in such a case, our identification result reduces to those of Rosenbaum1983. On the other hand, when the outcome of interest is subject to random censoring but one ignores this feature and treats $Q$ as $Y$, the \textquotedblleft standard\textquotedblright\ IPW estimand can be expressed as

In other words, if one ignores the censoring problem, one would only identify a weighted average of functionals of $\left( i\right) $ $\left( Y\left( t\right) ,X,t\right) $ for the uncensored subpopulation ($Y\left( t\right) \leq C\left( t\right) ~a.s.$) and $\left( ii\right) $ $\left( C\left( t\right) ,X,t\right) $ for the censored subpopulation ($Y\left( t\right) >C\left( t\right) ~a.s.$). Such weighted average representations do not have any clear causal interpretation as these subpopulations (being censored or not) are directly related to the potential outcomes. We bypass the additional source of confounding introduced by the censoring problems by using the multivariate Kaplan-Meier distribution $F_{Q,\mathbf{X}|T}^{km}$ as an integrating measure.

It is important to remark that, because of the censoring problem, nonparametric identification of statistical characteristics that depend on the entire support of $Y\left( t\right) $ such as $\mathbb{E}\left[ Y\left( t\right) \right] $, $t\in \left\{ 0,1\right\} $, crucially depends on the local structure of the distribution of $Y\left( t\right) $ and $C\left( t\right) $ at their endpoint. If $\tau _{Y}<\tau _{C}$, identification is guaranteed for any (well defined) functional of interest; if $\tau _{Y}>\tau _{C}$, only restricted moments can be identified; and if $\tau _{C}=\tau _{Y}$, identification is guaranteed unless

where $F_{Y\left( t\right) ,\mathbf{X}}\left( \left\{ \tau \right\} ,\mathbf{ x}\right) \equiv F_{Y\left( t\right) ,\mathbf{X}}\left( \tau ,\mathbf{x} \right) -F_{Y\left( t\right) ,\mathbf{X}}\left( \tau -,\mathbf{x}\right) .$ In particular, whenever $Y\left( t\right) $ is continuous, $\mathbb{E}\left[ h\left( Y\left( t\right) ,\mathbf{X,}t\right) \right] $ is nonparametrically identified if $\tau _{Y}\leq \tau _{C}$, otherwise one can only identify $ \mathbb{E}\left[ h\left( Y\left( t\right) ,\mathbf{X,}t\right) 1\left\{ Y\left( t\right) \leq \tau \right\} \right] .$ This is intuitive because relevant information about $F_{Y\left( t\right) ,\mathbf{X}}$ on $(\tau _{C},\tau _{Y}]$ will always be cut off as a result of the censoring. Such information cannot be recovered unless one is willing to rely on additional parametric/shape restrictions. As indicated by ((ref)) and ( (ref)), the case of discrete potential outcomes and potential censoring random variables is more delicate: if the the underlying distribution of $Y\left( t\right) $ has a positive mass point at $\tau $ but the $\mathbb{P}\left( C\left( t\right) \geq \tau \right) =0$, no information about $Y\left( t\right) $ at $\tau $ can be recovered from $\left( Q,\mathbf{ X},T\right) $\footnote{ Under Assumptions (ref) and (ref), and with no covariates, it is easy to see the lack of identification at the boundary of the support may appear when $H$ is not absolutely continuous. In light of ((ref)) and ((ref)), if $F_{Y\left( t\right) }$ has a mass point at $\tau $, $F_{Y\left( t\right) }\left( \left\{ \tau \right\} \right) =F_{Y\left( t\right) }\left( \tau \right) -F_{Y\left( t\right) }\left( \tau -\right) $, one would naturally hope to identify if by using integrals of $H_{t,1}\left( \tau \right) -H_{t,1}\left( \tau -\right) =-\left( \left( 1-H_{t,1}\left( \tau \right) \right) -1-H_{t,1}\left( \tau -\right) \right) ,$ $t=0,1$. From the definition of $\tau $, we have that, for $t=0,1$, $0=1-H_{Q|T}\left( \tau |t\right) =\left( 1-F_{Y\left( t\right) }\left( \tau \right) \right) \left( 1-G_{C\left( t\right) }\left( \tau \right) \right) $. When $F_{Y\left( t\right) }\left( \tau -\right) <1$ but $ G_{C\left( t\right) }\left( \tau -\right) =1$, we have that $\tau _{\left( t\right) }\not\in A\left( t\right) $, and $1-H_{Q|T}\left( \tau -|t\right) =\left( 1-F_{Y\left( t\right) }\left( \tau -\right) \right) \times 0=0$. Thus, $H_{t,1}\left( \tau \right) -H_{t,1}\left( \tau -\right) =0,$ implying that if $Y\left( t\right) $ has a mass point at $\tau \not\in A\left( t\right) $, one cannot recover information about it from $\left( Q,T\right) $ . On the other hand, if $F_{Y\left( t\right) }\left( \tau -\right) <1$ but $ G_{C\left( t\right) }\left( \tau -\right) <1,$ then the identification follows naturally as one would be able to observe $Q=\tau $ for those with $ \delta =1$.}. As a direct consequence, in such a case, only restricted moments of potential outcomes can be nonparametrically point identified. Indeed, the second term in ((ref)) emphasizes this feature. In order to ease notation and avoid repetition of arguments, in the rest of the paper, we rule out ((ref)).

One should bear in mind that although nonparametric identification of general statistical characteristics is not always guaranteed, Lemma (ref) is still very powerful. For instance, applying Lemma (ref) with $h\left( Y,\mathbf{X,}T\right) =$ $1\left\{ Y\leq y\right\} 1\left\{ \mathbf{X}\leq \mathbf{x}\right\} ,$ we get that $ F_{Y\left( t\right) ,\mathbf{X}}\left( y,\mathbf{x}\right) $ and $F_{Y\left( t\right) |\mathbf{X}}\left( y|\mathbf{x}\right) $ are identified for $\left( y,\mathbf{x}\right) \in \lbrack -\infty ,\tau ]\times \mathcal{X}_{_{X}}$. This is in contrast to the results of Frangakis1999, Anstrom2001, and Frandsen2014, who restrict the analysis to $y\in \lbrack -\infty ,\bar{\tau}]$, with $\bar{\tau}<\tau $. In practice, given that there is no general rule on how to appropriately choose $\bar{\tau}$, an ad hoc choice of \textquotedblleft small\textquotedblright\ $\bar{\tau}$ can lead to undesirable loss of information. The results in Lemma (ref) completely avoid such a drawback\footnote{ The aforementioned papers rely on inverse probability weighting techniques to achieve identification. As such, when $y$ approaches $\tau ,~$the probability of censoring approaches one, implying that the associated inverse probability of censoring weights approach infinity. Therefore, it is not straightforward to see how one can adapt the identification arguments used in these papers to the case where $\tau =\infty $, for example. Finally, we note that these boundary problems can become even more perverse when one turns attention to estimation.}.

Characterization of the null hypothesis

From Lemma (ref), we have that, for $y\in \lbrack -\infty ,\tau ]$, the conditional DTE ${\small \Upsilon }\left( y|\cdot \right) $ is identified from the data and, therefore, we are able to characterize the null hypothesis ((ref)) in terms of observables.

One approach to constructing tests for ((ref)) is to combine Lemma (ref) with smoothing techniques, estimate the conditional DTE $ \Upsilon \left( y|\mathbf{\cdot }\right) $, and then compare how close $ \Upsilon \left( y|\mathbf{\cdot }\right) $ is to zero. The main drawback of this strategy\ is that, when the dimension of covariates $\mathbf{X}$ is moderate as is commonly the case in policy evaluation, tests based on this local approach suffers from the \textquotedblleft curse of dimensionality\textquotedblright ; see e.g. Fan1996 for related tests in a different context. In the next Lemma, we show that, by exploiting alternative characterizations of ((ref)), one can avoid estimating $ \Upsilon \left( y|\mathbf{\cdot }\right) $, alleviating the drawback associated with the local approach described above. To do so, we rely on the \textquotedblleft integrated moment approach\textquotedblright\ used in the goodness-of-fit test literature, see e.g. Bierens1982, Stute1997, Escanciano2006a, among others.

Lemma (ref) adapts Lemma 1 of Escanciano2006a to the present context. Examples of parametric families $w\left( \cdot ,\mathbf{x} \right) $ such that the equivalence ((ref)) holds are the exponential function $w\left( \mathbf{X},\mathbf{x}\right) =\exp (i\mathbf{x}^{\prime } \mathbf{X})$ with $\mathbf{x}\in \mathbb{R}^{k}$, as in Bierens1982, and the indicator function $w\left( \mathbf{X},\mathbf{x}\right) =1\left\{ \mathbf{X}\leq \mathbf{x}\right\} $ with $\mathbf{x}\in \mathcal{X}_{X},$ as in Stute1997; for alternative weight functions, see e.g. Stinchcombe1998.

From ((ref)), it is indeed clear that in order to test for ((ref) ), it suffices to estimate $I_{w}\left( y,\mathbf{x}\right) $ and check if it is \textquotedblleft sufficiently close to zero\textquotedblright\ for all values of $\left( y,\mathbf{x}\right) \in \mathcal{W}\subseteq [-\infty ,\tau ]\times \Pi $. Of course, the notion of \textquotedblleft sufficiently close\textquotedblright\ depends on $(i)$ the norm one uses, say, the $L_{2}$ -norm, which leads to a Cram\'{e}r-von Mises-type test, or the $\sup $-norm, which leads to a Kolmogorov-Smirnov-type test; and $\left( ii\right) $ the sampling distribution of the estimator of $I_{w}\left( y,\mathbf{x}\right) $ . In the next section, we discuss in detail how one can actually estimate $ I_{w}\left( y,\mathbf{x}\right) $, and how one can use these estimators to form test-statistics for ((ref)).

Test statistic

The characterization of the null hypothesis in ((ref)) suggests using functionals of an estimator of $I_{w}\left( \cdot ,\cdot \right) $ as test statistics. Therefore, we must first estimate $I_{w}\left( \cdot ,\cdot \right) $ using the sample $\left\{ \left( Q_{i},\delta _{i},T_{i},\mathbf{X} _{i}\right) \right\} _{i=1}^{n}$. From Lemma (ref) and the Total Law of Probability, we have that, for $\left( y,\mathbf{x}\right) \in \lbrack -\infty ,\tau ]\times \Pi \mathcal{,}\ t\in \left\{ 0,1\right\} ,$

Thus, to estimate $I_{w}\left( \cdot ,\cdot \right) $, we have to estimate $ \mathbb{P}\left( T=t|\mathbf{X}\right) ,$ $F_{Q,\mathbf{X}|T}^{km}\left( y, \mathbf{x}|t\right) $ and $\mathbb{P}\left( T=t\right) $, $t\in \left\{ 0,1\right\} $.

The task of estimating the propensity score $p_{0}\left( \cdot \right) $ is relatively standard. For instance, when the data comes from a randomized experiment, $p_{0}\left( \cdot \right) $ can be estimated by $ n^{-1}\sum_{i=1}^{n}T_{i}$. Alternatively, when the treatment allocation depends on observable characteristics, one can nonparametrically estimate $ p_{0}\left( \cdot \right) $ using the Series Logit Estimator (SLE) proposed by Hirano2003. To define the SLE, let $\boldsymbol{\lambda }=\left( \lambda _{1},\dots ,\lambda _{r}\right) ^{\prime }$ be a $r$-dimensional vector of non-negative integers with norm $\left\vert \boldsymbol{\lambda } \right\vert =\sum_{j=1}^{r}\lambda _{j}$. Let $\left\{ \boldsymbol{\lambda } \left( l\right) \right\} _{l=1}^{\infty }$ be a sequence including all distinct multi-indices $\boldsymbol{\lambda }$ such that $\left\vert \boldsymbol{\lambda }\left( l\right) \right\vert $ is non-decreasing in $l$ and let $\mathbf{x}^{\lambda }=\prod_{j=1}^{r}\mathbf{x}_{j}^{\lambda _{j}}$ . For any integer $L$, define $\mathbf{R}^{L}\left( \mathbf{x}\right) =\left( \mathbf{x}^{\lambda \left( 1\right) }\mathbf{,}\dots ,\mathbf{x} ^{\lambda \left( L\right) }\right) ^{\prime }$ as a vector of power functions. Let $\mathcal{L}\left( a\right) =\exp \left( a\right) /\left( 1+\exp \left( a\right) \right) $ be the logistic $CDF$. The SLE for $ p_{0}\left( \mathbf{x}\right) $ is defined as $\hat{p}_{n}\left( \mathbf{x} \right) =\mathcal{L}\left( \mathbf{R}^{L}\left( x\right) ^{\prime } \boldsymbol{\hat{\pi}}_{L}\right) $, where

We write $\mathbb{\hat{P}}_{n}\left( T=1|\mathbf{X}=\mathbf{x}\right) =\hat{p }_{n}\left( \mathbf{x}\right) $ and $\mathbb{\hat{P}}_{n}\left( T=0|\mathbf{X }=\mathbf{x}\right) =1-\hat{p}_{n}\left( \mathbf{x}\right) $.

Next, we move to the most challenging step: estimating $F_{Q,\mathbf{X} |T}^{km}\left( y,\mathbf{x}|t\right) $. Note that, as a result of the binary nature of $T$, we have to estimate two distribution functions: $F_{Q,\mathbf{ X}|T}^{km}\left( y,\mathbf{x}|1\right) $, and $F_{Q,\mathbf{X}|T}^{km}\left( y,\mathbf{x}|0\right) $. To this end, we divide the data $\left\{ \left( Q_{i},\delta _{i},T_{i},\mathbf{X}_{i}\right) \right\} _{i=1}^{n}$ into two sub-samples given by different values of treatment status $T$; $\left\{ \left( Q_{i},\delta _{i},\mathbf{X}_{i}\right) \right\} _{i=1}^{n_{1}}$ are those observations with $T_{i}=1\left( n_{1}=\sum_{i}T_{i}\right) $; and $ \left\{ \left( Q_{i},\delta _{i},\mathbf{X}_{i}\right) \right\} _{i=1}^{n_{0}}$ are those observations with $T_{i}=0\left( n_{0}=\sum_{i}\left( 1-T_{i}\right) \right) $. Then, the task of estimating $ F_{Q,\mathbf{X}|T}^{km}\left( y,\mathbf{x}|t\right) $ is reduced to estimating ((ref)) and plugging it into ((ref)). We estimate $\Lambda \left( y,\mathbf{x}|t\right) $ by replacing $H_{Q,\mathbf{X }|T}^{1}(y,\mathbf{x}|t)$ and $H_{Q,\mathbf{X}|T}(y-,\boldsymbol{\infty }|t)$ with their empirical analogues, leading to the estimator

where $Q_{1:n_{t}}\leq $ $\cdots \leq Q_{n_{t}:n_{t}}$ are the ordered $Q$ -values in the sub-sample with $\left\{ T=t\right\} $, and $\mathbf{X}_{ \left[ i:n_{t}\right] }$ and $\delta _{\left[ i:n_{t}\right] }$ are the concomitants of the $i-th$ order statistic, that is, the $\mathbf{X}$ and $ \delta $ paired with $Q_{i:n_{t}}$. Here, ties within outcomes of interest or censoring random variables are ordered arbitrarily, and ties among $Y$ and $C$ are treated as if the former precedes the latter. By plugging $\hat{ \Lambda}_{n}\left( y,\mathbf{x}|t\right) $ into ((ref)), and noticing that $\hat{\Lambda}_{n}\left( y,\mathbf{x}|t\right) $ is a step function, we have that a natural estimator for $F_{Q,\mathbf{X} |T}^{km}\left( y,\mathbf{x}|t\right) $ is

which is the multivariate extension of the time-honored Kaplan1958 product limit estimator proposed by Stute1993. As $\hat{F}_{Q,\mathbf{ X}|T,n}^{km}\left( y,\mathbf{x}|t\right) $ is a step function, it can be seen from ((ref)) and ((ref)) that

where, for $1\leq i\leq n_{t},$

is the Kaplan-Meier weight attached to $Q_{i:n_{t}},$ $t\in \left\{ 0,1\right\} $.

Finally, given the discrete nature of $T$, we can nonparametrically estimate $\mathbb{P}\left( T=t\right) $ by its relative frequency $n_{t}/n.$ Putting all these pieces together, we have that

where, for $t\in \left\{ 0,1\right\} $,

In the absence of censoring, $W_{in_{t}}=n_{t}^{-1}~a.s.$, and ((ref)) is reduced to the empirical analogue of ((ref)). Thus, it is evident that our proposal is suitable for both censored and uncensored outcomes.

With $\hat{I}_{w,n}\left( y,\mathbf{x}\right) $ at hand, testing the null hypothesis ((ref)) is relatively straightforward: for a given weighting function $w\left( \cdot ,\mathbf{x}\right) $, we just need to compare how close $\sqrt{n}\hat{I}_{w,n}\left( y,\mathbf{x}\right) $ is to zero. We consider the usual $sup$ and $L_{2}$ norms, with the indicator weighting function $w\left( \mathbf{X},\mathbf{x}\right) =1\left\{ \mathbf{X}\leq \mathbf{x}\right\} $, leading to the Kolmogorov-Smirnov ($KS$) and Cram\'{e} r-von Mises ($CvM$) test statistics

respectively, where $\hat{I}_{1,n}\left( y,\mathbf{x}\right) $ is defined as $\hat{I}_{w,n}\left( y,\mathbf{x}\right) $ with $w\left( \mathbf{X},\mathbf{x }\right) =1\left\{ \mathbf{X}\leq \mathbf{x}\right\} $, $\hat{H}_{n}\left( y, \mathbf{x}\right) $ denotes the sample analog of $H\left( y,\mathbf{x} \right) =\mathbb{P}\left( Q\leq y,\mathbf{X}\leq \mathbf{x}\right) $, and $ \mathcal{W}=[-\infty ,\tau ]\times \mathcal{X}_{X}.$ Obviously, different test statistics could be developed by applying other distances, or choosing alternative weighting functions $w$, but for ease of exposition, we concentrate of $KS_{n}$ and $CvM_{n}$. To avoid cumbersome notation, in the rest of the article, we consider $\mathcal{W}=[-\infty ,\tau ]\times $ $ \mathcal{X}_{X}$.

Asymptotic Theory

Asymptotic linear representation

We now discuss the asymptotic theory for our test statistics using the following notation. For $t$ $\in \left\{ 0,1\right\} $, let $H_{t}\left( y\right) =\mathbb{P}\left( Q\leq y,T=t\right) $, $H_{t,0}\left( y\right) = \mathbb{P}\left( Q\leq y,\delta =0,T=t\right) $, and $H_{t,11}\left( y,x\right) =\mathbb{P}\left( Q\leq y,\mathbf{X}\leq \mathbf{x},\delta =1,T=t\right) $. Define

Let

and

where

Put

Some remarks are necessary. First, ((ref)) relies only on the \textquotedblleft known\textquotedblright\ functions $\xi _{t}$, $t\in \left\{ 0,1\right\} $. Then, as discussed in Stute1995a,Stute1996a, the first term of $\eta _{t,i}\left( y,\mathbf{x}\right) $ has expectation $ \mathbb{E}\left[ \xi _{t}\left( Q,\mathbf{X},T;y,\mathbf{x}\right) \right] $ . The second and third terms have identical expectations and appear because of the censoring. As is expected and desired, in the absence of censoring, $ \gamma _{t,0}\left( \cdot \right) =1$ $a.s.$, and $\gamma _{t,1}\left( \cdot \right) =\gamma _{t,2}\left( \cdot \right) =0$ $a.s.$.

Given that $\hat{I}_{1,n}(\cdot ,\cdot )$ is the difference of two empirical KM integrals, define

the difference of ((ref)) between the treated and control group.

To discuss the estimation effect coming from not knowing $p_{0}\left( \cdot \right) $ in the KM-integrals, let

Notice that $\alpha _{1}\left( \cdot ;y,\mathbf{x}\right) $ and $\alpha _{0}\left( \cdot ;y,\mathbf{x}\right) $ are nothing more than the conditional expectation of the (functional) derivative of ((ref)) and ( (ref)) with respect to $p_{0}(\cdot )$, respectively. Similarly to ((ref)), define

In order to present our asymptotic results, we need to assume some additional regularity conditions related to the estimation of the propensity score $p_{0}\left( \cdot \right) $, and some integrability conditions to guarantee that the variance of our test statistics is finite and that the censoring effects do not dominate in the right tail. These technical assumptions are stated in Appendix (ref).

Asymptotic null distribution

Using the uniform representation from Lemma (ref), we next establish the weak convergence of the processes $\sqrt{n}\hat{I}_{1,n}\left( y,\mathbf{ x}\right) $ under the null hypothesis

Now, we can apply the continuous mapping theorem to characterize the limiting null distribution\ of our test statistics using the $\sup $ and $ L_{2}$ distances.

Let $T_{n}$ be a generic notation for $KS_{n}$ and $CvM_{n}$. From Corollary (ref), it follows immediately that

where $c_{\alpha }^{^{T}}=\inf \left\{ c\in \lbrack 0,\infty ):\lim_{n\rightarrow \infty }\mathbb{P}\left\{ T_{n}>c\right\} =\alpha \right\} .$

Asymptotic power against fixed and local alternatives

Now we analyze the asymptotic properties of our tests under the fixed alternative $H_{1}$. Under $H_{1}$, $\mathbb{P}\left( \Upsilon \left( y| \mathbf{X}\right) =0\right) <1$ for some $y\in \lbrack -\infty ,\tau ]$, implying that $I_{1}\left( y,\mathbf{x}\right) \not=0$ for some $\left( y, \mathbf{x}\right) \in \mathcal{W}$. Therefore,\ our test statistics $KS_{n}$ and $CvM_{n}$ diverge to infinity. Given that the critical values are bounded, it follows that our tests are consistent. We formalize this result in the next theorem.

Given that our test statistics diverge to infinity under fixed alternatives, it is desirable studying the asymptotic power of these tests under local alternatives. To this end, we study the asymptotic behavior of $\hat{I} _{1,n}\left( y,\mathbf{x}\right) $ under alternative hypotheses converging to the null at the parametric rate $n^{-1/2}$.

Consider the following class of local alternatives:

In the sequel, we need that ((ref)) satisfies the following regularity condition.

From the above theorem and straightforward application of the continuous mapping theorem, we see that our test statistics, under local alternatives of the form of ((ref)), converge to a different distribution because of the presence of a deterministic shift function $R$. This additional term guarantees the good local power property of our tests.

Estimation of critical values

From the above theorems, we see that the asymptotic distribution of $\sqrt{n} \hat{I}_{1,n}\left( \cdot ,\cdot \right) $ depends on the underlying data generating process and standardization is complicated. To overcome this problem, we propose computing critical values with the assistance of a multiplier bootstrap. The proposed procedure has good theoretical and empirical properties, is straightforward to verify its asymptotic validity, is computationally easy to implement and does not require computing new parameter estimates at each bootstrap replication.

To implement the bootstrap, we need nonparametric estimators for all the terms in the asymptotic linear representation of Lemma (ref), namely the propensity score $p_{0}\left( \cdot \right) $, $\eta \left( y,\mathbf{x} \right) $ as in ((ref)), and $\alpha \left( \cdot ;y,\mathbf{x} \right) $ as in ((ref)).

As already discussed, we estimate $p_{0}\left( \cdot \right) $ using the SLE of Hirano2003. In order to estimate $\eta \left( y,\mathbf{x}\right) $ , we notice that after plugging in $\hat{p}_{n}\left( \cdot \right) $, each $ \gamma $ only depends on $H$-functions and is, therefore, estimable by just replacing the $H$-terms by their empirical counterparts. Then, we estimate $ \eta \left( y,\mathbf{x}\right) $ by its empirical analogue

where, for $t\in \left\{ 0,1\right\} ,$

where $\hat{\xi}_{1,n}\left( \cdot ,\cdot ,\cdot ;y,\mathbf{x} \right) $ and $\hat{\xi}_{0,n}\left( \cdot ,\cdot ,\cdot ;y,\mathbf{x} \right) $ are defined as in ((ref)) and ((ref)), respectively, but with the true propensity score $p_{0}\left( \cdot \right) $ replaced by its SLE $\hat{p}_{n}\left( \cdot \right) $, and

are the sample counterparts of $H_{t}\left( \bar{w}\right) $, $H_{t,0}\left( \bar{w}\right) $ and $H_{t,11}\left( \bar{w},\mathbf{\bar{x}}\right) $, respectively.

Finally, we must consider nonparametric estimators for $\alpha \left( \cdot ;y,\mathbf{x}\right) =$ $\alpha _{1}\left( \cdot ;y,\mathbf{x}\right) -\alpha _{0}\left( \cdot ;y,\mathbf{x}\right) $, $\alpha _{1}\left( \cdot ;y, \mathbf{x}\right) $ and $\alpha _{1}\left( \cdot ;y,\mathbf{x}\right) $ being defined in ((ref)). To this end, we must estimate $F_{Y\left( 0\right) |\mathbf{X}}\left( y|\mathbf{\cdot }\right) $ and $F_{Y\left( 1\right) |\mathbf{X}}\left( y|\mathbf{\cdot }\right) $. In the absence of censored data, Donald2013 propose estimating these functions using nonparametric series regression. Given that the outcome of interest $Y$ is subjected to censoring, such a procedure is not at our disposal. Notwithstanding, by using the Kaplan-Meier weights as discussed in Sections (ref) and (ref), we can overcome such a problem and estimate $F_{Y\left( 0\right) |\mathbf{X}}\left( y|\mathbf{x}\right) $ and $ F_{Y\left( 1\right) |\mathbf{X}}\left( y|\mathbf{x}\right) $ by the Kaplan-Meier series estimators:

and

where $\mathbf{R}^{L}\left( \cdot \right) $ is the same power series used in the SLE estimator, with a potentially different number of series. Armed with ((ref)) and ((ref)), we can estimate $\alpha \left( \cdot ;y,\mathbf{x}\right) $ by

Once we have nonparametric estimators for $p_{0}\left( \cdot \right) $, $ \eta \left( y,\mathbf{x}\right) $, and $\alpha \left( \cdot ;y,\mathbf{x} \right) $, the bootstrapped version of $\hat{I}_{1,n}\left( y,\mathbf{x} \right) $ is given by

where $\hat{\eta}_{n}\left( y,\mathbf{x}\right) =\hat{\eta}_{1,n}\left( y, \mathbf{x}\right) -\hat{\eta}_{0,n}\left( y,\mathbf{x}\right) $, and the random variables $\left\{ V_{i}\right\} _{i=1}^{n}$ are $iid$ with bounded support, zero mean and variance one, being independently generated from the sample $\left\{ \left( Q_{i},\delta _{i},T_{i},\mathbf{X} _{i}\right) \right\} _{i=1}^{n}$. A popular example involves $iid$ Bernoulli variables $\left\{ V_{i}\right\} $ with $\mathbb{P}\left( V=1-\kappa \right) =\kappa /\sqrt{5}$ and $\mathbb{P}\left( V=\kappa \right) =1-\kappa /\sqrt{5}$, where $\kappa =\left( \sqrt{5}+1\right) /2,$ as suggested by Mammen1993.

Replacing $\hat{I}_{1,n}\left( y,\mathbf{x}\right) $ with $\hat{I} _{1,n}^{^{\ast }}\left( y,\mathbf{x}\right) ,$ we get the bootstrap versions of $KS_{n}$ and $CvM_{n}$, $KS_{n}^{^{\ast }}$ and $CvM_{n}^{^{\ast }}$, respectively. The asymptotic critical values are estimated by

where $\mathbb{P}_{n}^{^{\ast }}$ means bootstrap probability, i.e. conditional on the sample $\left\{ \left( Q_{i},\delta _{i},T_{i},\mathbf{X} _{i}\right) \right\} _{i=1}^{n}.$ In practice, $c_{n,\alpha }^{^{KS,~\ast }}$ and $c_{n,\alpha }^{^{CvM,~\ast }}$ are approximated as accurately as desired by $\left( KS_{n}^{^{\ast }}\right) _{B(1-\alpha )}$ and $\left( CvM_{n}^{^{\ast }}\right) _{B(1-\alpha )}$, the $B\left( 1-\alpha \right) -th $ order statistic from $B$ replicates $\left\{ KS_{n}^{^{b,\ast }}\right\} _{l=1}^{B}$ of $KS_{n}^{^{\ast }}$ or $\left\{ CvM_{n}^{^{b,\ast }}\right\} _{l=1}^{B}$ of $CvM_{n}^{^{\ast }}$, respectively.

The following algorithm presents a complete procedure to implement our bootstrap-based tests for the null hypothesis of zero conditional distributional treatment effect as in ((ref)). We focus on the Cram \'{e}r-von Mises test but the Kolmogorov-Smirnov test is formed analogously.

Some extensions of the basic setup

Testing for the Zero Conditional Average Treatment Effect

So far, we have only discussed tests for the existence of distribution treatment effects. Although the proposed tests for zero conditional distribution treatment effect are able to detect a very broad set of alternative hypotheses, they are still not able to pin down the direction of the departure from the null. For instance, if we reject the null ((ref) ), we unfortunately do not know if the policy affects the conditional mean or, instead, any other particular feature of the outcome distribution (e.g., its 5th moment). Being able to differentiate such cases is important: policy makers may be in favor of implementing a job training that reduces the average unemployment durations, but may be more reluctant to implement such policy if there is evidence that it affects only the other higher order moments. Given the major role played by the average treatment effect, in this section we show how to adapt our DTE tests to focus on this particular TE measure.

Let $\Upsilon ^{^{cate}}\left( \mathbf{X}\right) \equiv $ $\mathbb{E}\left[ Y\left( 1\right) |\mathbf{X}\right] -$ $\mathbb{E}\left[ Y\left( 0\right) | \mathbf{X}\right] $ be the conditional average treatment effect. From Lemma (ref), we have that identification of $\Upsilon ^{^{cate}}\left( \mathbf{\cdot }\right) $ is not guaranteed unless the support of the censoring variable is larger than or equal to the support the potential outcomes of interest. Given that in follow-up studies such condition is usually violated, it may be more appropriate to focus on the restricted conditional average treatment effect (CATE),

see e.g. Zucker1998. From Lemma (ref) we know that $ \Upsilon _{\bar{\tau}}^{^{cate}}\left( \mathbf{\cdot }\right) $ is nonparametrically identified for all $\bar{\tau}\leq \tau $.

We are concerned with the following hypothesis:

Under $H_{0}^{^{cate}}$, the restricted average treatment effect (ATE) is equal to zero for all subpopulations defined by covariates. The alternative hypothesis $H_{1}^{^{cate}}$ is the negation of the null $H_{0}^{^{cate}}$.

From the same reasoning of Lemma (ref), we can rewrite ((ref)) as

where $I_{\bar{\tau}}^{^{cate}}(\mathbf{x})=I_{\bar{\tau}}^{^{1,cate}}\left( \mathbf{x}\right) -I_{\bar{\tau}}^{^{0}}\left( \mathbf{x}\right) $, with

Then, following the same steps as in Section (ref), our $KS$ type test statistic for hypothesis ((ref)) is

where $\hat{I}_{\bar{\tau},n}^{^{cate}}\left( y,\mathbf{x}\right) $ is defined in ((ref)) at the Appendix (ref). The discussion for the $CvM$ test is the same and is, therefore, omitted. Notice that when $\bar{ \tau}=\tau $, $1\left\{ Q\leq \tau \right\} =1$ $a.s.$ and, therefore, no user-chosen trimming is necessary. This is of particular importance because, in this case, we are using all the information about the average treatment effect available in the data. In the simulation and application in Sections (ref) and (ref), we follow exactly this convention.

Under similar conditions to those in Section (ref), we can derive the asymptotic linear representation of $\sqrt{n}\hat{I}_{\bar{\tau} ,n}^{^{cate}}(\mathbf{x}).$ Using an analogous procedure to the one described in Section (ref), let $c_{\bar{\tau},\alpha ,n}^{^{cate,\ast }}$ denote the bootstrap critical value of the $KS_{\bar{ \tau},n}^{^{cate}}$.

The results in Theorem (ref) are related to Crump2008. In the absence of censoring, Crump2008 propose a test for $ H_{0}^{^{cate}}$ based on smooth estimates of the conditional average treatment effect. In particular, they use a series approach to estimate $ \mathbb{E}\left[ Y\left( 1\right) |\mathbf{X}\right] $ and $\mathbb{E}\left[ Y\left( 0\right) |\mathbf{X}\right] $ and then compare how close the smooth estimate of $\Upsilon ^{^{cate}}\left( \cdot \right) $ is to zero. Given that Crump2008 test is based on the \textquotedblleft local approach\textquotedblright , their test for $H_{0}^{^{cate}}$ is not able to detect local alternatives of the type of $H_{1,n}^{^{cate}}$ and may suffer from the \textquotedblleft curse of dimensionality.\textquotedblright\ This is in contrast with the results in Theorem (ref). Finally, note that as highlighted in Remark (ref), our tests naturally cover the case where researchers are interested in testing for zero CATE only for a subset of covariates $\mathbf{X}_{1}$ of all available characteristics $\mathbf{X}$ , whereas Crump2008 test does not. Thus, one can see that even when censoring is not an issue, our proposal is attractive when compared to Crump2008.

Testing for Homogeneous Conditional Average Treatment Effect

In this section, we show how one can adapt our baseline framework to test whether there is heterogeneity in the (restricted) ATE with respect to observed characteristics. In simple terms, we want to assess whether individuals with different background characteristics have different ATE. Such hypothesis is particularly relevant for policy makers interested in extending a pilot program to a larger population; if there is strong evidence against the hypothesis of homogeneous effect, one may be more concerned with targeting the appropriate population who should receive the treatment, see e.g. Manski2004 and Crump2008.

As in Section (ref), we focus on the restricted CATE. We seek to test

The alternative hypothesis $H_{1}^{^{\hom }}$ is the negation of $ H_{0}^{^{\hom }}.$

Note that we can rewrite ((ref)) as

where $I_{\bar{\tau}}^{^{\hom }}\left( \mathbf{x}\right) =I_{\bar{\tau} }^{^{1,\hom }}\left( \mathbf{x}\right) -I_{\bar{\tau}}^{^{0,\hom }}\left( \mathbf{x}\right) $,

$t\in \left\{ 0,1\right\} $, and $I_{\bar{\tau}}^{^{ate}}$ is the restricted average treatment effect,

Based on this characterization of $H_{0}^{^{\hom }}$, our proposed test statistic for ((ref)) is

where $\hat{I}_{\bar{\tau},n}^{^{\hom }}\left( \mathbf{x}\right) $ is defined in ((ref)) in Appendix (ref). Let $c_{\bar{\tau},\alpha ,n}^{^{\hom ,\ast }}$ denote the bootstrap critical value of the $KS_{\bar{ \tau},n}^{^{\hom }}$.

The results in Theorem (ref) are related to Crump2008, who also proposed a test for $H_{0}^{^{\hom }}$ in a context in which censoring is not present. The test in Crump2008 is not able to detect local alternatives of the type of $H_{1,n}^{^{\hom }}$, and is not suitable to assess the presence of ATE heterogeneity when the conditioning vector in ( (ref)) is $\mathbf{X}_{1}$, a strict subset of $\mathbf{X}.$ As discussed in Remark (ref), our test easily accommodates this situation. Given these attractive features, we argue that, even when censoring is not an issue, the results in Theorem (ref) are of interest to applied researchers and policy makers.

Testing within the Local Treatment Effect setup

In many important applications, the assumption that treatment allocation is exogenous may be too restrictive. For instance, when individuals do not comply with their treatment assignment, or more generally when they sort into treatment based on expected gains, Assumption (ref) is likely to be violated. The goal of this section is to show that, if the unconfoundedness assumption does not hold, our tests are still applicable to the local treatment effect (LTE) setup introduced by Imbens1994 and Angrist1996.

The LTE setup presumes the availability of a binary instrumental variable $Z$ for the treatment assignment. Denote $T\left( 0\right) $ and $T\left( 1\right) $ the value that $T$ would have taken if $Z$ is equal to zero or one, respectively. The realized treatment is $T=ZT\left( 1\right) +\left( 1-Z\right) T\left( 0\right) .$ Thus, the observed sample consists of $iid$ copies $\left\{ \left( Q_{i},\delta _{i},T_{i},Z_{i},\mathbf{X}_{i}\right) \right\} _{i=1}^{n}$. Denote $q_{0}\left( \mathbf{X}\right) \equiv $ $ \mathbb{P}(Z=1|\mathbf{X})$.

To identify the LTE for the subpopulation of compliers, that is, individuals who comply with their actual assignment of treatment and would have complied with the alternative assignment, we need the following assumptions.

Assumption (ref) is standard in the literature, see e.g. Abadie2002, Abadie2003, Frolich2007. Assumption (ref) is analogous to Assumption (ref) and is necessary owing to the censoring. It is important to notice that Assumption (ref) does not restrict how treatment status and instruments affect the censoring variable, which is weaker than typical assumptions used in the literature, see e.g. Frandsen2014.

Because treatment effects are allowed to be arbitrarily heterogeneous, one is only able to identify effects for the complier subpopulation, see e.g. Abadie2003, Frandsen2014 and SantAnna2016. Let $ \Upsilon ^{^{ldte}}\left( y|\mathbf{X}\right) \equiv F_{Y\left( 1\right) | \mathbf{X}}\left( y|\mathbf{X},pop=comp\right) -F_{Y\left( 0\right) |\mathbf{ X}}\left( y|\mathbf{X},pop=comp\right) $. Thus, our goal is to test the null hypothesis

against $H_{1}^{^{ldte}}$, which is simply the negation of ((ref)). The null ((ref)) is analogous to ((ref)) within the LTE setup. For conciseness, we concentrate our attention on $H_{0}^{^{ldte}}$, but of course, we can also adapt the hypotheses discussed in\ Sections (ref) and (ref) to the LTE setup in a routine fashion. Such extensions are presented in the Supplementary Appendix.

In order to proceed, we must show that $\Upsilon ^{^{ldte}}\left( y|\mathbf{ \cdot }\right) $ can be written in terms of observables $\left( Q,\delta ,T,Z,\mathbf{X}\right) $. In the Supplementary Appendix, we show that this is the case by extending Lemma (ref) to the LTE setup. Then, using the integrated moment approach analogous to Lemma (ref), we can show that $H_{0}^{^{ldte}}$ is true if and only if

where $I^{^{ldte}}\left( y,\mathbf{x}\right) =I^{^{1,ldte}}\left( y,\mathbf{x }\right) -I^{^{0,ldte}}\left( y,\mathbf{x}\right) $, and for $t\in \left\{ 0,1\right\} $,

Then, following the discussion in Section (ref), our $KS$ type test statistic for hypothesis ((ref)) is

where $\hat{I}_{n}^{^{ldte}}(y,\mathbf{x})$ is defined in ((ref)) in Appendix (ref). Let $c_{\alpha ,n}^{^{ldte,\ast }}$ denote the bootstrap critical value of the $KS_{n}^{^{ldte}}$.

The results of Theorem (ref) are related to Abadie2002. In the absence of censoring, Abadie2002 proposes a test for the unconditional analogue of $H_{0}^{^{ldte}}$. Of course, by taking $w\left( \mathbf{X},\mathbf{x}\right) =1~a.s.$, we are back to Abadie2002 proposal. Thus, one may interpret Theorem (ref) as extensions of Abadie2002 in two different dimensions: it allows for covariates and for randomly censored outcomes. We are not aware of other proposals that can accommodate either these features.

Monte Carlo simulations

In this section, we conduct a small-scale Monte Carlo exercise in order to study the finite sample properties of our test statistics for the null hypotheses ((ref)), ((ref)) and ((ref)). The $\left\{ V_{i}\right\} _{i=}^{n}$ used in the bootstrap implementations are independently generated as $V$ with $\mathbb{P}\left( V=1-\kappa \right) =\kappa /\sqrt{5}$ and $\mathbb{P}\left( V=\kappa \right) =1-\kappa /\sqrt{5} $, where $\kappa =\left( \sqrt{5}+1\right) /2$, as proposed by Mammen1993. The bootstrap critical values are approximated by Monte Carlo using $1,000$ replications and the simulations are based on $10,000$ Monte Carlo experiments. We report rejection probabilities at the $5\%$ significance level. Results for 10% and 1% significance levels are similar and available upon request.

We consider the following three designs:

where $X$ is distributed as $U\left[ 0,1\right] $, independently of $e\left( 0\right) ,e\left( 1\right) ,C\left( 0\right) $ and $C\left( 1\right) $, $ \varepsilon \left( 0\right) $ and $\varepsilon \left( 1\right) $ are independent standard normal random variables and the parameters $a$ and $b$ are chosen such that the percentage of censoring is equal to 0, 10 or 30 percent in the whole sample. In all designs, $\mathbb{P}\left( T=1|X\right) =\exp \left( -0.5X\right) /\left( 1+\exp \left( -0.5X\right) \right) .$ When testing ((ref)) and ((ref)), Design $\left( i\right) $ falls under the null, whereas Designs $\left( ii\right) -\left( iii\right) $ fall under the alternative. When testing ((ref)), Designs $\left( i\right) -\left( ii\right) $ fall under the null and Design $\left( iii\right) $ falls under the alternative. We set $\bar{\tau}=\tau $ when testing ((ref)) and ((ref)), implying that we do not rely on any truncation.

We report the proportion of rejections for sample sizes $n=100$, $300$ and $ 500$. We estimate $p\left( \cdot \right) $ using the SLE: with $n=100$ we use $1,X$, with $n=300$ we use $1,X,X^{2}$ and with $n=500$ we use $ 1,X,X^{2},X^{3}$ as power functions in the estimation procedure. The proportion of rejections for our tests are presented in Table (ref) . $KS_{n}$ and $CvM_{n}$ stand for the $KS$ and $CvM$ test statistics for the null of zero conditional distribution treatment effect. $ KS_{n}^{^{cate}} $ and $CvM_{n}^{^{cate}}$ are the analogous test statistics for the null of zero conditional average treatment effect and $ KS_{n}^{^{\hom }}$ and $CvM_{n}^{^{\hom }}$for the null of homogeneous average treatment effect across covariate values.

We observe that our tests exhibit good size accuracy even when $n=100$. When the censoring level is 30%, we have that the proposed tests have size below their nominal levels, but as we increase the sample size, such size distortions are minimized. With respect to power, our $KS$ and $CvM$ test statistics reach satisfactory levels for $n=100$, the only exception being when testing for homogeneous ATE with\ a censoring level of 30%. Nonetheless, as the sample size increases, all tests present satisfactory power properties, regardless of the censoring level considered. As one should expect, the power of all tests increases with sample size, and decreases with the degree of censoring. Overall, these simulations show that the proposed bootstrap tests exhibit excellent finite sample properties.

Illinois Reemployment Bonus Experiment

In this section, we demonstrate that our proposed tests can be useful in practice. We analyze data from the Illinois Reemployment Bonus Experiments, which is freely available at the W.E. Upjohn Institute for Employment Research.

From mid-1984 to mid-1985, the Illinois Department of Employment Security conducted a social experiment to test the effectiveness of bonus offers in reducing the duration of insured unemployment. \ At the beginning of each claim, the experiment randomly divided newly unemployed people into three groups:

An important aspect of the Illinois Reemployment Bonus Experiment is that participation was not mandatory. Once claimants were assigned to the treatment groups, they were asked if they would like to participate in the demonstration or not. For those selected to the Job Search Incentive group, 84% agreed to participate, whereas just 65% of the Hiring Incentive group agreed to participate.

Several studies including Woodbury1987, Meyer1996, and Bijwaard2005 have analyzed the impact of the reemployment bonus on the unemployment duration measured by the number of weeks receiving unemployment insurance. Spells which reached the maximum amount of benefits or the state maximum number of weeks, 26, are censored, leading to censoring proportions of 38, 41 and 42 percent for the JSI, HI and the control group, respectively. Apart from the duration data, some information about claimants' background characteristics is also available: age, gender (Male =1), ethnicity (White =1), pre-unemployment earnings and the weekly unemployment insurance benefits amount. For a complete description of the experiment and the available dataset, see Woodbury1987.

Our goal in this application is to assess the effect of reemployment bonuses on unemployment duration. Given the differences between JSI and HI, we analyze these two treatments separately. That is, we consider two sub-samples: one with individuals who are in JSI or in the control group, and one with individuals who are in HI or in the control group.\footnote{ Given that the allocation to each treatment arm is random, i.e., the data come from a randomized control trial, there is no loss of generality of restricting our attention to these two sub-samples. In cases where one has more than two treatment groups and selection into these groups is not completely random, we foresee that one would be able to combine the tools developed in this paper with those in Cattaneo2010 and Ao2019, under the unconfounded setup. When treatment is endogenous, though, it is not yet clear how one can extend the analysis to accomodate multiple instruments; see Mogstad2019 for some recent results in this direction. We leave a detailed discussion of these extensions to future research.} Furthermore, we consider two types of analysis. First, we consider an intention to treat (ITT) analysis, where $T=1$ if an individual is offered to participate in the demonstration, and $T=0$ if an individual was in the control group. In this case, we completely ignored the non-compliance with treatment allocations. Second, in an attempt to disentangle the effects of being offered and actually receiving treatment, we consider a local treatment effect analysis, using the random assignment as an instrumental variable.

Table (ref) reports the results of all our proposed tests, based on 10,000 bootstrap replications. We consider the nulls of $(a)$ zero conditional (local) DTE, $(b)$ zero conditional (local) ATE, and $(c)$ homogeneous (local) ATE across covariate values. The conditioning vector considered consists of all available claimants characteristics described above.

To implement all tests, we estimate the propensity score $p_{0}\left( \cdot \right) $ and the instrument propensity score $q_{0}\left( \cdot \right) $ using the SLE where all covariates enter the model linearly. Given that the data comes from an experimental design, consistency of the propensity score models is guaranteed.

Let us start interpreting the results for the JSI sample. For both the ITT and LTE setup, we reject the null of zero conditional (local) DTE at the 5% level using either the Kolmogorov-Smirnov or the Cram\'{e}r-von Mises test statistic. Such evidence suggests that offering a reemployment bonus to job-searchers has affected the distribution of unemployment duration.

To shed some light on which part of the distribution is affected, we test the null of zero conditional (restricted) ATE as in ((ref)). In the LTE setup, we consider the analogous null of zero conditional local (restricted) ATE

where

For details about how one can construct tests for $H_{0}^{^{clate}}$, see Section S.1.1 of the Supplementary Appendix.

We set $\bar{\tau}=26$, so all the available data is used. From Table (ref), we have that $H_{0}^{^{cate}}$ and $H_{0}^{^{clate}}$ are both rejected at the 5% level when using the $KS$ test and at the 10% level when using the $CvM$ test. Such evidence suggests that the reemployment bonus affected the average unemployment duration. However, we note that SantAnna2016 unconditional (restricted) ATE and LATE estimators are $-0.2221$ and $1.9745$, respectively, and both are not statistically significant at the 10% level. Thus, a researcher who relied only on \textquotedblleft traditional\textquotedblright\ unconditional tests of a zero average effect would have missed the presence of treatment effects in this program.

When conditional (local) ATE is heterogeneous, it may be harder to identify the subpopulation of individuals that the treatment effect is non-zero. However, if the conditional ATE is homogeneous, the task is trivial. With this in mind, we test the null of homogenous conditional (restricted) ATE as in ((ref)). In the LTE setup, we consider the analogous null of homogenous conditional local (restricted) ATE,

For details about how one can construct tests for $H_{0}^{^{l\hom }}$, see Section S.1.2 of the Supplementary Appendix.

As before, we set $\bar{\tau}=26$. For the ITT setup, the null of homogeneous conditional ATE is rejected at the 5% level when using the $KS$ test, and at the 10% level when using the $CvM$ test. When the endogeneity of the selection into treatment is taken into account, we fail to reject the null of homogenous conditional local (restricted) ATE at usual confidence levels. From these results, one concludes that the average treatment effect of being offered versus not being offered into the bonus experiment is heterogeneous. On the other hand, once we restrict our attention to the complier subpopulation, we fail to find enough evidence against the null of homogeneous ATE of actually participating in the JSI program versus not participating.

Next, we analyze the results for the HI sub-sample. Interesting enough, at the 5% level, we fail to reject each considered null hypothesis regardless of the test statistic used. This finding suggests that offering a reemployment bonus to the employer does not affect the time unemployed individuals take to find a job at all.

Overall, the results of our proposed tests suggest that offering an unemployment bonus to the job searcher was effective in changing the length of the unemployment spell. On the other hand, offering the bonus to the employer rather than to the job-searcher seems to be ineffective in changing the unemployment duration.

Conclusion and Suggestions for further Work

In this article, we proposed a variety of nonparametric tests for treatment effect heterogeneity that can accommodate randomly censored outcomes and endogenous treatment allocations. We derived the asymptotic properties of the proposed tests, and have proved that critical values can be easily computed via a relatively simple multiplier bootstrap procedure. Furthermore, in contrast to other proposals, we proved that our tests are able to detect local alternatives converging to the null at the parametric rate. Our Monte Carlo simulations show that our proposed tests have good finite sample properties. Finally, our empirical application concerning the effect of unemployment bonus on unemployment duration showed the feasibility and appeal of our tests in relevant scenarios. Given the desirable features of our tests and the importance of treatment effect heterogeneity in assessing external validity, we argue that the tests proposed in this article are important additions to the applied researcher's toolkit.

For concreteness and compatibility of the procedures for both duration and non-duration outcomes, we framed the article within the context of time-invariant treatment allocation, i.e. when treatment allocation happens at beginning of the duration spell. Nonetheless, when treatment allocation is dynamic, the results of this article still apply to testing for treatment effect heterogeneity between those individuals treated at time $t$ and those not yet treated at time $t,$ as in Sianesi2004. Once the treatment and control groups are defined, the implementation of our proposed tests are exactly the same as described in the text.

In the rest of this section, we discuss extensions of our proposed methodology to other situations of practical interest. For example, we note that researchers may be interested in tests for conditional stochastic dominance, or more generally, tests based on conditional moment inequalities, see e.g. Andrews2013, Andrews2017, Delgado2013, Chang2015 and Hsu2013. As discussed in Section 5.1 of Andrews2013, pointwise asymptotics results do not provide good approximations to the finite-sample properties of test statistics in conditional moment inequality models. Thus, in order to consider tests based on conditional moment inequalities under random censoring, one should first establish a uniform in DGP Bahadur expansion of a suitable two-step Kaplan-Meier integrals; for setting without censoring, see e.g. Lemma 3.1 and Lemma 3.2 in Hsu2013. However, as noted in Remark (ref) , establishing such uniform Bahadur representation for Kaplan-Meier integrals is very technically challenging because traditional empirical process tools such as those in chapter 2.8 of VanderVaart1996 and in chapter 10 of Dudley2014 are not directly applicable to Kaplan-Meier processes. Indeed, to the best of our knowledge, no such result is yet available in the literature, even for the classical univariate Kaplan-Meier estimator for the CDF, let alone more general classes of functions. In light of this observation, we argue that a formal treatment of tests based on conditional moment inequalities under random censoring is beyond the scope of this article and is left for future research.

Another interesting extension one may whish to pursue is constructing tests for treatment effect heterogeneity with randomly right-censored outcomes that impose less stringent conditions on the censoring mechanism than those allowed under Assumption (ref). For instance, one may wish to assume that, for $t\in \left\{ 0,1\right\} $, $\left( Y\left( 0\right) ,Y\left( 1\right) \right) $ $ \protect\mathpalette{\protect\independentT}{\perp} \left( C\left( 0\right) ,C\left( 1\right) \right) |T,\mathbf{X}$, and propose tests based on conditional Kaplan-Meier estimators using procedures similar to Akritas1994, Gonzalez-Manteiga1994 and Lopez2011. Although such an assumption is less restrictive than Assumption (ref), constructing tests for treatment effect heterogeneity based on conditional Kaplan-Meier estimators would rely on an additional set of assumptions. For instance, it would $\left( a\right) $ require additional stronger smoothness and differentiability assumptions; $ \left( b\right) $ rule out empirically relevant situations with discrete duration data as in our empirical application; $\left( c\right) $ involve choosing additional tuning parameters to estimate the conditional CDF; $ \left( d\right) $ and perhaps even more importantly, the statistical analysis would rely on truncation arguments, which in turn would exclude from the analysis some important classes of functions such as (conditional) average treatment effects. At the cost of restricting how covariates may affect the probability of being censored, the tests proposed in this article bypass all these additional challenges. Given that these two procedures rely on different non-nested assumptions, and involves a very different type of statistical analysis, it is hard to generally rank them. We leave such task for future research.

\onehalfspacing