Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
51,740 characters · 13 sections · 61 citation commands
Extreme Changes in Changes
The difference-in-differences (DID) approach is a widely employed empirical strategy for program evaluation in the presence of policy events in time. The common DID methods critically depend on parallel trend assumptions and focus on identifying (conditional) average effects. An alternative empirical strategy is the changes in changes (CIC) method proposed by athey2006identification. At the cost of alternative distributional assumptions, the CIC gets around the common trend assumption and further can identify distributions of counterfactual outcomes as opposed to just their averages. Thus, CIC can be used to analyze heterogeneous individuals via quantile treatment effects under rank invariance.
As are the cases with other quantile-based estimands, however, the existing CIC estimator only works for intermediate quantiles in theory. Practically, for instance, such an estimator is accurate for intermediate quantile levels such as $q \in (0.05,0.95)$ between the fifth to the ninety-fifth percentiles. This limitation for the existing CIC estimator rules out causal inference for those individuals at the extreme top and extreme bottom quantiles. Yet, it is sometimes rather at extreme quantiles that treatment effects are more relevant to social policy analysis. For instance, policymakers often care about treating economically disadvantaged subpopulations like the poorest individuals characterized by the limit $q \rightarrow 0$. The treatment effect for these tail subpopulations could be substantially larger than that for the mid-sample subpopulations, and hence it is imperative for such policymakers to have methods with which they can accurately assess treatment effects at the subpopulations in the tail.
In this paper, we propose an alternative CIC estimator that more accurately estimates the treatment effects at the tails, technically in the limits as $q \rightarrow 0$ and $q \rightarrow 1$. We also develop asymptotic normality for this estimator and propose an easy-to-construct confidence interval. Based on our simulation studies, we provide the following practical recommendation. For the intermediate quantiles, use the existing estimator by athey2006identification along with its standard error. For the extreme quantiles, on the other hand, use our proposed estimator along with its standard error. We suggest using the log-log plot to choose the switching point and demonstrate a combined use of both estimators in our empirical application.
With the proposed econometric method, we revisit the study by hoynes2015income in which they use the 1993 event of EITC reform to evaluate the effects of income gains on infant birth weights. While they analyze average effects via the DID, we focus on the effects at the low quantiles to see if such income gains can improve infant birth weights, particularly for those at the most critical birth weight conditions. This empirical question is of interest because low infant birth weight is known to have long-lasting impacts on the health and economic well-being in adulthood currie2011inequality as well as an immediate impact on infant mortality.
Literature. In contrast to the nowadays extensive body of literature on DID, the literature on CIC is relatively thin. Since its first proposal by athey2006identification, the CIC framework has been extended to fuzzy treatment assignments de2014fuzzy, models with covariates melly2015changes, continuous treatments dhaultoeuille2022nonparametric, and correction of attrition bias ghanem2022correcting. To our best knowledge, however, no preceding paper investigates extreme quantiles in the context of CIC, despite the aforementioned policy relevance. On the other hand, there are a few papers that investigate treatment effects at extreme quantiles outside the context of CIC -- see chernozhukov2005extremal, chernozhukov2011inference, d2018extremal, zhang2018extremal, and deuber2021estimation to list but a few. None of the existing papers on extremal treatment effects consider DID or CIC frameworks.
Organization. Section (ref) provides a review of CIC. Section (ref) introduces the proposed method, and Section (ref) derives asymptotic properties. Section (ref) discusses some practical issues, and Section (ref) extends the proposed method to allow for covariates. Section (ref) shows simulation studies, and Section (ref) presents the empirical application. Section (ref) presents additional simulation results calibrated to the empirical dataset, and Section (ref) concludes.
Stata Command. This paper is accompanied by a Stata command, ecic (extreme changes in changes). The package can be installed from SSC archive with the following command line: ssc install ecic. After the installation, run \underline{\texttt{help} \texttt{ecic}} for usage of the command.
This section briefly reviews the CIC estimator following athey2006identification. The goals here are to introduce the data-generating model and the treatment parameter of interest, as well as to fix notations to be used in the rest of this paper.
Individual $i$ belongs to group $G^{i}\in \{0,1\}$, where value of 0 (respectively, 1) indicates the control (respectively, treatment) group. Each individual is observed in one of the two time periods $T^{i}\in \{0,1\}$ . For each draw $i=1,...,n$ from the population, the group identity $G^i$ and time period $T^i $ are treated as random variables. Letting $Y^{i}$ denote a continuous outcome, econometricians observe a random sample of $ (Y^{i},G^{i},T^{i})$.
The underlying structure to generate $Y^i$ is as follows. Let $Y^{i}_{N}$ (respectively, $Y^{i}_{I}$) denote the potential outcome for individual $i$ under no treatment (respectively, under treatment). The potential outcome $ Y_N$ under no treatment is generated by
where $U$ represents unobserved characteristics, $h\left(\cdot,t\right) $ is strictly increasing for each $t\in \{0,1\}$, and $U \perp T|G$. Let $ I^{i}\in \{0,1\}$ indicate that individual $i$ receives a treatment. In the two-group-two-period setting, we have $I^{i} = G^{i} T^{i}$. The realized outcome $Y^{i}$ is generated by
We now introduce the following short-hand notations:
For any distribution function $F$, we define its left-inverse $F^{-1}$ by $ F^{-1}\left( q\right) =\inf \{y:F \left( y\right) \geq q\}$. In this setup and with these notations, athey2006identification establish
for all $y$, provided that $U|G=1$ is a subset of the support of $U|G=0$.
For each quantile $q\in (0,1)$, the quantile effect of the treatment is thus identified by
The conventional estimator athey2006identification for $ \tau_q^{CIC}$ performs well in middle quantiles, such as $q \in (0.05, 0.95)$ , but may perform less desirably in extreme quantiles (e.g., $q \in (0.00,0.05] \cup [0.95,1.00)$), as are the case with common quantile estimators. Indeed, the asymptotic theory for the conventional estimator rules out extreme values of $q$. In this section, we present our proposed method of estimating $\tau^{CIC}_q$ as $q = q_n \rightarrow 1$ in the right tail. A symmetric argument applies to the limit on the other side of the distribution in the left tail as $q \rightarrow 0$. To stress the drifting sequence of limiting parameters of our interest, we use the notation $\tau _{q}^{eCIC}$ for extreme CIC. In other words, `e' in “eCIC” is used to remind readers that this parameter drifts with $q$ as the sample size increases.
Suppose that the distribution function $F_{Y_{gt}}$ of $Y_{gt}$ has regularly varying tails for each $\{g,t\}\in \{0,1\}^{2}$. Specifically, we assume
for each $\{g,t\}\in \{0,1\}^{2}$. Here, the parameter $\alpha _{gt}>0$ is referred to as the Pareto exponent. Our extreme CIC estimation is built on estimating the Pareto exponent. We emphasize that this assumption is quite mild and most of the common families of parametric distributions as well as a large class of nonparametric distributions satisfy it. For example, the Student-t distribution with $\nu$ degrees of freedom satisfies this condition with $\nu$ being the Pareto exponent. See, for example, de2007extreme and resnick2007book for reviews on this condition.
Let $Y_{gt}^{(1)}\geq Y_{gt}^{(2)}\geq ...\geq Y_{gt}^{(n_{gt})}$ denote the order statistics of the realized outcomes in the group $\{g,t\}$, where $ n_{gt}$ denotes the subsample size in this group. Choose the largest $ k_{gt}+1$ of them, that is
Then, $\alpha _{gt}$ can be estimated by the Hill estimator hill1975simple
As $q \rightarrow 1$, $F_{Y_{gt}}^{-1}\left( q\right) $ is estimated by
Moreover, the tail probability can be estimated by
as $q \rightarrow 1$. See, for example, de2007extreme.
By combining the identifying formula (ref) with the component estimators (ref)--(ref), we obtain the following estimator for the extreme CIC, $\tau _{q}^{eCIC}$.
By simple algebraic manipulations, this expression simplifies as
We thus propose (ref) as the extreme CIC estimator, which is quite simple to implement. The next section presents asymptotic properties based on $k_{gt}\rightarrow \infty$ as $n_{gt}\rightarrow \infty$ for all $g$ and $t$.
We close this section with a discussion of the identifiability of the extreme CIC, $\tau^{eCIC}_q$. While we informally reviewed the identification result of athey2006identification in Section (ref), we should emphasize that it relies on a common support condition athey2006identification. Namely, for the identifying equality (ref) to hold for all $q \in (0,1)$, the support of $U|G=1$ needs to be a subset of the support of $U|G=0$. If this condition fails, then $F_{Y^N_{11}}$ remains unidentified outside of the support of $Y|G=0,T=1$ athey2006identification. Such an unidentified region of $q$ generally contains extreme quantiles. Hence, the common support condition is crucial especially in the context of extreme quantiles. If $F_{Y^N_{11}}$ is bounded away from zero and one on the support of $Y|G=0,T=1$, then we can deduce that the common support condition may be violated.
In this section, we derive a limit distributional property for the proposed extreme CIC estimator. This result paves a way for statistical inference about the extreme CIC.
Let $\{Y_{gt}^{i}\}_{i=1}^{n_{gt}}$ denote the subsample of observed outcomes in group $g$ and time $t$. We state the following set of conditions, followed by discussions of each piece.
Conditions
We provide some discussions about these conditions. Following athey2006identification, Condition 1 assumes random sampling within each time and treatment group, and independence across time periods and groups. Thus, it presumes repeated cross sections rather than panel data. Condition 2 imposes the regularly varying tail conditions on all four conditional distributions of the outcome. More generally, the regularly varying tail condition is equivalent to that the underlying distribution belongs to the domain of attraction of the extreme value distribution with a positive tail index. See, for example, de2007extreme. Since we derive the convergence rate, the second-order Pareto tail approximation is inevitable. The second-order parameter $\rho _{gt}$ governs the distance between the true underlying distribution and the Pareto one. As remarked previously, this condition imposes a rather mild restriction and also satisfies the common support condition. Condition 3 requires that the sample sizes of all subsamples are asymptotically of the same order of magnitude.
Condition 4 specifies the order of the tail thresholds used in estimation. For simplicity of illustration, we select $k_{gt}$ to be of a smaller order than $n_{gt}^{2\rho _{gt}/\left( 1+2\rho_{gt}\right) }$ so that the estimators incur negligible asymptotic biases relative to variances. This requirement is similar in spirit to under-smoothing bandwidths in kernel estimation or under-smoothing dimensions in sieve estimation. On the other hand, if we select $k_{gt}$ to be of the same order of $n_{gt}^{2\rho _{gt}/\left( 1+2\rho_{gt}\right) }$, the asymptotic bias becomes non-negligible, whose expression is complicated. In particular, the asymptotic bias involves the second-order parameter $\rho_{gt}$ de2007extreme. Estimation of this parameter is challenging since it requires further restrictions on the underlying distribution cheng2001confidence, haeusler2007, CarpentierKim2014, which are hard to interpret and hard to justify. Furthermore, such bias estimators entail slower rates of convergence. Given these limitations, we focus on the asymptotics based on undersmoothing for a better statistical inference.
Condition 5 imposes restrictions on the rate at which the quantile level $q$ under investigation tends to the unit in the drifting sequence. In particular, $q$ should tend to one sufficiently fast so that the quantile under investigation is extreme. Otherwise, the $q$ quantile is not in the tail and can be better estimated by the standard CIC method. This condition is also common in the extreme quantile literature de2007extreme. Note that this condition allows for $n_{gt}(1-q)\rightarrow 0$. When this happens, the other part of this condition implicitly imposes a lower bound of $1-q$ and equivalently that the extrapolation cannot be pushed too far in the right tail de2007extreme. To see this, observe that the condition $\log \left[ n_{gt}\left( 1-q\right) \right] =o\left( \sqrt{k_{gt}}\right)$ implies $1-q>n^{-1}\exp(-\varepsilon\sqrt{k_{gt}})$ for each $\varepsilon>0$.
Condition 6 requires that the limit of the counterfactual outcome ratio is finite as $q$ tends to the unit. For simplicity, we consider $\varsigma \in (0,\infty)$. If $\varsigma$ is $0$ or $\infty$, however, the estimator ${\hat{F}_{Y_{gt}}^{-1}(\cdot) }$ has a different convergence rate across $g$ and $t$, and consequently, we could ignore the estimation error for some pairs of $g$ and $t$.
The following theorem establishes the asymptotic normality for the extreme CIC estimator (ref) under these conditions.
A proof is relegated to Appendix (ref).
In finite samples, the asymptotic variance can be estimated by substituting $ \hat{\alpha}_{gt}$, $\hat{\lambda}_{11/gt}=k_{11}/k_{gt}$, and $\hat{ \varsigma}=\hat{F}_{Y_{11}}^{-1}\left( q\right) /\hat{F}_{Y_{01}}^{-1} \circ \hat{F}_{Y_{00}} \circ \hat{F}_{Y_{10}}^{-1}\left( q\right)$ for $\alpha _{gt}$, $\lambda _{11//gt}$, and $\varsigma $, respectively, in the formula of the asymptotic variance $\Omega $ provided in the statement of Theorem (ref). Under the same conditions, this estimator of $\Omega $ is also consistent. The 95% confidence interval is then constructed as
In practice, it is recommended to replace $d_{11}$ by $d_{11} \vee \underline{d}$ for some $\underline{d} > 1$ to ensure a positive value of the logarithm in (ref). We set $\underline{d} = 10$ in the subsequent simulation studies and empirical application. Finally, we remark that $\Omega $ simplifies to
in the special case where $\alpha _{g,t}$ is the same across $\{g,t\}$, say $ \alpha_{g,t}=\alpha$ for all $\{g,t\}$, although we do not impose this restriction in the subsequent numerical analyses. This could happen if the treatment effect is a constant shift of the outcome. Given that $\hat{\alpha} _{g,t}$ is asymptotically independent across $g$ and $t$, we can perform the standard t-test for their equivalence.
This section collects discussions on the remaining practical issues.
The number $k_{gt}$ of order statistics is the key tuning parameter in our method. We propose to use the empirical choice rule proposed by guillou2001diagnostic. We present the detailed procedure here for convenience of readers.
Since the identical algorithm applies to each pair of $g$ and $t$, we suppress these subscripts in this subsection for notational simplicity. Given a random sample $\{Y^1,Y^2,\ldots,Y^n\}$, we first sort them descendingly and denote the order statistics by $Y^{(1)}\geq Y^{(2)} \geq \ldots \geq Y^{(n)}$. Define $Z_{i}=i\log(Y^{(i)}/Y^{(i+1)})$ for $i=1,\ldots ,n-1$. For each $k=1,\ldots, n-1$, construct
where $w_{i}=\text{sgn}\left( k-2i+1\right)\left\vert k-2i+1\right\vert $, $\hat{\xi} = 1/\hat{\alpha}$, and $U_{k}\equiv\sum_{i=1}^{k}w_{i}Z_{i}$. When the Pareto tail approximation performs well, $\mathcal{T}_{k}$ should have its mean close to zero and variance close to one. Accordingly, we can minimize the following criteria based on a moving average of $\mathcal{T}_{k}^{2}$:
The optimal value $k^\ast$ of $k$ is
We now discuss how to define the domain $[\underline q, 1)$ of $q$ on which one may use this extreme CIC estimator, as opposed to the conventional CIC estimator. We suggest to make a scatter plot of $\{\log Y_{gt}^{(i)}\}_{i=1}^{n_{gt}}$ against $\{\log i\}_{i=1}^{n_{gt}}$, called the log-log plot. This plot is linear near small values of $i$ if the tail is approximately Pareto, and our estimator is accurate where it appears linear. In this light, one can choose the boundary point $\underline q$ such that this log-log plot appears linear for $i \in \{1,\cdots,\lfloor n_{gt}(1 - \underline q) \rfloor\}$. We concretely illustrate this procedure in our empirical application in Section (ref).
Our proposed method can be easily extended to allow for covariates. Similarly to athey2006identification, we first regress the outcome variable on the covariates and then apply the proposed extreme CIC estimator to the regression residuals. We formalize this procedure as follows.
Consider the linear model
where $W_{gt}^{i}$ denotes the outcome variable for the $i$-th individual in group $g$ and time $t$, and $X_{gt}^{i}$ denotes the covariate vector. The coefficient $\beta _{gt}$ can be different across $g$ and $t$, and hence the above regression can be conducted separately for each $g$ and $t$. For notational simplicity, we continue using $Y_{gt}^{i}$ to denote the error term, which is now unobserved. Given an estimate $\hat{\beta}_{gt}$, we treat the residuals $\hat{Y}_{gt}^{i}=W_{gt}^{i}-\left( X_{gt}^{i}\right)^{\prime }\hat \beta _{gt}$ as effective observations and construct the proposed extreme CIC estimator based on them. Specifically, we order the residuals as
and replace $\{Y_{gt}^{(i)}\}_{i=1}^{k_{gt}+1}$ with $\{\hat{Y}_{gt}^{(i)}\}_{j=1}^{k_{gt}+1}$ in (ref)--(ref).
In additional to Conditions 1-6, we require the following additional condition.
Condition
Condition 7 is proposed recently by girard2021, who study an estimator of tail features in a more general setup. This condition is mild and satisfied by the least square estimator in the linear model (ref) girard2021. In particular, when $X_{gt}^{i}$ has a compact support, and the regression estimator $\hat{\beta} _{gt}$ is $\sqrt{n}$-consistent, $\vert \hat{Y}_{gt}^{i}-Y_{gt}^{i}\vert $ becomes $\vert\vert X_{gt}^{i}\vert\vert \cdot \vert\vert \hat{\beta}_{gt}-\beta _{gt} \vert \vert = O_p(\sqrt{n_{gt}})$ . Then Condition 7 follows from that $k_{gt}/n_{gt}\rightarrow 0$. In summary, this condition requires that the estimation error is sufficiently small and consequently the CIC estimator based on $\{\hat{Y}_{gt}^{(i)} \}_{i=1}^{k_{gt}+1}$ is asymptotically the same as that based on $ \{Y_{gt}^{(i)}\}_{i=1}^{k_{gt}+1}$.
The following corollary summerzies the result.
A proof is relegated to Appendix (ref).
We use the following data generating design based on our baseline model. Generated first are the group and time period indicators according to
To allow for the endogenous dependence between the group $G^{i}$ and the unobservables $U^{i}$, we in turn generate $U^{i}$ conditionally on $G^{i}$ as follows.
Here, we use the uniform distribution under $G^{i}=1$ for ease of analytic tractability of both the Pareto exponent and the quantile treatment effects and for the purpose of accurate evaluations of simulation results with analytically known true parameter values. We also remark that the conditional independence assumption $U^{i} \perp T^{i}|G^{i}$ of athey2006identification is satisfied in this design by construction. In this two-group-two-period setting, the treatment indicator is in turn defined by $I^{i} = G^{i} T^{i}$.
The potential outcomes are generated through the model
where $F_{t_{\alpha}}^{-1}$ denotes the quantile function of the Student-t distribution with $\alpha$ degrees of freedom. Now, the observed outcomes are in turn generated by
There are three notable features in this data generating process. First, $ F_{Y^N_{11}}$ and $F_{Y^I_{11}}$ in (ref)--(ref) have Pareto exponents of $\alpha$. Second, the second term $U$ on the right-hand side of (ref), but not of (ref), causes heterogeneous treatment effects characterized as follows
Finally, we remark that the monotonicity assumption of athey2006identification for the identification is satisfied in this model.
We evaluate the finite sample performance of our proposed extreme CIC estimator $\hat{\tau}_q^{eCIC}$ given in (ref) along with its standard error estimator (ref). The order statistics $k_{gt}$ are chosen based on guillou2001diagnostic for each subsample $(g,t)$ as described in Section (ref). We also present simulation results for the conventional estimator $\hat{\tau}_q^{CIC}$ of athey2006identification with its standard error estimator athey2006identification. For the standard error estimation for $ \hat\tau_q^{CIC}$, we use Epanechnikov kernel and Silverman's rule of thumb for bandwidth selection. Before presenting the results, we want to stress that we focus on the extreme quantiles $q \in [0.90,1.00)$ on which comparisons are necessarily unfair for the estimator $\hat{\tau}_q^{CIC}$ of athey2006identification, which presumes intermediate quantiles in theory. We confirm and acknowledge that the estimator $\hat{\tau}_q^{CIC}$ of athey2006identification performs better in the intermediate quantiles $q \in (0.10,0.90)$.
Figure (ref) shows Monte Carlo averages and inter-quartile ranges of the estimates under the design with $(\pi_G,\pi_T,\pi_A,\pi_B, \alpha)=(0.1,0.5,1.0,2.0,10)$. The dashed curves on the left column of the figure indicate the average estimates based on the conventional estimator $ \hat{\tau}_q^{CIC}$. The dotted curves on the right column of the figure indicate the average estimates based on our proposed estimator $\hat{\tau} _q^{eCIC}$. In each panel, the shaded regions indicate the inter-quartile ranges of the estimates by the respective methods. The results are shown at the extreme quantiles $q \in [0.90,1.00)$ and for sample sizes $N \in \{2500,5000\}$. The solid curves indicate the true treatment effects. Observe that the conventional estimator $\hat{\tau}_q^{CIC}$ tends to give biased estimates as $q \rightarrow 1$. On the other hand, our proposed estimator $\hat{\tau}_q^{eCIC}$ yields significantly less biased estimates even in the limit $q \rightarrow 1$. We ran many additional simulations with varying design parameter values $(\pi_G,\pi_T,\pi_A,\pi_B,\alpha)$, and the results indicate similar patterns across sets of simulations.
Figure (ref) shows Monte Carlo frequencies that the true treatment effects are covered by the 95% confidence intervals. The dashed curves indicate the results based on the conventional estimator $\hat{\tau} _q^{CIC}$ and the dotted curves indicate the results based on our proposed estimator $\hat{\tau}_q^{eCIC}$. The results are shown at the extreme quantiles $q \in [0.90,1.00)$ and for sample sizes $N \in \{2500,5000\}$. Observe that the coverage frequency based on the conventional method deviates away from the nominal probability of 0.95 as $q \rightarrow 1$. In contrast, the coverage frequency based on our proposed method is close to the nominal probability of 0.95 at each point $q \in [0.90,1.00)$ in the extreme quantiles. We remark again that we ran many additional simulations with varying design parameter values $(\pi_G,\pi_T,\pi_A,\pi_B,\alpha)$, and the results indicate similar patterns across sets of simulations.
In light of these simulation results, we provide the following practical recommendation. Use the conventional estimator $\hat{\tau}_q^{CIC}$ of athey2006identification along with its standard error estimator athey2006identification for intermediate quantiles. On the other hand, use our proposed estimator $\hat{\tau}_q^{eCIC}$ in (ref) along with the standard error estimator (ref) for extreme quantiles. The switching point can be chosen by using the log-log plot described in Section (ref). We also follow this practical guideline for the empirical application to be presented in the next section.
There is a long history in health economics research to study causes and prevention of low infant birth weight. It is an important topic from policy viewpoint because low infant birth weight has been identified to have long-lasting impacts on the health and economic well being in adulthood currie2011inequality as well as they are well known to have immediate impact on infant mortality. Some economic and behavioral factors affecting infant birth weight include maternal smoking almond2005costs,currie2009air, maternal stress aizer2009maternal,camacho2008stress,evans2014giving, and economic resources hoynes2015income, among others.
With studies of average effects as in most of the existing empirical studies, it still remains unknown if these causal factors would have positive impacts on the most vulnerable subpopulation, namely those infants born with extremely low birth weights. There are a few papers chernozhukov2011inference,sasaki2022fixed that study extreme quantiles of infant birth weights, but causal interpretations of their estimation results require to assume exogeneity of the explanatory variable of interest conditional on other observed covariates. In empirical settings admitting a changes-in-changes design, on the other hand, we can handle flexible endogeneity in the treatment choice and study treatment effects for the most vulnerable subpopulation at the extremely low quantiles using the method proposed in this paper.
hoynes2015income use the difference-in-differences (DID) design based on EITC reform (Omnibus Reconciliation Act of 1993, OBRA93) to evaluate the effects of income gains through the EITC on infant health outcomes. They find significant average effects of income shocks on the incidence of low birth weight and the average infant birth weight. In this paper, we aim to complement the work of hoynes2015income by analyzing the heterogeneous effects of the income gains through the EITC on infant birth weight at extremely low quantiles, as opposed to those on average.
Following the prior work by hoynes2015income, we use the U.S. Vital Statistics Natality Data, 1989--1999. We also adopt their DID design for our extreme CIC analysis by following their two key assumptions. First, the effects of the EITC on infant birth weights run through the cash available to the family which arrives through tax refunds and the cash is spent over the subsequent 12 months. Second, we focus on the effects during the sensitive development stage in the three months prior to birth. Consequently, following the cash-in-hand assignment rule of hoynes2015income, we include births in May 1994 or after in the “Post” group ($T=1$) associated with the policy event of OBRA93. The eligibility criteria for the EITC includes the requirement that a taxpayer has a qualifying child. In this light, we include all the second- or higher-order live births as the treatment group ($G=1$). The sample sizes are $n_{00}=2372001$, $n_{01}=1287185$, $n_{10}=2652321$, and $n_{11}=1325598$.
hoynes2015income define subpopulations by year, state, parity, education, race, and mother's age. Then, they treat such a subpopulation as a unit of observation, and use the average birth weight within a subpopulation as the outcome value for the unit. However, this procedure will not allow us to analyze individual heterogeneity with the quantile treatment effect because aggregation eliminates individual heterogeneity. Hence, we use each birth as a unit of observation unlike hoynes2015income. Otherwise, we follow their empirical approach as follows. First, we use year and state fixed effects. Since hoynes2015income use parity, education, race, and mother's age to define their subgroups of aggregation, we instead use this list of variables as covariates in our analysis. To accommodate these covariates, the extended method introduced in Section (ref) is employed. Second, we focus on single women with high school education or less as in hoynes2015income.
To determine the switching point $\overline q$ between our extreme CIC estimator and the conventional CIC estimator, we draw the log-log plots for $-\hat Y_{00}$, $-\hat Y_{01}$, $-\hat Y_{10}$, and $-\hat Y_{11}$ in Figure (ref). Observe in each figure that the plot is reasonably linear up to around the 2.5-th or 5-th percentile, and thereby starts to curve downward. In light of the discussion in Section (ref) and noting that our current focus is on the left tail, we choose the switching point $\overline q$ such that the log-log plot is linear for $i \in \{1,\cdots,\lfloor n_{gt} \overline q \rfloor\}$. To guarantee a well Pareto tail approximation, we define the 2.5-th percentile as our switching point.
Figure (ref) illustrates estimates and confidence intervals for $\tau_q^{CIC}$. The estimates by our proposed method for the extreme quantiles $q \in (0.000,0.025]$ are indicated by dotted curves, and the estimates by athey2006identification for the intermediate quantiles $q \in (0.025,0.200]$ are indicated by the dashed curves. The gray shades indicate pointwise 95 percent confidence intervals.
Observe that the point estimates are unambiguously positive for all the quantiles $q \in (0.000,0.200)$. Furthermore, these income effects are statistically significant at each quantile $q \in (0.000,0.200]$. Therefore, we can conclude that income gains will causally improve the infant birth weights at low quantiles.
While hoynes2015income discover positive effects of the EITC income gains on average, we further find positive effects at the low quantiles in particular. This progress in empirical research is important as causal effects for extremely low infant birth weights are more relevant to policy analysis. Low infant birth weight is known to have have long-lasting impacts on the health and economic well being in adulthood currie2011inequality as well as they are known to have immediate impact on infant mortality. Our findings focusing on the low quantiles imply that income support during pregnancy may help mitigate these adverse health and economic outcomes. We want to stress that, for us to reach this important empirical conclusion, both the conventional estimator $\hat\tau_q^{CIC}$ by athey2006identification and our proposed estimator $\hat\tau_q^{eCIC}$ along with their standard errors are indispensable.
Section (ref) presents simulation studies based on data generated from an artificial design. In this section, we present additional simulation studies with resamples from the empirical data which we use in Section (ref).
Let $\{\hat Y_{gt}^i\}_{i=1}^{n_{gt}}$ denote the residualized sample we obtain in Section (ref) for each $g$ and $t$. For each $g \in \{0,1\}$, we draw a one-percent subsample of size $\lfloor 0.01 \cdot n_{g0} \rfloor$ from $\{\hat Y_{00}^i\}_{i=1}^{n_{00}} \cup \{\hat Y_{10}^i\}_{i=1}^{n_{10}}$ with replacement, and define this subsample as a simulated sample of $Y_{g0}$. Similarly, from each $g \in \{0,1\}$, we draw a one-percent subsample of size $\lfloor 0.01 \cdot n_{g1} \rfloor$ from $\{\hat Y_{01}^i\}_{i=1}^{n_{01}} \cup \{\hat Y_{11}^i\}_{i=1}^{n_{11}}$ with replacement, and define this subsample as a simulated sample of $Y_{g1}$. Since we pool the source samples between the control and the treatment groups for each $t$, the true quantile treatment effect $\tau^{CIC}_q$ is zero for all $q$ by construction. Recall from Section (ref) that the original sample sizes are $n_{00}=2372001$, $n_{01}=1287185$, $n_{10}=2652321$, and $n_{11}=1325598$. Hence, simulation sample sizes are $\lfloor 0.01 \cdot n_{00} \rfloor=23720$, $\lfloor 0.01 \cdot n_{01} \rfloor=12871$, $\lfloor 0.01 \cdot n_{10} \rfloor=26523$, and $\lfloor 0.01 \cdot n_{11} \rfloor=13255$. Under this empirical Monte Carlo design, we run the same set of estimation and inference as in Section (ref), except that we focus on the left tail $q \in (0.00,0.10]$ as opposed to the right tail $q \in [0.90,1.00)$.
The top row of Figure (ref) shows Monte Carlo averages and inter-quartile ranges of the estimates, analogously to Figure (ref) in Section (ref). The dashed curve on the left panel indicates the average estimates based on the conventional estimator $\hat{\tau}_q^{CIC}$. The dotted curve on the right panel indicates the average estimates based on our proposed estimator $\hat{\tau}_q^{eCIC}$. In each panel, the shaded region indicates the inter-quartile ranges of the estimates by the respective methods. The solid curves indicate the true treatment effects. Since the true treatment effects are homogeneously zero for all $q$ under the current data generating design, there is little bias in the both estimators. Therefore, the inter-quartile ranges are nicely symmetric for the both estimators. This feature of the results contrasts with that in Section (ref), where non-trivial biases exist for the conventional estimator $\hat\tau_q^{CIC}$ at the extreme quantiles.
The bottom row of Figure (ref) shows Monte Carlo frequencies that the true treatment effects are covered by the 95% confidence intervals, analogously to Figure (ref) in Section (ref). The dashed curve indicates the results based on the conventional estimator $\hat{\tau}_q^{CIC}$ and the dotted curve indicates the results based on our proposed estimator $\hat{\tau}_q^{eCIC}$. The results are shown at the extreme quantiles $q \in (0.00,0.10]$. Although the conventional estimator does not suffer from bias under the current design, its statistical inference still suffers from size distortions. Our proposed extreme CIC estimator $\hat\tau_q^{eCIC}$ yields substantially less size distortions than the conventional estimator $\hat\tau_q^{CIC}$.
In this paper, we propose a new CIC estimator to accurately estimate the treatment effects at extreme/tail quantiles. We also derive its asymptotic normality result for statistical inference. Our proposal of these new methods is motivated by the fact that policy analysts are often interested in treating subpopulations near tails of the distributions of outcome variables (e.g., extremely poor individuals and infants with extremely low birth weights) while existing CIC estimators are tailored to middle quantiles.
Simulation studies demonstrate that the new extreme CIC estimator along with its standard error estimator performs better than the conventional method in the tails. Based on our observations of these results, we propose to use our proposed CIC estimator for extreme quantiles, while the conventional CIC estimation should be used for intermediate quantiles.
Applying the proposed method to U.S. Vital Statistics Natality Data, we study the effects of income gains from the 1993 EITC reform on infant birth weights for those in the most critical conditions. We find significant positive effects of the income gains on infant birth weights for the subpopulation at the low quantiles of birth weight.
Finally, we remind the readers that this paper is accompanied by a Stata command, ecic (extreme changes in changes). The package can be installed from SSC archive with the following command line: ssc install ecic. After the installation, run \texttt{help} \texttt{ecic} for usage of the command.