Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
124,840 characters · 12 sections · 61 citation commands
Fuzzy Difference-in-Discontinuities: Identification Theory and Application to the Affordable Care Act
\abstract{{ \linespread{0.1}{This paper explores the use of a fuzzy regression discontinuity design where multiple treatments are applied at the threshold. The identification results show that, under the very strong assumption that the change in the probability of treatment at the cutoff is equal across treatments, a difference-in-discontinuities estimator identifies the treatment effect of interest. The point estimates of the treatment effect using a simple fuzzy difference-in-discontinuities design are biased if the change in the probability of a treatment applying at the cutoff differs across treatments. Modifications of the fuzzy difference-in-discontinuities approach that rely on milder assumptions are also proposed. Our results suggest caution is needed when applying before-and-after methods in the presence of fuzzy discontinuities. Using data from the National Health Interview Survey, we apply this new identification strategy to evaluate the causal effect of the Affordable Care Act (ACA) on older Americans' health care access and utilization. \\ Keywords: Fuzzy Difference-in-Discontinuities, Identification, Regression Discontinuity Design, Affordable Care Act.\\ JEL classification: C13, I12, I13, I18. }}}
This paper proposes a method to identify and estimate the partial effect of the treatment of interest when multiple non-mutually-exclusive treatments have been assigned in a fuzzy manner at the same cutoff. We refer to this new approach as a “fuzzy difference-in-discontinuities" design. This name follows grembifiscal and EggersFreierGrembiNannicini2018, who propose a “difference-in-discontinuities” approach that combines features of regression discontinuity (RD) and difference-in-differences designs. As we will describe below, our methodology generalizes grembifiscal and EggersFreierGrembiNannicini2018's results. Our econometric problem can be viewed as a specific case of a general question: how to evaluate the pure effect of a policy intervention in the presence of confounding interventions. The use of non-mutually-exclusive treatments relates our work to the literature on competing risks in survival analysis (see fine1999proportional for a description of competing risk models). In survival analysis, a life may end due to one of many risks, similar to how in policy analysis, an outcome can be caused by the policy of interest or a confounding factor. The main difficulty in policy analysis is that the treatment decision is usually endogenous, while the treatment's effects are heterogenous. How confounding policies affect other policy evaluation methods, such as problem structuring methods, difference-in-differences designs, synthetic control matching or instrumental variables, is left for future research. We use this method to identify the causal effect of the Affordable Care Act (ACA) on health care access and utilization for seniors at age 65.\footnote{As we describe below, our empirical application essentially combines card2008impact's fuzzy regression discontinuity design with the difference-in-differences design.}
Our “fuzzy difference-in-discontinuities" method requires panel data or a pooled cross-sectional sample of the population, where at least one age group is eligible for treatment by all of the policies, while others are eligible for all but the policy of interest. Our identification results show that, under the assumption that the change in the probability of a treatment applying at the cutoff is equal across treatments, a fuzzy difference-in-discontinuities regression identifies the treatment effect of interest. If the treatment probabilities are not equal, a point estimate of the treatment effect using the fuzzy difference-in-discontinuities is biased. For this scenario, we propose alternative estimands of the treatment effect under an alternative set of assumptions. Our identification results cover cases with and without selection at the cutoff and are widely applicable. In general, our results suggest caution is needed when applying before-and-after methods in presence of fuzzy discontinuities.
Our method builds on past findings related to regression discontinuities and the use of before-and-after methods. We specify a set of conditions under which a fuzzy difference-in-discontinuities estimator identifies a local average treatment effect. We propose identification results similar to those by hahn2001identification, but we generalize them to multiple treatments. grembifiscal and EggersFreierGrembiNannicini2018 propose and implement a sharp difference-in-discontinuities estimator that exploits “before-and-after" and discontinuous policy variations (See also Leonardi2013 and BenedettoPaolo2018, who use a difference-in-discontinuities approach.) We extend these works to the case of fuzzy discontinuities.\footnote{To our knowledge, Jackson2019 is the only study that has combined a difference-in-differences design with a fuzzy regression discontinuity design, but it does not develop new theory or make explicit the assumptions that underlie this kind of specification.} The potential outcomes framework enables us to clarify the conditions under which a particular treatment effect of interest can be identified when many treatments are applied.\footnote{In this respect, see gilraine2017multiple, who estimates the effect of class size on student performance in a sharp discontinuity setup.} Our results show that fuzzy treatment assignment leads to very restrictive identification conditions, and therefore should not be ignored.
In the presence of selection on unobservables near the cutoff, a fuzzy RD design can be understood as identifying a local average treatment effect on the compliers. Therefore, this paper is related to the large and growing literature on instrumental variables estimation with multiple treatments (see for instance kirkeboen2016field, kline2016evaluating, hull2018isolateing).\footnote{Also see lee2018identifying, who discuss the identification of a multivalued treatment effect in the presence of multidimensional unobserved heterogeneity.} In a context with mutually exclusive treatments and multiple instruments, kirkeboen2016field establish a set of conditions for point identification. In settings where one instrument shifts two treatments or when there are multiple counterfactual treatments, kline2016evaluating and hull2018isolateing consider the use of covariate-instrument interactions as additional instruments. We complement this literature by assuming that the treatment options under consideration are not necessarily mutually exclusive and may not have additive effects on the outcome.
There is a vast and growing literature evaluating the ACA's effects. Some studies have looked at the effect of a specific aspect of the ACA (e.g. Medicaid expansion) on access to care in particular U.S. states (for instance, sommers2016 and Courtemanche_ambulance). sommers2016 use data from Kentucky, Arkansas and Texas, and a difference-in-differences specification, to assess changes in access to care among low-income adults after two years of ACA implementation. They find that Kentucky's Medicaid program and Arkansas's private option were associated with significant increases in access to primary care among low-income adults. Courtemanche_ambulance confirm that the ACA increased health insurance coverage in states that expanded Medicaid, and also look at the ability of health care service providers to meet demand. Importantly, they find that ambulance response times increased substantially with the implementation of the ACA, which is consistent with a supply-adjustment cost coming from an increase in demand. The coverage gains from the ACA's implementation are well documented. For example, Cohen2015 show that the ACA has reduced the uninsured rate from 16.0% in 2010 to 9.1% in 2015. However, relatively little is known about the effects of the ACA on access to and utilization of health care, despite the fact that the expansion of health insurance coverage was expected to increase the ability of a large proportion of the population to pay for health care services. In a recent review, Laxmaiah2017 find that access to care seems to have diminished under the ACA. This paper provides new evidence on how the ACA affects older Americans' utilization of health care services. Our findings suggest that the ACA exacerbated cost barriers to health care for seniors. In 2014 (relative to 2012), more 65-year-olds delayed care due to costs (an increase of 3.6%), could not afford to pay for prescription drugs (an increase of 7.0%), could not afford to see a specialist (an increase of 7.2%), and could not have a follow-up treatment (an increase of 5.5%). Interestingly, the effects of the ACA are heterogenous across ethnicities and education levels.
The remainder of the paper is organized as follows. Section 2 develops our fuzzy difference-in-discontinuities estimator. Section 3 contains the empirical application. Section 4 discusses the results and concludes.
Consider a population of $N$ individuals, each born in one of $G$ years. Let $Y_{ic}$ be an outcome (e.g. a health-related indicator), where $i=1,...,N$ indexes the individuals, and $c=1,...,G$ indexes the yeas. Define $O_{ic}$ as an indicator variable that identifies whether individual $i$ born in year $c$ is affected by the policy of interest.
Before the introduction of $O_{ic}$, another policy was in place. Let $M_{ic}$ be an indicator variable that identifies whether individual $i$ born in year $c$ participated in this original policy. The selection of participants in $M_{ic}$ is partially determined by a forcing variable $X_{ic}$, and changes discontinuously at the cutoff or threshold $t$. Specifically, we say that an individual $i$ born in year $c$ is treated --- with a higher probability --- when $X_{ic}>t$. The fact that the treatment status is partially determined by a forcing variable $X_{ic}$ means that individuals for whom $X_{ic}<t$ may also be treated by the policy. In this sense, program participation is fuzzy. The selection of participants in $O_{ic}$ is only partially determined by $X_{ic}$ and $t$; it also depends on the year of birth of individual $i$. In this respect, we distinguish between two types of individuals, young and old, denoted by $L$ and $\bar{L}$ respectively, and say that individual $i$ is treated by $O_{ic}$ only if they belong to the younger group, $c \in L$.
Even though we focus on a fuzzy setting, it is useful to describe this assignment mechanism when $O_{ic}$ and $M_{ic}$ are deterministic functions of the running variables. In this case,
and
We define $Y_{ic}(o,m)$ as the potential outcome for individual $i$ born in year $c$ if $O_{ic}=o$ and $M_{ic}=m$, where $m $, $o$ $\in \{ 0, 1\}$, with 1 corresponding to the individual being treated and 0 otherwise. By ((ref)) and ((ref)), the observed outcome is equal to
Our aim is to identify the causal effect of $O_{ic}$ on $Y_{ic}$. We focus on the average treatment effect of $O_{ic}$ at $t$ for $c \in L$, which we denote by $ATE_O(t)$, and which we define as
If $O_{ic}$ is the only treatment using the cutoff $t$, the cross-sectional regression discontinuity estimand would identify the average treatment effect of $O_{ic}$ at $t$. However, in our setting, this estimator will lead to a biased estimate of $ATE_O(t)$ because of the difficulty of separating the effect of $O_{ic}$ from the effect of $M_{ic}$.
Let us define $ATE(t)$ as the cross-sectional fuzzy regression discontinuity estimand and let $ATE_M(t)$ be the fuzzy regression discontinuity estimand of the effect of $M_{ic}$ without application of $O_{ic}$. In the case of a sharp discontinuity, grembifiscal and EggersFreierGrembiNannicini2018 show that $ATE_O(t)$ can be identified using what they call a difference-in-discontinuities estimand. Specifically, they show that $ATE_O(t)=ATE(t)-ATE_M(t)$. However, we show that, in a fuzzy scenario, this result often does not hold without additional assumptions. As described by lee2010regression, in many settings of economic interest, the cutoff only partly determines the treatment status. It is, therefore, possible that the change in the probability of participation differs over time and across different policies.
In the following section, we investigate assumptions under which the difference in the fuzzy discontinuities identifies a policy-relevant quantity when multiple treatments are applied at the same cutoff. Our theoretical framework follows hahn2001identification's model, extending it to multiple treatments using panel or pooled cross-sectional data. The theoretical discussion on identification considers, as a natural departure point from grembifiscal's and EggersFreierGrembiNannicini2018's difference-in-discontinuities estimators, a fuzzy counterpart. We first assume that there is no selection on unobservables, but include the possibility of heterogeneous treatments. This allows us to focus on the importance of the changes in the proportion of individuals affected by the treatment at the cutoff. We further relax the assumption of no selection on unobservables, which might be more realistic. By allowing for selection on unobservables, the causal parameter of interest becomes a local average treatment effect and, as previously mentioned, the results are related to recent developments in the estimation with instrumental variables with multiple alternatives.
Let $Z_{ic}$ be a random variable, and define the limits $Z^+$, $Z^-$ and $Z$ as $Z^+=\lim_{x\rightarrow t^+}E[Z_{ic}|X_{ic}=x]$, $Z^-=\lim_{x\rightarrow t^-}E[Z_{ic}|X_{ic}=x]$, and $Z=\lim_{x\rightarrow t}E[Z_{ic}|X_{ic}=x]$. For any $Z_{ic}$, also define $\tilde{Z}_{ic}= 1\{c\in L\} Z_{ic}$ and $\bar{Z}_{ic}= 1\{c\in \bar{L}\} Z_{ic}$.
To identify the marginal causal effect of $O_{ic}$, we consider the following estimand
where $T_{ic}= O_{ic}M_{ic}$.
We call $\tau_{O}^{FRD}$ in ((ref)) a “fuzzy difference-in-discontinuities” estimator because, like grembifiscal's and EggersFreierGrembiNannicini2018's estimators, it rests on the intuition of combining difference-in-differences and RD strategies, but in our setting, the RD design is fuzzy. The choice of this estimand is motivated as a simple natural extension of grembifiscal's estimand.
In this section, we provide a set of assumptions under which $\tau_{O}^{FRD}$, as defined in ((ref)), identifies the $ATE_O(t)$ in ((ref)). All the assumptions will be conditional on $X_{ic}$ being in the neighborhood of the cutoff $t$.
Assumption 1. The conditional expectation of each potential outcome is continuous in $x$ at $t$, i.e., $E[Y_{ic}(o, m)|X_{ic}=x]$ is continuous in $x$ for all $c$ and all $o, m \in \{ 0,1\}$.
This first assumption is standard in the RD literature, and states that the conditional expectation of all potential outcomes is continuous at the cutoff point.
Assumption 2. $M_{ic}$ and $O_{ic}$ are independent of $Y_{ic}(o,m)$, where $o,m=0,1$.
This second assumption states that the determination of whether an individual is subject to the treatment is independent of the potential outcomes near the cutoff, i.e. individuals cannot self-select into the treatment based on their expected benefits. This assumption will be relaxed later to allow for some self-selection. As we are departing from grembifiscal's estimand, this assumption is a natural first step.
Assumption 3. The effect of the confounding policy $M_{ic}$ when there is no treatment ($O_{ic}=0$) is constant across year of birth: $Y_{c_1} (0, 1) - Y_{c_1} (0, 0) = Y_{c_2}(0, 1) - Y_{c_2}(0, 0)$ for any $c_2 \in L$ and $c_1 \in \bar{L}=C\smallsetminus L$, where $C$ is the set of all birth years in the sample.
In Assumption 3, the confounding policy must have the same effect before and after the treatment of interest. This assumption can be tested by investigating the treatment effect of several consecutive periods with only one treatment at the cutoff or by comparing the groups that did not receive the treatment before and after the treatment period.
Assumption 4. (i) The limits $O^+=\lim_{x\rightarrow t^+}E[O_{ic}|X_{ic}=x]$, $O^-=\lim_{x\rightarrow t^-}E[O_{ic}|X_{ic}=x]$, $M^+=\lim_{x\rightarrow t^+}E[M_{ic}|X_{ic}=x]$, $M^-=\lim_{x\rightarrow t^-}E[M_{ic}|X_{ic}=x]$, $T^+=\lim_{x\rightarrow t^+}E[T_{ic}|X_{ic}=x]$ and $T^-=\lim_{x\rightarrow t^-}E[T_{ic}|X_{ic}=x]$ exist\\ (ii) $O^+\neq O^{-}$, $M^+\neq M^{-}$ and $T^+\neq T^-$.
Assumptions 4 (i) and 4 (ii) are standard RD assumptions for the two policies.
Assumption 5. The discontinuity in the probability of the treatment applying is the same for all policies at the threshold, i.e. $\tilde{O}^+- \tilde{O}^-=\tilde{T}^+-\tilde{T}^-=\tilde{M}^+ -\tilde{M}^{-}.$
Assumption 5 is new, and is one of the contributions of this paper. It requires that the discontinuity in the probability of selection of each policy be the same as well as the joint probability of selection. This assumption is clearly satisfied when the discontinuity is sharp.
The following theorem gives conditions for the identification of the treatment of interest.
Note that the proof of Theorem 1 consists of two steps: first, Assumptions 1 to 4 lead to the difference-in-discontinuity expression in equation ((ref)); then, when Assumption 5 is applied, all the terms other than $ATE_O(t)$ are cancelled out.
Theorem 1 provides conditions allowing us to identify the causal effect of the treatment of interest. Assumption 5, while being strong, is a testable assumption: the three terms to which Assumption 5 imposes a strict equality represent the discontinuities in program participation at the threshold. Theorem 1 can be viewed as a negative result because it shows that the simple extension of the grembifiscal difference-in-discontinuities estimand to the fuzzy case identifies the treatment of interest only under very restrictive assumptions.
In empirical applications, Assumptions 1 to 4 (i) and (ii) can be easily satisfied. As previously mentioned, these assumptions are similar to those used in a standard RD design. However, the Assumption 5 double equality is a strong assumption. The following assumption relaxes it, and imposes a sort of inclusion of the confounding treatment in the treatment of interest.
Assumption 4'. (i) The limits $O^+=\lim_{x\rightarrow t^+}E[O_{ic}|X_{ic}=x]$, $O^-=\lim_{x\rightarrow t^-}E[O_{ic}|X_{ic}=x]$, $M^+=\lim_{x\rightarrow t^+}E[M_{ic}|X_{ic}=x]$ and $M^-=\lim_{x\rightarrow t^-}E[M_{ic}|X_{ic}=x]$ exist;\\ (ii) $O^+\neq O^{-}$ and $M^+\neq M^{-}$; and \\ (iii) It is almost certain that $O_{ic}\geq M_{ic}$ and $\tilde{O}^+- \tilde{O}^-=\tilde{M}^+ -\tilde{M}^{-}= \bar{M}^+-\bar{M}^-$.
The following theorem gives an alternative set of conditions under which our fuzzy difference-in-discontinuities estimator identifies the treatment effect of interest.
The assumptions under which Theorem 2 holds are slightly less restrictive than those for Theorem 1. Moreover, and importantly, the restrictions $O_{ic}\geq M_{ic}$ and $\tilde{O}^+- \tilde{O}^-=\tilde{M}^+ -\tilde{M}^{-}$ are empirically testable. In our empirical application, these two relations together imply that the change in the participation probability in the treatments as a result of being older than 65 should be constant. The strict equality in Assumption 4' (iii) is still very strong (even though it is less restrictive than Assumption 5), as it means that in the case of strict inclusion, the difference on both sides of the cutoff should be similar (i.e. that $\tilde{O}^+-\tilde{M}^+= \tilde{O}^--\tilde{M}^{-}$). If there is selection on unobservables, the assumption may not hold. The following assumption provides another alternative to Assumption 5.
Assumption 5'. The two non-mutually-exclusive treatments interact in an additive manner, i.e. $\tilde{Y}(1,1)-\tilde{Y}(0,1)=\tilde{Y}(1,0)-\tilde{Y}(0,0)$.
In Assumption 5', we also assume that the effect of the second treatment would have been the same with or without the confounding treatment. This assumption may not be empirically testable. Non-mutually-exclusive treatments could amplify or mitigate the effect of the treatment of interest. Nevertheless, using this untestable assumption, we are able to relax the equality assumption.
So far, our set and point identification results have assumed that there was no selection based on potential outcomes (i.e. Assumption 2). The following assumption allows us to relax Assumption 2, generalizing our identification results to scenarios with selection on unobservables.
Assumption 6. (i) $(Y_{ic}(o,m)-Y_{ic}(o_1,m_1), O_{ic}(x))$ and $(Y_{ic}(o,m)-Y_{ic}(o_1,m_1), M_{ic}(x))$ are jointly independent of $X_{ic}$ near the cutoff $t$, with $m, m_1, o, o_1 \in \{0,1\}$ and $ O_{ic}(x)$ and $M_{ic}(x)$ are treatment states given $X_{ic}=x$. \\ (ii) There exists an $\varepsilon>0$ such that $O_{ic}(t+e)\geq O_{ic}(t-e)$, $M_{ic}(t+e)\geq M_{ic}(t-e)$ and $T_{ic}(t+e)\geq T_{ic}(t-e)$ for all $0<e<\varepsilon$.\\ (iii) There exists an $\varepsilon>0$ such that if $e>0$ and is sufficiently small (i.e. $0<e<\varepsilon$),\\ $E[Y_{ic_1}(0,1)-Y_{ic_1}(0,0)| \{M_{ic_1}(t+e)- M_{ic_1}(t-e)=1\}]=E[Y_{ic_2}(0,1)-Y_{ic_2}(0,0)| \{M_{ic_2}(t+e)- M_{ic_2}(t-e)=1\}]$ for any $c_2 \in L$ and $c_1 \in \bar{L}=C\smallsetminus L$.
Assumption 6 (i) means that the choice of the cutoff is exogenous. It allows for selection based on potential outcomes. Assumption 6 (ii) is similar to the monotonicity assumption in the instrumental variables literature. Assumption 6 (iii) is the analogue of Assumption 3 when there is selection on unobservables.
It is important to note that under Assumptions 1, 4', and 6, it can be shown that the fuzzy difference-in-discontinuities model identifies the marginal local average treatment effect (LATE) of the policy or treatment of interest. Moreover, a transformation similar to that obtained in Theorem 2 will also point identify the LATE of the second treatment.
We have shown that a difference-in-discontinuities design can help separate the effects of a policy of interest from those of confounding treatments. We are interested in cases where the treatment is not mutually exclusive and may affect the outcome in a non-additive manner. Identification can be achieved even when there is selection at the threshold based on the potential benefits of a policy. Theorem 4 shows that our fuzzy difference-in-discontinuities estimator identifies the LATE at the discontinuity point.
The identification results presented in this section show conditions for point identification of the $ATE_O(t)$. We have shown that point identification can occur in two scenarios. First, the changes in the treatment probability for both treatments as well as their joint probabilities are equal at the cutoff point. Alternatively, the joint probabilities of treatments might not be needed, as long as the pre-existing treatment is included in the treatment of interest when its application starts (this corresponds to empirical situations where the second treatment is a reinforcement of the existing one). Additionally, we can relax the assumption of equality of treatment probability changes at the cutoff point, replacing it with the assumption that treatment effects are additive (i.e. that $Y(1,1)-Y(0,1)=Y(1,0)-Y(0,0)$). However, this assumption may not be testable.
In all these cases, point identification using a fuzzy difference-in-discontinuities approach relies on strong testable assumptions. In the case of strict inclusion of the pre-existing treatment in the treatment of interest, the assumption of equality of treatment probability changes at the cutoff point means that the difference should stay exactly the same above and below the cutoff. When the equality of treatment probability changes assumption is relaxed, an additivity assumption is required for point identification, ruling out the case of strict superadditivity ($Y(1,1)-Y(0,1) > Y(1,0)-Y(0,0)$) or subadditivity ($Y(1,1)-Y(0,1)<Y(1,0)-Y(0,0)$), and the estimator used is not a direct difference-in-discontinuities estimator.
The estimation and inference of the treatment effect of interest (i.e. of $\tau_{O}^{FRD}$ in last section) can be done using a nonparametric approach. In this subsection, we present the steps of the nonparametric procedure.
The estimation of the treatment effect of interest is obtained using a fuzzy difference-in-discontinuities design via a difference in two ratios. The theorems of the previous section show assumptions under which the difference in the ratios
identifies the treatment effect of the relevant policy at $X=t$. Therefore, to obtain a consistent estimator for $\tau_{F}^{FRD}$, we can use consistent estimators of $\hat{\tilde{Y}}^+$, $\hat{\tilde{Y}}^-$, $ \hat{\tilde{T}}^+$, $\hat{\tilde{T}}^-$ $\hat{\bar{Y}}^+$, $\hat{\bar{Y}}^-$, $ \hat{\bar{M}}^+$ and $\hat{\bar{M}}^-$.
These quantities are commonly estimated using nonparametric regression techniques (see hahn2001identification, and porter2003estimation, otsu2015empirical). The parameters can be estimated by local linear regression estimators, which are optimal (see for instance porter2003estimation) and have better boundary properties than traditional kernel regressions (for example, see fan1992design).
The estimator for $\tilde{Y}^+$ is given by a solution to the following weighted least squares problem, where $\hat{\tilde{Y}}^+=\hat{a}$:
where $ \mathbb{K}$ is the kernel function and $h=h_N$ is the bandwidth satisfying $h\rightarrow 0$ as $N\rightarrow \infty$.
The other quantities included in the first ratio on the right of ((ref)) are estimated using the same type of procedure as in ((ref)). Depending on the quantity we are interested in, $Y_{ic}$ is replaced by $T_{ic}$ or $M_{ic}$. The minimization is made on $X_{ic}\geq t$ or $X_{ic}\leq t$ to get the upper and lower limit estimators, respectively. Note that in the estimation of this first ratio, we use individuals from the year of birth to which both policies are applied.
To obtain the treatment effect of our policy of interest, we need an estimate of the second ratio on the right side of ((ref)). To estimate the terms comprising this second ratio, we follow a similar procedure to that applied to the elements of the first ratio, but with one difference: the sample now consists of those individuals in the age group to which only one policy (the confounding policy) is applied. For instance, the estimator for $\bar{Y}^+$ solves the following weighted least squares problems with respect to $a$, i.e. $\hat{\bar{Y}}^+=\hat{a}$:
The use of two independent samples to evaluate the two ratios ensures the independence of these two quantities. Following Theorem 4 of hahn2001identification, the asymptotic distribution of the estimator is normally distributed, with its mean given by the difference in means of the two ratios, and the variance given by the sum of the variances. The speed of convergence is $n^{\frac{2}{5}}$, and $h=O_p(n^{-\frac{1}{5}})$ where $n=min(N_1, N_2)$ ($N_1$ is the number of individuals in $P$ and $N_2$ is the number of individuals in $\bar{L}$). The asymptotic results can be established with a balanced sample in the two age groups. If the samples are not balanced, we can drop the excess randomly. The conventional Wald-type confidence set for $\tau_{O}^{FRD}$ can be obtained by estimating asymptotic variances of the non-parametric estimator, or by using an appropriate bootstrap method. Another alternative may be to use the empirical likelihood-based inference methods proposed by otsu2015empirical, which circumvent the asymptotic variance estimation issues and have data-determined shapes. However, the procedure needs to be extended to account for a potentially heteroscedastic panel data set.
This non-parametric approach is implemented by selecting a smoothing parameter, $h$. For a standard regression discontinuity design, this parameter can be optimally chosen using data-driven selection methods (see imbens2012optimal and calonico2014robust). In the case of a fuzzy discontinuity, imbens2012optimal suggest proceeding as in imbens2008regression by estimating two optimal bandwidths: one for the main regression outcome and a second for the treatment. To apply this recommendation to our case, we must select four optimal bandwidths. The selection of these bandwidths are theoretically based on homoscedasticity assumptions that may not hold for the pooled cross-section data we are using. While a set of bandwidths might be optimal in the sense of minimizing the integrated mean-squared error, its effect on inference is also of interest. Indeed, calonico2014robust show that confidence intervals constructed using bandwidths that minimize the integrated mean-squared errors are not valid. They propose new theory-based, more robust confidence interval estimators for average treatment effects. To our knowledge, no study has generalized this theory to difference-in-discontinuities settings. The generalization of this theory to these settings (sharp and fuzzy) is important and deserves a careful investigation, which we leave for future research.
Our identification theory shows how to tease out the effect of one specific policy when two treatments are applied at the cut-off. In the case of two distinct policies (none of them is included in the other), Assumption 5 is key for identification. When one of the treatments is included in the other, Assumption 4’ must hold instead of Assumption 5. In this section, we applied our identification theory to analyze the effects of ACA on access to health care services. Enrolling in Medicare before or after 2013 (the date of the application of the ACA) can be driven by factors that are unobservable to econometricians but known to the agent also, the ACA version of Medicare builds on the pre-existing Medicare. Thus, after 2013 the Medicare offer to the elderly includes the pre-existing Medicare and the new benefits brought by the ACA. These extra policies are those we would like to assess the effects. We acknowledge that the lack of distinction between the enrolee of pre-existing Medicare and ACA Medicare version could be viewed as a limitation of this empirical application. However, the fact that the 2014 Medicare is a combination of the pre-existing Medicare and the changes of ACA implies an inclusion, so Assumption 4' applies and enables the use of our identification theory to extract the net effects of the ACA.
The ACA brought the most substantial changes to U.S. health care policy since the creation of Medicare and Medicaid in 1965. These changes were intended to reduce Medicare costs, expand access to health care services, improve quality of care and expand drug coverage. Prior to the ACA, at age 65, people who had worked 40 quarters or more in covered employment were eligible for Medicare, and could also be eligible for Medicaid if their incomes were below a threshold. These eligibility criteria continue under the ACA, but the ACA is more generous for medium-income individuals and slightly more restrictive for high-income seniors.
Medicare (including the ACA's version of Medicare) has four parts. Part A, hospital insurance, provides broad coverage of inpatient expenses including hospital visits, care in skilled nursing facilities, hospice care and home health services. Coverage is free of charge. Part B, medical insurance, covers medical services including physician fees, nursing fees and preventative services. Enrollees pay a modest monthly premium. Part C, Medicare Advantage, is provided by private insurance; it covers the essentials of Part A and Part B benefits, plus urgent and emergency care services. Its monthly premiums vary widely across private insurers.\footnote{See \url{https://www.medicareresources.org/medicare-benefits/medicare-advantage/}.} Part D, prescription drug coverage, was enacted in 2003 to reduce costs, increase efficiency and improve access to prescription medications for seniors and disabled persons.
When the ACA was introduced in 2010, it came with some improvements/changes to Medicare. This included a gradual reduction in the cost of private insurance premiums (Part C): on average, the payment amount per enrollee decreased by about 6% in 2014. The ACA has reduced out-of-pocket expenses for medication of Medicare Part D beneficiaries from 100% of the coverage gap to 50% in 2011, making prescription drugs more affordable. Moreover, under the ACA, Medicare beneficiaries (of whom there were over 20 million in 2011) have access to free preventative care services. This includes mammograms, prostate cancer screenings, depression screenings, obesity screenings and counseling, diabetes screenings and screenings for heart disease. The ACA introduced an important modification to care providers' compensation systems under Medicare by moving away from a fee-for-service system to a capitation system with some quality requirements. For example, hospitals with high readmission rates now receive lower payments. Moreover, the new payment system includes financial incentives for care providers to report on different quality measures, including measures that account for the patient's experience.
The main ACA coverage provisions had taken effect by 2014 Obama2016. Figure 1 in Obama2016 shows that the percentage of individuals without insurance in the U.S. substantially dropped in 2014. This is consistent with the results in sommers2016 and Courtemanche_ambulance.
Medicare after the ACA contains the main characteristics of Medicare before ACA, with some additional benefits and changes in the U.S. health care system. We use “Medicare” to refer to the pre-existing Medicare program, and consider the additional elements to be a different policy (ACA).
We use a linear model to estimate the effect of the ACA on the utilization of health care services by elderly Americans. In addition to sidestepping the theoretical limitations of the non-parametric approach, this specification enables us to compare our results with those of card2008impact.
We restrict our attention to linear regression functions using observations distributed within a distance of $10$ years on both sides of the age 65 cutoff, before and after the implementation of the ACA. We also explore robustness to the inclusion of second-order polynomial terms of age along with interactions and the use of a smaller bandwidth. As discussed below, the estimated discontinuities are generally robust.
We aim to estimate the following model:
and $ M_{ic}=\tau_0+\tau_1 X_{ic}^*+\tau_2 D_c+\tau_3 D_c X_{ic}^*+f(X_{ic},D_c)+\varsigma_{ic}$ and $O_{ic}= \pi_0+\pi_1 X_{ic}^*+\pi_2 D_c+\pi_3 D_c X^*_{ic}+f(X_{ic},D_c)+\upsilon_{ic}$, where $X_{ic}$ is the age of individual $i$ in year $c$, $X_{ic}^*$ is a dummy equal to one if this individual is above the age-65 threshold, $D_c$ is an indicator for the post-ACA period, and $f(X_{ic},D_c)$ is a polynomial function of $X_{ic}$ whose terms include interactions with $D_c$. As the design is not sharp, $M_{ic}$ (participation in Medicare) and $O_{ic}$ (participation in the ACA) are only partly determined by crossing the age-65 cutoff. Indeed, some individuals are eligible for Medicare before 65 for disability reasons, and being eligible after 65 is contingent on having worked at least 40 quarters in covered employment. The estimator of the coefficient $\tau_{O}^{FRD}$ is our fuzzy difference-in-discontinuities estimator, and we obtain it through a two-stage-least-squares-type estimation.
The model in equation ((ref)) has been specified to reflect the general theoretical framework proposed in the previous section. However, in practice, the implementation of the ACA for individuals at age 65 included an extension of pre-existing Medicare benefits. This means that the model estimated is simpler, given by $$Y_{ic}= \alpha_1 + \alpha_2 M_{ic}+\alpha_3 D_c+ \delta D_c \times M_{ic}+f(X_{ic},D_c)+\omega_{ic},$$ with $M_{ic}=\tau_0+\tau_1 X_{ic}^*+\tau_2 D_c+\tau_3 D_c\times X_{ic}^*+f(X_{ic},D_c)+\varphi_{ic}$.
We consider several outcome variables ($Y_{ic}$), all related to health care access or use: whether a person delayed care last year for cost reasons; whether a person did not get care last year for cost reasons; whether a person saw a doctor or went to the hospital last year; whether a person could afford prescription medications, see a specialist, or receive follow-up care last year; and whether a person could get an appointment soon enough last year.
We use survey data from the National Health Interview Survey (NHIS).\footnote{This data is available at \url{https://www.cdc.gov/nchs/nhis/data-questionnaires-documentation.htm}} In our baseline specification, we focus on 2012 and 2014 because, as previously described, major policy changes occurred in many states in 2013. Thus, for those states in which these changes occurred, 2013 is a reasonable choice for ACA implementation. Then, we take 2012 and 2014 as representing two moments in which crucial components related to the ACA had either been implemented or not.\footnote{Because of restrictions related to the birthdate of people surveyed, we could not include data for 2015 or 2016.}
For 2012 and 2014, the NHIS reports respondents' birth years and birth months, and what quarter of the calendar quarter the survey took place. We use this information to identify the age (rounded to the nearest quarter) of the respondents. As in card2008impact, we assume that a person who reaches his 65th birthday in the interview quarter has an age of 65 years and 0 quarters. Assuming a uniform distribution of interview dates, we can say that about one-half of these people will be 0-6 weeks younger than 65, and one-half will be 0-6 weeks older.
We limit our analysis to people who are over 55 and under 75, and to regions in which most states implemented the ACA by 2014. In classifying regions, we follow the scheme in the public NHIS data. This identifies the four Census Regions (Northeast, Midwest, South and West). In our baseline specification, we limit our analysis to the Northeast, Midwest and West regions, where most states implemented the ACA by 2014 (see the Kaiser Family Foundation, at \url{https://www.kff.org/}; see also \url{https://www.advisory.com/daily-briefing/resources/primers/medicaidmap}). Thus, we exclude the District of Columbia and the following states: AL, AR, DE, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA and WV. Because of data use restrictions, in our main specification we include a few states that did not implement the ACA by 2014 (i.e. ID, KS, ME, MO, NE, SD, UT, WI and WY) and exclude a few jurisdictions that implemented the ACA by 2014 (i.e. AR, DE, DC, KY, LA, MD and WV). However, our analysis is robust to the exclusion of the Midwest. The final sample size is 25,291 individuals, although some outcomes are only available for a smaller subsample.
As previously argued, our proposal for identifying the ACA's effects can be best described as a LATE, and relies on the application of the LATE version of Theorem 2 (i.e. Theorem 4). For this Theorem, the relevant Assumptions are 1, 4' and 6. In this section we discuss the plausibility of these assumptions to our case study.
Assumption 1 is consistent with most of the relevant literature, which uses age 65 as a threshold in the U.S. (see card2008impact for an exhaustive list).
Participation in both Medicare and the ACA is partially determined by the same 65-year age threshold for eligibility. Figure (ref) illustrates this by showing the age profiles of health insurance coverage estimated separately for each treatment (2012 for Medicare, plotted with circles, and 2014 for the ACA, plotted with diamonds). The figure shows that for each treatment, there is a significant increase in the coverage rates. This suggests that the age threshold of 65 provides a credible source of exogenous variation in insurance status for both policies. This also means that Assumptions 4' $(i)$ and 4' $(ii)$ are likely to be satisfied.
Figure (ref) also illustrates a second and important relationship between the likelihood that a person is eligible for Medicare and the likelihood they are eligible for the ACA, both at the same age-65 threshold: the rise in the share of coverage rates at age 65 is virtually the same for 2012 and 2014. This provides evidence that the probability of selection into Medicare and the ACA's Medicare are likely the same. This is consistent with the second part of Assumption 4' (iii) (i.e. that $\tilde{O}^+- \tilde{O}^-=\tilde{M}^+ -\tilde{M}^{-}= \bar{M}^+-\bar{M}^-$).
Table (ref) confirms the results in Figure (ref) by showing the effects of reaching age 65 on the insurance status for Medicare (Panel A) and the ACA (Panel B) on five insurance-related variables: the probability of having Medicare coverage, the probability of having any health insurance coverage, the probability of having private coverage, the probability of having two or more forms of coverage, and the probability that an individual's primary health insurance is a managed care program. Column (1) in Panels A and B shows that reaching age 65 significantly increases the probability of having Medicare in 2012 (Panel A) and 2014 (Panel B) and, importantly, that the increase in both probabilities is the same. Panel C confirms this result by showing the estimates of the difference-in-discontinuities estimator where the dependent variable is insurance status. Column (1) in Panel C shows that the probability of having Medicare coverage is not affected by the ACA's rollout. Columns (3) to (5) show that this result holds for the probability of having private coverage, the probability of having two or more forms of coverage, and the probability that an individual's primary health insurance is a managed care program.
Two policies that use the same cutoff are likely to be complements or substitutes. As previously argued, in our scenario the policies seem to be complement. In this respect, note that all 2014 Medicare users are treated by the ACA's Medicare program. This is consistent with the first part of Assumption 4' (iii) (i.e. that $O_{ic}\geq M_{ic}$).
Assumptions 1 and 6 are difficult to test. However, we propose a set of placebo regressions to evaluate their plausibility. Assumption 6 (iii) stipulates that in the absence of the ACA program, the effect of Medicare on the utilization of health care services should be the same. In Tables (ref) to (ref), we construct placebo fuzzy difference-in-discontinuities estimates for age groups or regions only affected by Medicare at age 65. The results in Table (ref) suggest that if we consider the Midwest and South regions, the difference in the discontinuity in the level of access to care and health service utilization for seniors at age 65 is not different from zero at any conventional statistical level. As for Assumptions 6 (i) and (ii), we will assume that no senior will refuse to enroll in Medicare as a result of turning 65 in 2012 or 2014, and we allow for the decision to use the Medicare treatment to be related to returns. Unfortunately, to the best of our knowledge, there is no test for these assumptions for panel data. We assume that they hold in our setting.
Finally, Table (ref) shows fuzzy difference-in-discontinuities estimates for several consecutive groups of years (prior to 2014). All but two of the differences are statistically indistinguishable from zero. These placebo regressions suggest that Medicare has the same effect across age groups in absence of the ACA policy. Overall, the placebo regressions suggest that Assumption 3 is reasonable for our sample.
Panel A of Table (ref) presents the fuzzy difference-in-discontinuities estimates for the effect of the ACA on access to care and health care services utilization for 65-year-old Americans. We consider three self-reported access to care outcomes from the NHIS questionnaires: $(i)$ “During the past 12 months, has medical care been delayed for the individual because of worry about the cost?” (first column) $(ii)$ “During the past 12 months, was there any time when the individual needed medical care but did not get it because the individual could not afford it?” (second column) $(iii)$ Did the individual have at least one doctor visit in the 12 months? (third column). In the last column, we report estimated $\tau_{O}^{FRD}$ values for health care services utilization, specifically individuals' overnight hospital stays in the previous year.
The results show that, overall, individuals who turned 65 in 2014 were 3.6% more likely to delay care due to costs. However, the estimated fuzzy difference-in-discontinuities coefficients on the other two access to care outcomes in columns (2) and (3) are not significant at the standard levels. These results suggest that the effect of the ACA on cost-related access to care is mixed. With respect to health care service utilization, we note a significant 4.8% increase in hospitalization rates for 65-year-olds in 2014. Panel B of Table (ref) shows the effect of the ACA on several cost-related access to care outcomes for individuals at age 65. Overall, the proportion of individuals who reported that they could not afford to pay for prescription drugs, see a specialist or have a follow-up treatment increased by 7.0%, 7.2%, and 5.5%, respectively.
We also perform a subgroup analysis considering ethnicity and education. These results should be treated with caution because identification assumptions may not hold for some subgroups. Table (ref) presents the results by ethnicity. It reveals that for both whites (non-Hispanics) and minorities (blacks or Hispanics), there is no significant ACA effect on access to care or health care services utilization (see the first panel of Table (ref)). Interestingly, we observe a clear heterogeneity in the ACA's effects within individuals' ethnicity. The ACA increased the proportion of blacks aged 65 who had seen a doctor the previous year by 36.7%, and the proportion of whites (non-Hispanic) with a least one hospitalization by 5.1%, but 15.6% more Hispanics forewent access to care the previous year for cost-related reasons. Moreover, panel B of Table (ref) shows that the proportion of whites (non-Hispanics) who could not afford prescription drugs, a specialist visit or follow-up care all increased as a consequence of the ACA. The proportion of Hispanics who could not afford prescription drugs also increased by 23.3%. Panel B in Table (ref) shows that the ACA significantly increased the proportion of high-school-dropout seniors who could not afford a specialist visit or follow-up care, compared to more educated seniors. An additional 11.4% of seniors with a college education could not afford prescription drugs.
The results suggest that, in general, the ACA exacerbated cost-related access barriers for seniors. In 2014, more 65-year-olds delayed care, could not see a specialist, or could not maintain continuity of care due to costs. This might be in part due to the fact that the implementation of the ACA is associated with the increase in Medicare Part B premiums and the reduction of the government's payment per enrollee to private insurance companies. The ACA increased the proportion of seniors who could not afford prescription drugs. This is surprising, since the ACA was set to reduce Medicare Part D enrollees' out-of-pocket expenses. The increase in hospitalization rates might arise from paying physicians under the ACA based on the quality of services provided and penalizing hospitals with high readmission rates. Interestingly, the ACA significantly improved access to physicians' services for blacks, and increased hospital stays for whites (non-Hispanics). However, under the ACA, more Hispanics were unable to access to care for cost-related reasons.
Identifying the effect of the ACA on access to care requires that all other factors that might affect a 65-year-old's access to care trend smoothly card2008impact. An example of a confounding factor is an individual's employment status, since 65 is the typical retirement age, and employed older adults have been found to have better health outcomes than unemployed older adults Kachan2015. This may lead to a biased $\tau_{O}^{FRD}$ if employment status had a significant impact on individuals' health outcomes at the discontinuity (age 65) in 2014.
The estimated effects of the discontinuity at age 65 on employment status are presented in Table (ref). We consider two employment variables: whether an individual is employed, and whether the individual is a full-time employee. The results show non-significant coefficients, which suggests that there are no discontinuities at 65 in both cases. Figure (ref) illustrates the continuity at age 65 for employment. We also perform the same test using different subgroups: ethnicity and education. The results are presented in Tables (ref) and (ref). Again, in all cases the results show no evidence of a discontinuity for employment. We obtain similar results with smaller bandwidths (see Table (ref)). Therefore we rule out employment as a confounding factor when estimating the ACA's effect on access to health care services.
We also check the robustness of the results obtained in the previous section to the inclusion of second-order polynomial terms for age (Tables (ref)- (ref)) and the use of a smaller bandwidth (Tables (ref)-(ref)). Overall, the results are qualitatively similar to those presented in section (ref). Finally, to re-estimate the ACA's effects, we split the sample into two parts: respondents who are enrolled in Medicare Part D, and respondents who are enrolled in Medicare Part A, B or C. The results in Tables (ref) and (ref) show similar patterns for the ACA's effects on access to care.
We implement an additional robustness check by removing individuals who turned 65 in the first half of 2014 from the sample. The motivation for this additional check is that for most of our outcomes, the information we use is from the 12 months prior to when the person was interviewed, and if an individual turned 65 at the beginning of 2014, the effect that we are capturing may be more likely to correspond to an event from 2013 instead of 2014. Tables (ref) shows that for those outcomes related to the alternative measures of access to care, the results are similar to those presented in section (ref), and for the other outcomes, the results are equal in sign but of lesser statistical significance.
The ACA has generated significant media attention since 2009. Evaluating its effects on the U.S. health care system is necessary to inform the debate on the importance of the ACA. We develop and apply an identification strategy in a fuzzy difference-in-discontinuities design to tease out the causal effect of the ACA on the U.S. population's access to care. Our identification results rely on the presence of a pooled cross-sectional or panel dataset to which a “before-and-after" policy evaluation can be applied. The partial effect of the policy of interest --- in our application, the ACA --- can be identified under a strict condition of equality in the treatment probability changes at the cutoff point. For the ACA, this condition is likely to be satisfied for the overall sample. We apply our fuzzy difference-in-discontinuities method to self-reported access to care outcomes, using NHIS data over a three-year period (2012-2014).
Our results show that the ACA had an adverse impact on access to health care services for cost reasons. In particular, under the ACA, the likelihood of delaying care due to cost, and the likelihood of being unable to afford a prescribed drug, a specialist visit, or a follow-up treatment, have increased. These results suggest that an increasing number of seniors (aged 65 or older) reported unmet health care needs because of a lack of financial resources. This should concern policymakers, as people who report unmet health care needs face higher risks of mortality Alonso1997 and of deterioration in their health status Okumura2013.
Two mechanisms might explain why the ACA increased cost-related barriers: it increased the demand for health care services by increasing coverage, and it reduced the supply of health services by replacing a generous, uncritical fee-for-service payment model with a capitation-based model in which care providers are paid a fixed amount for each patient to provide a bundle of pre-determined services.\footnote{ For more details, see the Public Law 111--148--MAR. 23, 2010 at \url{https://www.gpo.gov/fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf} (accessed in September 2018).} Note that a fee-for-service scheme motivates providers to increase the quantity of services provided MCGUIRE2000461. In contrast, capitation creates incentives to underprovide services, and may improve the quality of services ScottA2000. This suggests that along with facilitating access to insurance coverage, the ACA should have included measures or incentives to increase the supply of health services and prevent the increase of insurers' premiums and beneficiaries' out-of-pocket expenses. Our results also show that the ACA improved hospital stays for patients as a result of moving away from a fee-for-service model to a capitation system, which is designed to reward quality instead of quantity.
Our results for access to health care services for the previously insured population may be capturing short-term effects of the ACA. For example, if the ACA successfully increased the quality of care by providing more preventive services, the number of patients per physician might decrease over time, reducing the demand for care and the quantity consumed. This in turn could reduce access to care issues in the long term. Though estimating such effects will require long-term panel data, which do not yet exist, our identification and estimation strategy will still be valid.
\hbox
\hbox