Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
61,630 characters · 19 sections · 66 citation commands
Extrapolating Away from the Cutoff in Regression Discontinuity Designs
}
\singlespacing
Regression discontinuity (RD) designs have been one of the most popular empirical strategies for causal inference and program evaluation using non-experimental data since they were first introduced in thistlethwaite_campbell_1960. In canonical sharp RD designs, each unit takes a value of the running variable. The treatment assignment rule is contingent upon whether the running variable value of a unit surpasses the specified cutoff. A unit receives treatment if its running variable exceeds the threshold; otherwise, the unit remains in the control group. This type of treatment assignment rule generates a discontinuity of the probability of receiving treatment near the cutoff, which researchers leverage to identify treatment effects.
In the continuity framework introduced by hahn_etal_2001, if the potential outcome functions are assumed to be continuous in the running variable at the cutoff, and the density of the running variable near the cutoff is positive, the causal effect of the treatment at the cutoff can be identified and estimated by the difference between the observed outcome functions. RD designs establish strong internal validity of the treatment effect for units with running variable values around the cutoff because, within a local neighborhood around the cutoff, the treatment can be considered “as good as random” lee_2008. However, canonical RD estimates have limited external validity. Without additional assumptions, RD designs say little about the treatment effect for units with the running variable values away from the cutoff. Extrapolating treatment effects away from the cutoff is empirically relevant if researchers seek to understand the heterogeneous treatment effects for units with different running variable values battistin_rettore_2008, angrist_rokkanen_2015, cattaneo_keele_titiunik_2021. Furthermore, in most empirical applications, units near the cutoff do not solely represent the policy-relevant group. Canonical RD treatment effects cannot comprehensively capture the overall effect of a policy. Extrapolating treatment effects away from the cutoff enables researchers to better assess policy implications, such as evaluating the optimality of the status quo policy and gauging welfare adjustments due to changes in the cutoff yata_2021, zhang2022safe.
The fundamental challenge of identifying treatment effects away from the cutoff is that the counterfactual outcome under the alternative treatment status is never observed. An alternative interpretation of RD extrapolation is that it is essentially a missing data problem in the same spirit as the one studied in manski_1989. The key question for the researcher is how to impute the unobserved counterfactual outcome. The goal of this paper is to provide a methodological blueprint to identify treatment effects away from the cutoff in various empirical settings by offering a non-exhaustive list of assumptions on the counterfactual outcome in both single cutoff and multi-cutoff RD designs. These assumptions aim to bound the variation of the unobserved counterfactual outcome using the data sensitivity parameters to achieve set-identified treatment effects away from the cutoff. In contrast to previous literature where strong assumptions were made to get point identification, assumptions discussed in this paper are arguably weaker and the conclusions are more robust to slight violations of the point identification assumptions. It is up to the researcher to specify the appropriate assumption to use, and it should depend on the economic context of each empirical application. It is recommended for the researcher to conduct sensitivity analysis on the choice of the bounding parameter to investigate how causal conclusions change under varying levels of tightness of the assumption. In the simulations, this paper shows that the set-identified treatment effects achieved by imposing reasonable restrictions on the variation of the counterfactual outcome can be informative about the sign of the treatment effect.
The remainder of the paper is structured as follows. Section (ref) discusses the related literature; section (ref) introduces the setup of the problem; section (ref) establishes the identification results under both single and multiple cutoff designs; section (ref) demonstrates how to estimate the identification region; section (ref) presents empirical and simulation results using the proposed approach; lastly, section (ref) provides concluding remarks and discusses potential future directions.
This paper is related to the literature on causal inference, program evaluation, partial identification, and sensitivity analysis. Specifically, it contributes to the methodological literature on nonparametric identification of RD designs. Comprehensive surveys of RD designs can be found in cattaneo_titiunik_2022, choi_lee_2021, choi_lee_2017, imbens_lemieux_2008, vanderklaauw2008. This paper aims to bring ideas in partial identification and sensitivity analysis literature to study the identification of treatment effects away from the cutoff in RD designs.
Without any additional assumptions, canonical RD designs cannot identify treatment effects away from the cutoff. As summarized in cattaneo_keele_titiunik_2021, there are generally two approaches to extrapolate away from the cutoff. One approach is to leverage the availability of dependent variable predictors other than the running variable, such as covariates and pre-treatment outcomes. For example, angrist_rokkanen_2015 identify treatment effects away from the cutoff in single cutoff designs by imposing a conditional independence assumption (CIA) in the spirit of imbens_rubin_2015. Specifically, potential outcomes are assumed to be mean-independent of the running variable conditional on a set of observable covariates. palomba_2023 generalizes the result of angrist_rokkanen_2015 to multiple cutoffs, and provides a data-driven algorithm to search for a vector of covariates that satisfies CIA. Similarly, battistin_rettore_2008 and mealli_rampichini_2012 investigate extrapolation using matching-type estimators under CIA, with a specific focus on fuzzy RD designs. CIA has testable implications when imposed in RD designs angrist_rokkanen_2015. If the properly reweighted matching-type estimates, using the distribution of covariates at the cutoff, and the local RD estimates are similar, it is suggestive that the CIA is valid angrist_rokkanen_2015, battistin_rettore_2008. rokkanen_2015, on the other hand, models the source of omitted variable bias in RD designs using latent factors. Assuming the running variable is one of a number of available noisy measures of the latent factors, causal effects for all values of the running variable can be nonparametrically identified. Furthermore, bennett_2020 exploits predictive covariates to explain the correlation between the running variable and the outcome of interest to extrapolate away from the cutoff.
Another general approach to extrapolate away from the cutoff is to utilize information within the research design. bertanha_imbens_2020 leverage imperfect treatment compliance in fuzzy RD designs and impose that the potential outcomes are independent of compliance types to extrapolate away from the cutoff. dong_lewbel_2015 establish nonparametric identification of the treatment effect derivative (TED), referring to the derivative of the treatment effect with respect to the running variable at the cutoff. Given the estimated TED, they can extrapolate the treatment effect at a running variable value marginally above the cutoff. In the context of multi-cutoff RD designs, cattaneo_keele_titiunik_2021 adopt a “parallel-trend-like” assumption on the unobserved potential outcome functions for subpopulations exposed to different cutoffs for extrapolation. All of the approaches mentioned above impose strong enough assumptions to achieve point identification of the treatment effects away from the cutoff. However, these point identification results are sensitive to slight violations of the identification assumption. Building on these previous ideas, this paper relaxes those assumptions and imposes weaker ones to yield partial identification of the treatment effect of interest.
The partial identification approach, brought to the attention of economists firstly by works of manski_1989, manski_1990, aims to learn about a parameter of interest given the available data and a set of assumptions. Some assumptions are more credible if they are rooted in economic theory or based on the context of the empirical application, such as demand being downward sloping manski_1997. However, some assumptions are less credible if they are imposed only to complete a model or to achieve a desired level of precision of the problem, such as functional form and distributional assumptions molinari_2020, tamer_2010. The philosophy of doing partial identification is to be transparent about the identifying power of different assumptions and how they shape the conclusions molinari_2020, tamer_2010. The assumptions to identify treatment effects away from the cutoff proposed in this paper follow a top-down approach discussed in tamer_2010, and they can be mainly summarized into two types. The first type is the bounded variation assumption directly on the counterfactual outcome. Instead of imposing parametric assumptions, this paper proposes to bound the variation of the counterfactual outcome using assumptions such as Lipchitz continuity and smoothness restrictions kim_etal_2018, kolesar_rothe_2018, yata_2021. Additionally, in multi-cutoff designs, this paper relaxes the “parallel-trend-like” assumption in cattaneo_keele_titiunik_2021 by assuming the violations of parallel trends are bounded manski_pepper_2018,rambachan_roth_2023. The other type of assumption aims to relax the conditional independence assumption that leverages pre-intervention covariates by bounding the distance between the unobserved and the observed distribution of the outcome kline_santos_2013, masten_poirier_2018. In canonical RD designs, the role of additional covariates is mainly for estimation efficiency and falsification testing cattaneo_keele_titiunik_2023covariate, calonico_etal_2019. However, this paper explores how the availability of additional covariates helps with the identification of treatment effects away from the cutoff. All the assumptions in this paper involve sensitivity parameters, which allow the researcher to start with tight assumptions but gradually relax them. Researchers are recommended to conduct sensitivity analysis on the chosen parameter to assess how the resulting conclusions change. In the simulations, this paper imposes a sequence of weaker assumptions and discusses when non-zero treatment effect breaks down.
The setup of the extrapolation problem in this paper follows the conventional regression discontinuity setup in the literature cattaneo_Idrobo_Titiunik_2023, cattaneo_titiunik_2022, Lee_Lemieux_2010, imbens_lemieux_2008, vanderklaauw2008. Assume researchers observe an independently and identically distributed sample $(Y_i, X_i, W_i, C_i, D_i), i = 1, ..., n$, where $Y_i$ is the outcome of interest, $X_i$ is the running variable (score or index), $W_i$ is a vector of observed covariates, $C_i$ is the cutoff (threshold), and $D_i$ is the binary treatment status indicator. Denote the space of covariates as $\mathcal{W}$. Assume the running variable $X_i$ has a continuous positive density $f_X(x)$ on the support $\mathcal{X}$.
The cutoff indicator $C_i$ can be unit-specific, and takes values from a set $\mathcal{C} \subset \mathcal{X}.$ When $\mathcal{C}$ is a singleton, it is the setup of a single cutoff design, where every unit faces the same cutoff, denoted as $c_0$. When $|\mathcal{C}| \ge 2$, different units may be subject to different cutoffs, defining a multi-cutoff design. In particular, when thinking about multi-cutoff designs, this paper focuses on non-cumulative multiple cutoffs, where a unit with running variable $X_i$ can be potentially exposed to any but only one of the cutoffs $c \in \mathcal{C}$ cattaneo_Idrobo_Titiunik_2023, cattaneo_etal_2016. Intuitively, the whole population is divided into different subpopulations based on the cutoff each subpopulation is subject to, while the support of the running variable remains common across all subpopulations. In multi-cutoff designs with non-cumulative cutoffs, it is common to assume all treated units receive the same treatment regardless of their cutoffs cattaneo_Idrobo_Titiunik_2023, cattaneo_etal_2016. For simple exposition, assume there is only one low cutoff and one high cutoff, denoted as $\{\ell,h\}, \ell <h $.
To further simplify the problem, this paper focuses on sharp regression discontinuity designs. Consequently, the treatment assignment rule is expressed as follows: $$D_i = \mathds{1}\{X_i \ge C_i\},$$ which means that a unit receives treatment if and only if its running variable is above the cutoff the unit faces. Furthermore, following the potential outcome framework in imbens_rubin_2015, denote $Y_i(d), d \in \{ 0,1 \}$ as the potential outcomes for unit $i$. Then, the observed outcome is $Y_i = D_iY_i(1) + (1-D_i)Y_i(0).$ Lastly, denote the potential outcome regression functions for units facing cutoff $c$ with running variable value $x$ as $$\mu_{d,c}(x) = \mathds{E}[Y_i(d) \mid X_i = x, C_i = c], \text{ for } d \in \{ 0,1 \}, c \in \mathcal{C}.$$ Additionally, define the observed outcome regression function as $$\mu_{c}(x) = \mathds{E}[Y_i \mid X_i = x, C_i = c], \text{ for }c \in \mathcal{C}.$$ The potential outcome functions conditional on covariates $W_i = w$ can be defined analogously,
The general parameter of interest is the treatment effect of the population (or subpopulation) exposed to cutoff $c$ when the running variable takes the value $x$, which is defined as a function $\tau_c(x): \mathcal{C} \times \mathcal{X} \to \mathds{R}$, such that $$\tau_c(x) = \mathds{E}[Y_i(1) - Y_i(0) \mid X_i = x, C_i = c] = \mu_{1,c}(x) -\mu_{0,c}(x).$$ Assuming continuity of $\mu_{d,c}(x), d \in \{0,1\}$ within a local neighborhood of the cutoff $c$, the canonical sharp RD design can nonparametrically identify the treatment effect at the cutoff hahn_etal_2001. For example, the treatment effect at $c_0$ in the single cutoff design is $$\tau_{c_0}(c_0) = \mathds{E}[Y_i(1) - Y_i(0) \mid X_i = c_0, C_i = c_0] = \lim_{x \to c_0^+}\mu_{c_0}(c_0) - \lim_{x \to c_0^-}\mu_{c_0}(c_0),$$ and in the multi-cutoff design, the treatment effect at the low cutoff $\ell$ and at the high cutoff $h$ can be separately identified as $$\tau_{\ell}(\ell) = \mathds{E}[Y_i(1) - Y_i(0) \mid X_i = \ell, C_i = \ell] = \lim_{x \to \ell^+}\mu_{\ell}(\ell) - \lim_{x \to \ell^-}\mu_{\ell}(\ell),$$ $$\tau_h(h) = \mathds{E}[Y_i(1) - Y_i(0) \mid X_i = h, C_i = h] = \lim_{x \to h^+}\mu_{h}(h) - \lim_{x \to h^-}\mu_{h}(h).$$
When studying extrapolation of RD designs away from the cutoff, the specific target parameter this paper considers is the treatment effect of the treated units that are subject to cutoff $c$ but with running variable values $x$ such that $x > c$.\footnote{Similar approaches can be applied to extrapolate the treatment effect of the untreated units with running variable value below the cutoff by adapting the assumptions to bound the variation of the unobserved treated potential outcome regression functions.} Denote the specific target parameter as $$\theta = g(\tau_c(x)),$$ where $g: \mathcal{F} \to \mathds{R}$ is a known function of $\tau_c(x).$ $\mathcal{F}$ is the functional space that $\tau_c(x)$ belongs to, which later will be specified depending on the restrictions imposed on the potential outcome functions. If the researcher is interested in the treatment effect of units at a particular value of the running variable $x^*$, $g(\cdot)$ is the function such that $$\theta = \tau_c(x^*), \text{ for a particular } x^*.$$ Alternatively, if the researcher is interested in the average treatment effect of treated units with running variable in the range $[a,b], a, b \in \mathcal{X}$ above the cutoff angrist_rokkanen_2015, then $g(\cdot)$ is the integral function such that $$\theta = \int_a^b \tau_c(x) f_{X| x \in [a,b]}(x)dx,$$ where $f_{X| x \in [a,b]}(x)$ is the density of $x$ conditional on $x\in [a,b]$.
This paper focuses on discussing possible assumptions needed to extrapolate to the right of the cutoff, specifically the treatment effects on the treated units with running variable values higher than the cutoff. In single cutoff designs, the target parameter is $g(\tau_{c_0}(x)), x \ge c_0$; and in multi-cutoff designs, the target parameter is the treatment effect of the subpopulation subject to the low cutoff, but with running variable value between $\ell$ and $h$, $g(\tau_{\ell}(x)), x \in (\ell, h)$. The fundamental identification challenge is that the untreated potential outcome functions are never observed. The general approach is to impose restrictions to bound the variation of the untreated potential outcome functions above the cutoff. Section (ref) discusses assumptions in single cutoff designs that can be directly imposed on the evolution of the untreated potential outcome functions. Assumptions in single cutoff designs can also be used for extrapolation in multi-cutoff designs; furthermore, section (ref) discusses additional assumptions that can be imposed in multi-cutoff designs that exploit the existence of other subpopulations subject to different cutoffs.
To restrict the variation of the untreated potential outcome function, this paper proposes to specify $\mu_0(x) \in \mathcal{U}(\kappa)$, where $\mathcal{U}(\kappa)$ satisfy some properties that the researcher deems to be desirable depending on the empirical application, and $\kappa$ is a sensitivity parameter. By construction, the identification region of the treatment effect away from the cutoff is all the possible differences between the observed treated outcome and the restricted untreated potential outcome. Formally,
Next, I discuss some possible choices of $\mathcal{U}$.
One type of restriction is to assume the untreated potential outcome function is Lipschitz continuous with a Lipschitz constant $\kappa$. The Lipschitz continuity constraint bounds the maximum possible change in $\mu_{0,c_0}(x)$ given a shift in the running variable $x$ by one unit yata_2021. If the researcher is willing to additionally assume $\mu_{0,c_0}(x)$ is differentiable, the Lipschitz restriction is imposing the absolute value of the derivative of $\mu_{0,c_0}(x)$ is bounded by $\kappa$. Formally, this restriction is stated as
If the researcher has access to additional pre-intervention covariates, the Lipschitz constraint can also extend to potential outcome functions conditional on these covariates such that
When the covariates $W_i$ are finite and discrete, the canonical sharp RD treatment effect $\tau_{c_0}(x^*)$ at running variable value $x^*$ becomes a weighted average of the conditional RD treatment effects of each subgroup defined by $W_i = w$, with weights related to the conditional distribution of $W_i \mid X_i$ at $x^*$ cattaneo_keele_titiunik_2023covariate. The RD treatment effects conditional on each subgroup can be recovered by running a fully saturated polynomial regression with subgroup indicators, and can potentially be asymptotically more efficient under the correct implementation calonico_etal_2019. When the covariates are continuous and high-dimensional, it may be infeasible to estimate the average treatment effect for each subgroup; instead, a solution is to estimate conditional ATE for a large number of covariate partitions without specifying subgroups a priori using machine learning methods cattaneo_keele_titiunik_2023covariate.
Similarly, in addition to restricting the rate of change, the researcher can impose smoothness restriction on the untreated potential outcome function kolesar_rothe_2018. Assume $\mu_{0, c_0}(x)$ is twice differentiable, define the class of untreated potential outcome functions with bounded second derivatives, $\mathcal{U}^{SD}(\kappa)$, as
Analogously, given the availability of pre-intervention covariates, the researcher can impose the smoothness restriction conditional on covariates $W_i = w.$ The analysis goes through in the same spirit as in section (ref). This type of restriction can be extended to bound higher-order variation of the potential outcome functions with strengthened assumptions on the differentiability of $\mu_{0, c_0}(x).$
Instead of directly restricting the variation of the untreated potential outcome function, another type of popular assumption to extrapolate away from the cutoff is the "ignorability" conditional on observables assumption, which assumes that conditional on observable pre-intervention covariates, the potential outcome is independent or mean independent of the running variable angrist_rokkanen_2015, mealli_rampichini_2012, battistin_rettore_2008. Another interpretation of the conditional independence assumption is that the untreated potential outcome above the cutoff is missing at random. The "ignorability" conditional on observables is untestable, making its resulting causal conclusion susceptible to even small violations of this assumption. Instead of imposing that conditional independence holds exactly, this paper proposes to bound the deviation from conditional independence, or equivalently, the nonignorable selection on observables.
Following similar ideas in kline_santos_2013 and masten_poirier_2018, this paper bounds the deviation from conditional independence using the maximal Kolmogorov-Smirnov distance. Firstly, notice that the untreated potential outcome function is observed for $x \le c_0$. Next, define the probability of $\mu_{0,c_0}(x,w)$ being observed as
The distribution of interest is the distribution of $Y(0)$ conditional on covariate $W= w$ , denoted as $F_{Y(0) \mid w}(y) = P(Y(0) \le y \mid W = w)$. The researcher observes this distribution for $x \le c_0$, denoted as $F_{Y(0) \mid 0,w}(y) = P(Y \le y \mid X \le c_0, W = w)= P(Y \le y \mid D = 0, W = w)$. However, the researcher does not observe the distribution of $Y(0)\mid w$ for $x>c_0$, denoted as $F_{Y(0) \mid 1,w}(y) = P(Y \le y \mid X > c_0, W = w)= P(Y \le y \mid D = 1, W = w)$. The Kolmogorov-Smirnov (KS) distance is defined as
The conditional independence assumption is stated as $Y(0) \bot X \mid W$, which implies $F_{Y(0) \mid 0,w}(y) = F_{Y(0) \mid 1, w}(y)$; or equivalently, the KS distance between $F_{Y(0) \mid 0,w}(y) $ and $F_{Y(0) \mid 1,w}(y)$ is zero. Instead of assuming the KS distance is exactly zero, this paper assumes that the distribution of $Y(0) \mid 1,w$ belongs to the class of distributions with no more than $\kappa$ KS distance away from the observed distribution $Y(0) \mid 0,w$, denoted as
where $\kappa$ is a sensitivity parameter.
kline_santos_2013 discuss one intuitive interpretation of the sensitivity parameter $\kappa$ in this case. To put in perspective, consider $F_{Y(0) \mid 1,w}(y)$ as a mixture model such that
where $\Tilde{F}_{Y(0) \mid w}$ is any unknown distribution. Then, the KS distance can be re-written as
Hence, imposing $ F_{Y(0) \mid 1,w}(y) \in \mathcal{Y}(\kappa)$ is equivalent to assuming that there is at most $\kappa$ fraction of the unobserved untreated potential outcome not well represented in the observed data distribution conditional on covariates. Connecting to the foundational work of manski_2003, bounds on $F_{Y(0) \mid w}(y) $ can be derived using the law of total probability,
Given bounds on $F_{Y(0) \mid w}(y)$ and the specified support of $Y(0)$, the researcher can back out bounds on the expectation of $Y(0)$ at a particular value of the running variable conditional on covariates, and therefore partially identify the treatment effect away from the cutoff.
This section focuses on discussing the possible extrapolation assumptions in multi-cutoff RD designs. Firstly, it is worth noting that to extrapolate away from the cutoff in non-cumulative multi-cutoff designs, all the assumptions for single cutoff designs still apply because researchers can view the extrapolation problem for each subpopulation subject to different cutoffs as a separate single cutoff extrapolation problem. Additionally, due to the existence of other subpopulations with different cutoffs, there is more information that researchers can leverage for extrapolation.
cattaneo_keele_titiunik_2021 use a “parallel-trend-like” assumption to extrapolate away from the cutoff in multi-cutoff designs. Without loss of generality, consider two cutoffs, $\{ \ell, h \}$, with the goal to extrapolate the treatment effects, $g(\tau_{\ell}(x))$, for the subpopulation subject to cutoff $\ell$ but with running variable value $x \in (\ell, h)$. Refer to Figure (ref) as an example. Standard multi-cutoff RD designs can nonparametrically identify $\tau_{\ell}(\ell) = \lim_{x \to \ell^+}\mu_{\ell}(\ell) - \lim_{x \to \ell^-}\mu_{\ell}(\ell)$ and $\tau_{h}(h) = \lim_{x \to h^+}\mu_{h}(h) - \lim_{x \to h^-}\mu_{h}(h)$, which are the $de$ and $gh$ segment in Figure (ref). The identification challenge is that $\mu_{0,\ell}(x), x \in (\ell,h)$, is never observed. Without loss of generality, consider identifying the treatment effect at $x^*.$ What the researcher can observe in the data is the difference between the treated outcome function for units facing the lower cutoff and the untreated outcome function for units facing the higher cutoff at $x^*$, denoted as $\mu_{1,\ell}(x^*) - \mu_{0,h}(x^*)$, which corresponds to $ak$ in Figure (ref). If the difference between the untreated potential outcomes for units facing the lower cutoff, $\mu_{0,\ell}(x^*)$, and the untreated potential outcome for units facing the higher cutoff, $\mu_{0,h}(x^*)$, can be quantified, the researcher can identify treatment effect at $x^*$, denoted as $$\tau_{\ell}(x^*) = \Big(\mu_{1,\ell}(x^*) - \mu_{0,h}(x^*)\Big) - \Big(\mu_{0,\ell}(x^*) - \mu_{0,h}(x^*)\Big) = \mu_{1,\ell}(x^*) -\mu_{0,\ell}(x^*).$$
The difference between untreated potential outcomes for units with the same running variable value but different cutoffs can be interpreted as bias driven by both observable and unobservable predictors of the outcome other than the running variable across subpopulations cattaneo_keele_titiunik_2021. Formally, bias is defined as the following.
Under assumption (ref), where the potential outcome functions are continuous not only at the cutoff but also over the whole extrapolation interval $[\ell, h]$, and assumption (ref), which is equivalent to imposing $\mu_{0,\ell}(x)$ and $\mu_{0,h}(x)$ are parallel, cattaneo_keele_titiunik_2021 establish point identification of $\tau_{\ell}(x^*)$ as
Notably, this constant bias assumption is similar to the parallel trend assumption in difference-in-difference (DiD) designs, both of which are untestable assumptions. Even if the researcher observes that the untreated potential outcome functions across subpopulations are parallel for running variable values below the low cutoff, this does not guarantee that the bias remains constant above the low cutoff. If the true bias is not exactly constant, the identification result fails. This issue resembles the pre-trend testing concerns raised by roth_2022.
To derive more robust identification results, this paper aims to weaken the constant bias assumption. Instead of assuming the bias is exactly constant, this paper assumes that the variation of the unobserved bias can be bounded based on the bias observed in the data manski_pepper_2018, rambachan_roth_2023. This paper assumes that the bias function $B \in \mathscr{B}$, where $\mathscr{B}$ is a collection of functions satisfying some desired restrictions. To study the identification problem, this paper highlights the following causal decomposition of the observables in the data. Firstly, define the observables as a piece-wise function $\gamma(x)$, where $$\gamma(x) =
$$ For an illustration, see Figure \ref{fig:multi}. For $x \le \ell$, both subpopulations are untreated and the researcher observes the difference between the untreated potential outcome functions. For $x \in (\ell, h)$, the subpopulation exposed to cutoff $\ell$ is treated, but the subpopulation exposed to cutoff $h$ is not. Lastly, for $x \ge h$, both subpopulations are treated. Since I only consider extrapolating treatment effects in the region $(\ell, h)$, I focus the analysis on $x <h$ onward. Then the observables can be decomposed as the sum of the causal parameter of interest and a bias term:
where $\tau_{\ell}(x) = \mu_{1,\ell}(x) - \mu_{0,\ell}(x),$ and $B(x) = \mu_{0,\ell}(x) - \mu_{0,h}(x)$, as defined before. If the running variable is below the cutoff $\ell$, no treatment is given; therefore, the causal parameter of interest is mechanically zero, and all that is observed in the data is the bias. If the running variable exceeds the cutoff, the difference between observed outcome functions is the sum of the causal effect and the bias.
The partially identified treatment effect $\theta$ is the set of values for $\theta$ that are consistent with the observables, $\gamma(x)$, under the restriction $B\in \mathscr{B}$,
The class of bias functions $\mathscr{B}$ needs to be specified by the researcher, and the choice of $\mathscr{B}$ should depend on the context of each empirical application and the available data, $\gamma(x), x \le \ell$. Under this framework, this paper discusses several restrictions researchers can impose based on the idea of bounded variation in manski_pepper_2018 and rambachan_roth_2023.
In some empirical applications, researchers may be willing to assume that the magnitude of the bias across running variable values has bounded variation. Intuitively, this assumption suggests that in the data, the bias function oscillates within a band. One possible restriction is to impose the bias function $B \in \mathscr{B}^{AM}(\kappa)$ for $\kappa \ge 0$, where
with $\bar{B} = \max_{\Tilde{x} \le \ell}|B(\Tilde{x})| = \max_{\Tilde{x} \le \ell}|\gamma(\Tilde{x})|$ being a constant that can be estimated and $\kappa$ as a sensitivity parameter. When the researcher is willing to believe that the absolute magnitude of the bias for the running variable taking values above the threshold is no bigger than the biggest absolute magnitude bias they observe (bias for the running variable taking values below the threshold), a natural benchmark is to let $\kappa = 1.$
In some applications, it may be reasonable to assume the rate of change of the bias above cutoff $\ell$ is close to the rate of change of the bias below $\ell$. Then, the researcher can impose restrictions to bound the first derivatives of $B$. Firstly, assume $B$ is continuously differentiable; equivalently, $\mu_{d,\ell}(x), d \in \{0,1\}$, is continuously differentiable. Then the class of bias functions with bounded first derivatives belongs to $\mathscr{B}^{RM}(\kappa)$ for sensitivity parameter $\kappa \ge 0$, where
and $\Bar{B'} = \max_{\Tilde{x} \le \ell}|B'(\Tilde{x})| = \max_{\Tilde{x} \le \ell}|\gamma'(\Tilde{x})|. $ In particular, if the researcher observes the bias below $\ell$ to be approximately linear, it may be reasonable to let $\kappa = 1$ and assume the deviation from the linear trend of the bias above $\ell$ is not too large.
Depending on the context of the problem, even though the trend of the bias function is not linear, it may still be reasonable to assume that the evolution of bias is smooth after passing the lower threshold. Assume that $B$ is twice continuously differentiable; equivalently, $\mu_{d,\ell}(x), d \in \{0,1\}$ is twice continuously differentiable. The class of smooth bias functions, $\mathscr{B}^{SD}(\kappa)$, given sensitivity parameter $\kappa \ge 0$, can be defined as
where $\Bar{B''} = \max_{\Tilde{x} \le \ell}|B''(\Tilde{x})| = \max_{\Tilde{x} \le \ell}|\gamma''(\Tilde{x})|. $ This restriction imposes that the unobserved second derivatives of the bias are not bigger than $\kappa$ times the maximal second derivative observed in the data.
A generalization of the restrictions mentioned above is to bound higher-order variation of the bias function. Assume $B$ is $p$-times continuously differentiable; equivalently, $\mu_{d,\ell}(x), d \in \{0,1\}$ is $p$-times continuously differentiable, for some $p \in \{0,1,2, \cdots\}$. As shown in the appendix of cattaneo_keele_titiunik_2021, the bias functions can be approximated using a polynomial expansion as
where $B^{(s)}(x) = \mu^{s}_{0,\ell}(x) - \mu^{s}_{0,h}(x)$ and $\mu^{s}_{d,c}(x) = \frac{\partial^{s} \mu_{d,c}(x) }{\partial x^s}$. The researcher needs to specify $p$, and the choice of $p$ can depend on the optimal polynomial order approximation of the observed bias function for $x \le \ell.$ Then, the class of bias functions with bounded higher order derivatives, for $s \in \{0,1, \cdots, p\}$ and given sensitivity parameters $\kappa_s \ge 0$, can be defined as
where $\kappa_s$ are sensitivity parameters that can vary by $s$, and $\Bar{B}^{(s)} = \max_{\Tilde{x} \le \ell}|B^{(s)}(\Tilde{x})| = \max_{\Tilde{x} \le \ell}|\gamma^{(s)}(\Tilde{x})|$ is the maximum $s$-th derivative of $B(x)$ for $x \le \ell.$ Researchers can also let $\Bar{B}^{(s)} = 1$ to directly impose the $s$-th derivative of the bias function is no bigger than a constant $\kappa_s$.
Another way to incorporate all the assumptions discussed so far is to specify $\mathscr{B}^{IB}(\kappa)$ to be any intersection of the classes of bias functions mentioned above. The bounds of the identification region achieved by imposing $B \in \mathscr{B}^{IB}(\kappa)$ will be the intersection bounds of the identification region achieved by imposing $B$ belongs to each of the class of functions that defines $\mathscr{B}^{IB}(\kappa)$. For example, the researcher may be willing to impose that both the absolute and relative magnitudes of the bias function are bounded in the sense discussed in section (ref) and (ref). Assuming that $B$ is continuously differentiable in this case, the class of bias functions belongs to $\mathscr{B}^{IB}(\kappa) \equiv \mathscr{B}^{AM}(\kappa) \cap \mathscr{B}^{RM}(\kappa) $, for sensitivity parameter $\kappa \ge 0$, where
with $\bar{B}$ and $\Bar{B'}$ defined similarly as before.
The identification region given the restrictions discussed above is easy to estimate using nonparametric methods, such as series regression and kernel density estimation. Under some restrictions, it is also possible to derive a closed-form characterization of the identification region. As an example, I discuss estimating the identification region in multi-cutoff designs under the bounded absolute and relative magnitudes assumption in this section, which will be implemented in section (ref) to showcase the empirical illustration.
To start off, let the parameter of interest be the treatment effect at a particular running variable value $x^*$ above the low cutoff, denoted as $\tau_{\ell}(x^*)$. Imposing $B \in \mathscr{B}^{AM}(\kappa)$ is essentially putting restrictions on $\mu_{0,\ell}(x)$ such that
where $\kappa$ is the sensitivity parameter specified by the researcher, and $\mu_{0,h}(x)$ and $\bar{B}$ can both be estimated from the data using series regression. For the empirical illustration in section (ref), this paper uses the power basis $x^{j-1}, j \in \{1, 2, \cdots, J\}$. The optimal order of the polynomials, $J^*$, is chosen to be the one that minimizes the leave-one-out cross-validation criterion, which is shown to be a nearly unbiased estimator of mean-squared-forecast-error (MSFE) hansen_2014. Hence, the estimated bounds on $\mu_{0,\ell}(x)$ at $x^*$ are simply the ones with the estimated $\hat{\mu}_{h}(x^*)$ and $\hat{\bar{B}}$,
Given the bounds on $\mu_{0,\ell}(x)$, and the construction of the identification region, the estimated treatment effect $\tau_{\ell}(x^*)$ is
where $\hat{\mu}_{\ell}(x^*)$ is estimated in the same way using series regression with polynomial basis functions.
Similarly, by imposing $B \in \mathscr{B}^{RM}(\kappa)$, $B(x^*)$ attains its maximum above $\ell$ when $B(x)$ strictly increases at the rate of $\kappa \bar{B}'$, and attains its minimum above $\ell$ when $B(x)$ strictly decreases at the rate of $\kappa \bar{B}'$. Hence, the estimated bounds on the bias are the following
where all quantities are either estimated or given by the researcher. Lastly, by construction, the identification region is
Following the same logic, closed-form bounds of the identification region under high-order restrictions can be attained by backward induction.
In this section, I present two examples of the partially identified treatment effect away from the cutoff in multi-cutoff RD designs. The first example uses the empirical application in cattaneo_keele_titiunik_2021. The second example uses the simulated data set 1 in the rdmulti package.\footnote{The simulated data set can be accessed at \href{https://rdpackages.github.io/rdmulti/}{rdpackages.github.io/rdmulti}.} In both examples, I extrapolate the treatment effect to one particular running variable value above the lower cutoff, and I compare the identification results when imposing the bounded absolute magnitudes and the bounded relative magnitudes restriction with varying values of the sensitivity parameter, discussed in section (ref) and (ref).
ACCES is a subsidized loan program administered by the Colombia Institute for Educational Loans and Studies Abroad (ICETEX) to provide tuition credits to underprivileged populations. The eligibility criterion for this program depends on whether an individual's exam position score on a high school exit test, known as SABER 11, passes the specified cutoff. The position score of a student is their ranking out of 1000 quantiles of the SABER 11 scores among all students who took the exam that semester. To be consistent with the conventional RD treatment assignment rule, where treatment is granted if the running variable exceeds the cutoff, the SABER 11 position score is multiplied by $-1$. In 2008, the eligibility cutoff was set at $-850$, while in 2009, the cutoff was raised to $-571$. Even though the eligibility cutoff changed, the amount of ACCES credit was the same across the years. This policy change establishes a non-cumulative multi-cutoff RD setup, and the extrapolation goal is to identify the treatment effect for units subject to the lower cutoff $-850$, but with running variable values lying between $-850$ and $-571$, for instance, $-650$. The key identifying assumption for extrapolation in cattaneo_keele_titiunik_2021 is the difference between the untreated potential outcome functions, defined as bias, for the two subpopulations subject to the two cutoffs is constant within the extrapolation interval (Assumption (ref)). Under this assumption, cattaneo_keele_titiunik_2021 obtain a point estimate of $0.191$ for the treatment effect for units with running variable value $-650$. Furthermore, cattaneo_keele_titiunik_2021 conduct two falsification tests to demonstrate the validity of the constant bias assumption in this application. The first one uses global polynomial regressions to test globally if the untreated potential outcome functions for two subpopulations are parallel below the low cutoff based on the regression model
where $\mathbf{r}_p(X_i)$ is a vector of polynomials of $X_i$ up to the $p-$th order cattaneo_keele_titiunik_2021. The null hypothesis that corresponds to the validity of parallel trends is $\boldsymbol\delta = \mathbf{0}$. The second test uses nonparametric local polynomial methods to test the equality of the derivatives of the untreated potential outcome functions.
Under the bounded absolute magnitude assumption, all the estimated identified sets in Figure (ref) cover zero, which indicates no evidence to rule out zero treatment effect. Since the bias in this application is likely to be constant, restricting the magnitude of the unobserved bias to be bounded by the absolute magnitude of the maximum observed bias is not binding and does not have enough identifying power. As a sanity check, it is worth noting that in this setup, the case of constant bias corresponds to the upper bound when $\kappa=1$, which is $0.19$, and it aligns with the point estimate in cattaneo_keele_titiunik_2021. Figure (ref) shows the identified sets under the bounded relative magnitude restriction. When $\kappa$ is relatively small, specifically when $\kappa = 1,$ the result shows that the partially identified set is strictly above zero, suggesting a positive treatment effect. Additionally, the point estimate $0.19$ is contained in the identification region for $\kappa = 1.$ Figure (ref) shows the identified sets under both the bounded absolute and bounded relative magnitudes restriction. They have tighter bounds on the treatment effect of interest because they are the intersections of the bounds attained under each restriction. The overall result is coherent because if the true bias is constant, it requires tight restrictions on the rate of change of bias to be able to rule out zero treatment effect.
In the next example, I extrapolate away from the cutoff in a simulated multi-cutoff design to demonstrate how the identifying power of the restrictions manifests in cases where bias is not constant. The data generating process is defined as follows. Firstly define auxiliary random variables $\dot{X}_i$ and the cutoffs $\dot{C}_i$ as
and assign half of the draws of $\dot{X}_i$ to $\dot{C}_i = 0.33$, and the rest to $\dot{C}_i = 0.66$. If $\dot{C}_i = 0.33$, generate the outcome $Y_{\ell i}$ as
If $\dot{C}_i = 0.66$, generate the outcome $Y_{h i}$ as
Lastly, scale up $\dot{X}_i$ and the cutoffs $\dot{C}_i$ by 100 as the running variable and the cutoffs,
Figure (ref) is a plot of the simulated data with 2000 draws from the DGP described above. The support of the running variable is $[0,100]$, and the two cutoffs are $\{33, 66\}$. The goal is to extrapolate the treatment effect for units subject to the cutoff $33$ but with a running variable value of $50$.
Figure (ref) plots the estimated bias below the lower cutoff, which appears to follow a linear trend. Since the estimated bias is negative, a downward linear trend suggests that the difference between the untreated potential outcomes for the two subpopulations increases as $x$ increases. I run the two falsification tests of parallel trends discussed in cattaneo_keele_titiunik_2021. Table (ref) shows the results by running a global polynomial regression with maximal polynomial order of 2, which does not reject the null hypothesis of parallel trends because the coefficients of the interaction terms of $\mathds{1}\{C = h\}$ and polynomial orders of $X$ are not statistically significant. Table (ref) reports the results employing local polynomial methods. The difference between the derivatives of the untreated potential outcome functions for the two subpopulations is not statistically different from zero, which also fails to reject the parallel trend assumption. However, the true bias under this DGP is
which is clearly not constant over $x \le 33.$
In this case, the data fails to detect the non-constant bias below the lower cutoff. Hence, the treatment effect obtained using the approach in cattaneo_keele_titiunik_2021 is simply not valid. However, applying the method in this paper will give valid bounds on the treatment effect of interest. In this case, it is reasonable to assume bounded variation in the rate of change of the bias function for $x \in (\ell, h)$. As shown in Figure (ref), most of the identified sets are strictly above zero, indicating a positive treatment effect at $x = 50.$ The bounded relative magnitudes restriction imposes that the slope of the unobserved bias is not drastically different from the approximately constant slope observed in the data. Even with the assumption that the slope could be three times the magnitude of the observed slope, the resulting identified set remains strictly positive. Therefore, in this case, the researcher can conclude with greater confidence that the treatment effect at $x=50$ is indeed positive. In contrast, almost all of the identified sets under the bounded absolute magnitude restriction in Figure (ref) cover zero. Intuitively, restricting the absolute magnitude of the bias function does not capture the linear trend in bias across the running variable, and has limited identifying power, which is consistent with the observation that the bounds under the bounded absolute magnitudes assumption are much wider. Figure (ref) shows the identified sets by taking intersections of the bounds obtained under each restriction. Since the bounds under the bounded absolute magnitudes restriction is so wide, the intersection bounds almost perfectly overlap with the bounds under the bounded relative magnitudes restriction.
This paper studies the identification problem of extrapolating the treatment effects away from the cutoff in regression discontinuity designs, where the primary challenge for identification lies in quantifying the unobserved counterfactual outcome. Instead of relying on strong assumptions that point-identify the treatment effect but may result in identification failure in the case of slight violations, this paper proposes a non-exhaustive list of bounded variation assumptions on the evolution of the counterfactual outcome function along the running variable. This approach achieves partial identification of the treatment effect of interest and encompasses both single and multiple cutoff designs. Through simulations, the paper demonstrates that by applying reasonable restrictions based on the specific empirical setting, researchers can derive credible and informative bounds of the treatment effect away from the cutoff.
There are two primary directions for future research stemming from this paper. One ongoing effort involves establishing the statistical properties of the identification region and conducting inference, which will be built on previous work by imbens_manski_2004 and stoye_2009. Another direction pertains to exploring the policy implications that arise from partially identified treatment effects away from the cutoff. This could involve investigating whether the status-quo policy is optimal and whether policymakers should consider adjusting the cutoff in order to maximize overall social welfare.