Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
110,896 characters · 9 sections · 66 citation commands
Identification in Nonparametric Models for Dynamic Treatment Effects
This paper develops a nonparametric model that represents how sequences of outcomes and treatment choices influence one another in a dynamic manner. Often, treatments are chosen multiple times over a horizon, affecting a series of outcomes. Examples are medical interventions that affect health outcomes, educational interventions that affect academic achievements, job training programs that affect employment status, or online advertisements that affect consumers\textquoteright preferences or purchase decisions. Agents endogenously make decisions of receiving treatments, e.g., whether to comply with random assignments. The relationship of interest is dynamic in the sense that the current outcome is determined by past outcomes as well as current and past treatments, and the current treatment is determined by past outcomes as well as past treatments. Such dynamic relationships are clearly present in the aforementioned examples. A static model misrepresents the nature of the problem (e.g., nonstationarity, state dependence, learning) and fails to capture important policy questions (e.g., optimal timing and schedule of interventions).
In this setting, we are interested in identifying the dynamic causal effect of a sequence of treatments on a sequence of outcomes or on a terminal outcome that may or may not be of the same kind as the intermediate outcomes. We are interested in learning about the average of the outcome in each period, had a particular treatment sequence been assigned up to that period, which defines the potential outcome in this dynamic setting. We are also interested in the average treatment effects (ATE's) and the transition-specific ATE's defined based on the average potential outcome, unconditional and conditional on the previous outcomes, respectively. For example, one may be interested in whether the success rate of a particular outcome (or the transition probability) is larger with a sequence of treatments assigned in relatively later periods rather than earlier, or with a sequence of alternating treatments rather than consistent treatments. The treatment effect is said to be dynamic, partly because the effect can vary depending upon the period of measurement, even if the same set of treatments is assigned. Lastly, we are interested in the optimal treatment regimes, namely, sequences of treatments that maximize the (weighted) sum of the average potential outcomes, possibly less the cost of the treatments. For example, a firm may be interested in the optimal timing of advertisements that maximizes its aggregate sales probabilities over time, or a sequence of educational programs may be aimed to maximize the college attendance rate. We show that the optimal regime is a natural extension of a static object commonly sought in the literature, namely, the sign of the ATE. Analogous to the static environment, knowledge about the optimal treatment regime may have useful policy implications. For example, a social planner may wish to at least exclude specific sequences of treatments that are on average suboptimal.
Dynamic treatment effects have been extensively studied in the biostatistics literature for decades under the counterfactual framework with a sequence of treatments (robins1986new,robins1987graphical,robins1997causal, murphy2001marginal, murphy2003optimal, among others). In this literature, the crucial condition used to identify the average potential outcome is a dynamic version of a random assignment assumption, called the sequential randomization. This condition assumes that the treatment is randomized in every period within those individuals who have the same history of outcomes and treatments.\footnote{This assumption is also called sequential conditional independence or sequential ignorability. In the econometrics literature, vikstrom2018bounds consider treatment effects on a transition to a destination state, and carefully analyze what the sequential randomization assumption can identify in the presence of dynamic selection.} This assumption is only suitable in experimental studies with the perfect compliance of subjects, which is often infeasible (robins1994correcting,robins2004estimation). When interventions continue for multiple periods as in the examples described above, non-compliance may become more prevalent than in one-time experiments, e.g., due to the cost of enforcement or the subjects' learning. In addition to partial compliance in experimental settings, sequential randomization is invalid in many observational contexts as well.
The main contribution of this paper is to relax the assumption of sequential randomization widely used in the literature by establishing a flexible choice-theoretic framework for a sequence of endogenous treatments. To this end, we consider a simple nonparametric structural model for a dynamic endogenous selection process and dynamic outcome formation. In this model, individuals are allowed not to fully comply with each period's assignment in experimental settings, or are allowed to make an endogenous choice in each period as in observational settings. The heterogeneity in each period's potential outcome is given by recursively applying a switching-regression type of models with a sequential version of rank similarity. The joint distribution of the full history of unobservable variables in the outcome and treatment equations is still flexible, allowing for arbitrary forms of treatment endogeneity as well as serial correlation. Relative to the counterfactual framework, the dynamic mechanism is clearly formulated using this structural model, which in turn facilitates our identification analysis.
We show that the average potential outcome, or equivalently, the average recursive structural function (ARSF) given the structural model we introduce, is identified under a two-way exclusion restriction. That is, we assume there exist (possibly binary) instruments excluded from the outcome-determining process and exogenous variables excluded from the treatment-selection process. A leading example of the former is a sequence of randomized treatment assignments or randomized encouragements (sexton1984clinical) from, e.g., clinical trials, field experiments, and A/B testings, and other examples include sequential policy shocks. Examples of the latter include factors that agents cannot fully anticipate when making treatment or compliance choices but that determine the outcome. We show that such timing can be justified in this dynamic context, and some covariates in the outcome process may be valid candidates. Identification in nonseparable triangular models using this exclusion restriction is pioneered by VY07 and subsequently appears in SV11 and Han2018 among others, all in static settings. The dynamic structure introduced in this paper poses added challenges in using a similar strategy, since (i) the outcome and treatment structural functions depend on the vectors of lags, which in turn make each potential outcome a direct function of all the previous potential outcomes, (ii) the period specific knowledge analogous to that in VY07 does not directly recover any meaningful objects of interest in general, (iii) rank invariance substantially restricts heterogeneity in this dynamic setting, and (iv) the initial condition problem is present. In this paper, we address these challenges and show how to achieve identification. In particular, we introduce sets of unobservable vectors across periods as a simple way to express potential outcomes in the presence of complicated dynamics. We then recover period specific knowledge using the exclusion restriction, which is then iteratively incorporated across periods for identification by means of mathematical induction, obeying the recursive structure of the potential outcome. In doing so, we introduce sequential rank similarity which substantially weakens the naive rank similarity or rank invariance. The proof is constructive and provides a closed form expression for the ARSF. The identification of each period's ARSF allows us to point identify the ATE's and the optimal treatment regimes. In this paper, we also consider cases where the two-way exclusion restriction is violated in the sense that only a standard exclusion restriction holds or that the variation of the exogenous variables is limited. In these cases, we can calculate the bounds on the parameters. As an extension of our results, we consider another empirically relevant situation where treatments do not appear in every period, while outcomes are constantly observed. We show that the parameters of interest and the identification analysis can be easily modified to incorporate this situation.
This paper contributes to growing research on the identification of the effects of dynamic endogenous treatments that allows for treatment heterogeneity. cunha2007identification and heckman2007dynamic consider a semiparametric discrete-time duration model for the choice of the treatment timing and associated outcomes. Building on these works, heckman2016dynamic consider not only ordered choice models but also unordered choice models for up-or-out treatment choices.\footnote{As related works, the settings of angrist1995two, jun2016multiple, and lee2016identifying for multiple (or multi-valued) treatment effects may be applied to a dynamic setting. Also, see abbring2007econometric for a survey on dynamic treatment effects.} An interesting feature of their results is that dynamic treatment effects are decomposed into direct effects and continuation values. As an important feature, these papers consider attrition based on the irreversible treatment decisions; see also sasaki2015heterogeneity. Similar to our approach, heckman2007dynamic and heckman2016dynamic utilize exclusion restrictions. Unlike these papers, however, we do not necessarily invoke infinite supports of each period's exogenous variables but instead use the two-way exclusion restriction. abraham2018estimating, athey2018design, and callaway2018difference extend a difference-in-differences approach to dynamic settings without specifying fixed-effect panel data models. They consider the effects of treatment timing on the treated, where the treatment process is irreversible as in the previous works. Unlike all the papers mentioned in this paragraph, we consider nonparametric dynamic models for treatment and outcome processes with a general form of evolution, where the processes can freely change states. These models can include an irreversible process as a special case. Moreover, we consider different identifying assumptions than those in the previous works and focus on the identification of the ATE's and related parameters.
This paper's structural approach is only relative to the counterfactual framework of Robins. A fully structural model of dynamic programming is considered in the seminal work by rust1987optimal and more recently by, e.g., blevins-2014 and buchholz2016semiparametric. This literature typically considers a single rational agent's optimal decision, whereas we consider a large group of heterogenous agents with no assumptions on agents' rationality or strong parametric assumptions. Most importantly, our focus is on the identification of the effects of treatments formed as agents' decisions. The agnostic approach of this paper is, in spirit, similar to heckman2007dynamic and heckman2016dynamic, in that we remain flexible for the economic and non-economic components of the model. Lastly, torgovitsky2016partial extends the literature on dynamic binary response models (with no treatment) by considering a counterfactual framework without imposing parametric assumptions. In his framework, the lagged outcome plays the role of a treatment for the current outcome, and the “treatment effect” captures the state dependence. Here, we consider the effects of the treatments on the outcomes, and introduce a selection equation for each treatment as an important component of the model. As an extension of our analysis, we identify the transition-specific ATE, which is related to the effect of a treatment on the state dependence.
In the next section, we first introduce Robins's counterfactual outcome framework and discuss sequential randomization. Section (ref) introduces the main structural model of this paper with parameters of interest, followed by a motivating example in Section (ref). The main identifying conditions and identification results are present in Section (ref), and several extensions are discussed in Sections (ref)\textendash (ref). Section (ref) briefly concludes. In the Appendix, all the proofs are collected and estimation and inference are discussed.
In terms of notation, let $\boldsymbol{W}^{t}\equiv(W_{1},..,W_{t})$ denote a row vector that collects r.v.'s $W_{t}$ across time up to $t$, and let $\boldsymbol{w}^{t}$ be its realization. Note $\boldsymbol{W}^{1}=\boldsymbol{W}_{1}$. We sometimes write $\boldsymbol{W}\equiv\boldsymbol{W}^{T}$ for convenience. For a vector $\boldsymbol{W}$ without the $t$-th element, we write $\boldsymbol{W}_{-t}\equiv(W_{1},...,W_{t-1},W_{t+1},...,W_{T})$ with realization $\boldsymbol{w}_{-t}$. More generally, let $\boldsymbol{W}_{-}$ with realization $\boldsymbol{w}_{-}$ denote some subvector of $\boldsymbol{W}$. Lastly, for r.v.'s $Y$ and $W$, we sometimes abbreviate $\Pr[Y=y|W=w]$ and $\Pr[Y=y|W\in\mathcal{W}]$ to $\Pr[Y=y|w]$ (or $P[y|w]$) and $\Pr[Y=y|\mathcal{W}]$, respectively.
We first introduce Robins's counterfactual framework and state the assumption of sequential randomization commonly used in the biostatistics literature (robins1986new,robins1987graphical, murphy2001marginal, murphy2003optimal). For a finite horizon $t=1,...,T$ with fixed $T$, let $Y_{t}$ be the outcome at $t$ with realization $y_{t}$ and let $D_{t}$ be the binary treatment at $t$ with realization $d_{t}$. The underlying data structure is panel data with a large number of cross-sectional observations over a short period of time (and the cross-sectional index $i$ suppressed throughout, unless necessary). We call $Y_{T}$ a terminal outcome and $Y_{t}$ for $t\le T-1$ a intermediate outcome.\footnote{The terminal period $T$ may be an administrative end of follow-up time.} Let $\mathcal{Y}$ and $\mathcal{D}\subseteq\{0,1\}^{T}$ be the supports of $\boldsymbol{Y}\equiv(Y_{1},...,Y_{T})$ and $\boldsymbol{D}\equiv(D_{1},...,D_{T})$, respectively. There can be other time-varying covariates present in this setup, but we omit them here.
Consider a treatment regime $\boldsymbol{d}\equiv(d_{1},...,d_{T})\in\mathcal{D}$, which is defined as a predetermined hypothetical sequence of interventions over time, i.e., a sequence of each period's decisions on whether to treat or not, or whether to choose treatment $A$ or treatment $B$.\footnote{This is called a nondynamic regime in the biostatistics literature. A dynamic regime is a sequence of treatment assignments, each of which is a predetermined function of past outcomes. A nondynamic regime can be viewed as its special case, where this function is constant. See, e.g., murphy2001marginal,murphy2003optimal for related discussions.} Then, a potential outcome at $t$ can be written as $Y_{t}(\boldsymbol{d})$. This can be understood as an outcome for an individual, had a particular treatment sequence been assigned. Although the genesis of $Y_{t}(\boldsymbol{d})$ can be very general under this counterfactual framework, the mechanism under which the sequence of treatments interacts with the sequence of outcomes is opaque. The definition of $Y_{t}(\boldsymbol{d})$ becomes more transparent later with the structural model introduced in this paper.
Given these definitions, we state the assumption of sequential randomization by Robins: For each $\boldsymbol{d}\in\mathcal{D}$,
for $t=1,...,T$. This assumption asserts that, holding the history of outcomes and treatments (and potentially other covariates) fixed, the current treatment is fully randomized. Sequential randomization can be violated if agents make decisions $D_{t}$ based on time-varying or time-invariant factors, unobserved to the analyst. In the next section, we relax this assumption and specify dynamic selection equations for a sequence of treatments that are allowed to be endogenous, i.e., to be dependent on unobservable factors. Apart from this assumption, we maintain the same preliminaries introduced in this section.
We now introduce the main framework of this paper. Consider a dynamic structural function for the outcomes, where $Y_{t}$ depends on the entire history of outcomes ($\boldsymbol{Y}^{t-1}$) as well as the current and the entire history of treatments ($D_{t}$, $\boldsymbol{D}^{t-1}$), and that has the form of switching regression models: For $t=1,...,T$,
where $\mu_{t}(\cdot)$ is an unknown scalar-valued function, $X_{t}$ is a set of exogenous variables, which we discuss in detail later, and $Y_{0}$ is assumed to be exogenously determined, with $Y_{0}=0$ for convenience.\footnote{This assumption of an exogenous initial outcome is not necessary but only introduced to simplify our analysis; see Remark (ref) for alternative assumptions.} There can be other potentially endogenous covariates $W_{t}$, which is suppressed in the model. The unobservable variable satisfies $U_{t}(D_{t})=D_{t}U_{t}(1)+(1-D_{t})U_{t}(0)$, where $U_{t}(d_{t})$ is the “rank variable” that captures the unobserved characteristics or rank, specific to treatment state $d_{t}$ (chernozhukov2005iv). We allow $U_{it}(d_{t})$ to contain a permanent component (i.e., individual effects) and a transitory component.\footnote{In this case, it may make sense that the permanent component does not depend on each $d_{t}$, but that the transitory component does.} Given this structural equation, we can express the potential outcome $Y_{t}(\boldsymbol{d})$ using a recursive structure:
where each potential outcome at time $t$ is only a function of $\boldsymbol{d}^{t}$ (not the full $\boldsymbol{d}$). This is related to the “no-anticipation” condition (abbring2007econometric) or the “consistency” condition (robins2000marginal), which is implied from the structure of the model in our setting. The recursive structure provides us with a useful interpretation of the potential outcome $Y_{t}(\boldsymbol{d})$ in a dynamic setting, and thus facilitates our identification analysis. Imposing this structure in order to relax the sequential randomization assumption is the trade-off we exploit. Still, we allow rich channels in the evolution of potential outcomes, as $Y_{t}(\boldsymbol{d})$ is a function of all the past potential outcomes whose treatment indices are consistent with $\boldsymbol{d}$. Also, conditional on $\boldsymbol{X}^{t}\equiv(X_{1},...,X_{t})$, the heterogeneity in $Y_{t}(\boldsymbol{d})$ comes from the full vector $\boldsymbol{U}^{t}(\boldsymbol{d}^{t})\equiv(U_{1}(d_{1}),...,U_{t}(d_{t}))$. By an iterative argument, we can readily show that the potential outcome is equal to the observed outcome when the observed treatments are consistent with the assigned regime: $Y_{t}(\boldsymbol{d})=Y_{t}$ when $\boldsymbol{D}=\boldsymbol{d}$, or equivalently, $Y_{t}=\sum_{\boldsymbol{d}\in\mathcal{D}}1\{\boldsymbol{D}=\boldsymbol{d}\}Y_{t}(\boldsymbol{d})$.
In this paper, we consider the average potential terminal outcome, conditional on $\boldsymbol{X}=\boldsymbol{x}$, as the fundamental parameter of interest:
Again, we suppress that the quantity is conditional on $\boldsymbol{W}=\boldsymbol{w}$. We also call this parameter the average recursive structural function (ARSF) in the terminal period, named after the recursive structure in the model for $Y_{T}(\boldsymbol{d})$. Generally, in defining this parameter and all others below, we can consider the potential outcome in any time period of interest, e.g., $E[Y_{t}(\boldsymbol{d})|\boldsymbol{X}^{t}=\boldsymbol{x}^{t}]$ for any given $t$. We focus on the terminal potential outcome only for concreteness. The knowledge of the ARSF is useful in recovering other related parameters.
First, we are interested in the conditional ATE:
for two different regimes, $\boldsymbol{d}$ and $\tilde{\boldsymbol{d}}$. For example, one may be interested in comparing more versus less consistent treatment sequences, or earlier versus later treatments.
Second, we consider the optimal treatment regime:
with $\left|\mathcal{D}\right|\le2^{T}$, where $W_{0}$ is a vector of pre-treatment covariates in $W_{t}=(W_{0},W_{1t})$. That is, we are interested in a treatment regime that delivers the maximum expected potential outcome, conditional on characteristics $W_{0}=w_{0}$. Notice that, in a static model, the identification of $\boldsymbol{d}^{*}$ is equivalent to the identification of the sign of the static ATE, which is the information typically sought from a policy point of view. One can view $\boldsymbol{d}^{*}$ as a natural extension of this information to a dynamic setting, which is identified by establishing the signs of all possible ATE's defined as in (ref), or equivalently, by ordering all the possible ARSF's. The optimal regime may serve as a guideline in developing future policies. Moreover, it may be a realistic goal for a social planner to identify this kind of scheme that maximizes the average benefit, because it may be too costly to find a customized treatment scheme for every individual. Yet, the optimal regime is customized up to observed pre-treatment characteristics, as it is a function of $w_{0}$. Given $\boldsymbol{d}^{*}(w_{0})$, we may be interested in $E[Y_{T}(\boldsymbol{d}^{*}(w_{0}))]$ or the ATE for the effect of $\boldsymbol{d}^{*}(w_{0})$ relative to another treatment sequence (e.g., the second best). More ambitious than the identification of $\boldsymbol{d}^{*}(w_{0})$ may be recovering an optimal regime based on a cost\textendash benefit analysis, granting than each $d_{t}$ can be costly:
where
with $(w,\tilde{w})$ and $(\boldsymbol{w},\tilde{\boldsymbol{w}})$ being predetermined weights. The latter objective function concerns the weighted sum of the average potential outcomes throughout the entire period, less the cost of treatments. Note that establishing the signs of ATE's will not identify $\boldsymbol{d}^{\dagger}$, and a stronger identification result becomes important, i.e., the point identification of $E[Y_{T}(\boldsymbol{d})|W_{0}=w_{0}]$ for all $\boldsymbol{d}$ (or $E[Y_{t}(\boldsymbol{d})|W_{0}=w_{0}]$ for all $t$ and $\boldsymbol{d}$).
Lastly, we are interested in the transition-specific ATE:
for two different $\boldsymbol{d}$ and $\tilde{\boldsymbol{d}}$. The knowledge of the ARSF does not directly recover this parameter, but the identification of it (and its more general form introduced later) can be paralleled by the analysis for the ARSF and ATE.
In order to facilitate identification of the parameters of interest without assuming sequential randomization, we introduce a sequence of selection equations for the binary endogenous treatments, where $D_{t}$ depends on the entire history of outcomes and treatments ($\boldsymbol{Y}^{t-1}$ and $\boldsymbol{D}^{t-1}$): For $t=1,...,T$,
where $\pi_{t}(\cdot)$ is an unknown scalar-valued function, $Z_{t}$ is the period-specific instruments, $V_{t}$ is the unobservable variable that may contain permanent and transitory components, and $D_{0}$ is assumed to be exogenously given as $D_{0}=0$.\footnote{This is an alternative to simply assuming there is no treatment at $t=0$. We maintain the current assumption to avoid additional definitions for $\pi_{1}(\cdot)$ and other relevant objects.} This dynamic selection process represents the agent's endogenous choices over time, e.g., as a result of learning or other optimal behaviors. However, the nonparametric threshold-crossing structure posits a minimal notion of optimality for the agent. We take an agnostic approach by avoiding strong assumptions of the standard dynamic economic models pioneered by rust1987optimal, such as forward looking behaviors and being able to compute a present value discounted flow of utilities. If we are to maintain the assumption of rational agents, the selection model can be viewed as a reduced-form approximation of a solution to a dynamic programming problem. Lastly, due to the dynamic structure, this selection equation does not necessarily imply the monotonicity assumption of imbens1994identification or vice versa.
To simplify the exposition, we consider binary $Y_{t}$ and impose weak separability in the outcome equation as in the treatment equation. The binary outcome is not necessary for the result of this paper, and the analysis can be easily extended to the case of continuous or censored $Y_{t}$, maintaining weak separability; see Remark (ref). Then, the full model can be summarized as
In this model, the observable variables are $(\boldsymbol{Y},\boldsymbol{D},\boldsymbol{X},\boldsymbol{Z})$. All other covariates $W_{t}$ are suppressed in the equations for simplicity of exposition. Importantly, in this model, the joint distribution of the unobservable variables $(\boldsymbol{U}(\boldsymbol{d}),\boldsymbol{V})$ for given $\boldsymbol{d}$ is not specified, in that $U_{t}(d_{t})$ and $V_{t'}$ for any $t,t'$ are allowed to be arbitrarily correlated to each other (allowing endogeneity) as well as within themselves across time (allowing serial correlation, e.g., via time-invariant individual effects). Note that, because we allow an arbitrary form of persistence in the unobservables and the dependence of $Y_{t}$ and $D_{t}$ on the entire history, $(Y_{t},D_{t})$ is not a Markov process even after conditioning on the observables. This is in contrast to the standard dynamic economic models, where conditional independence assumptions or Markovian unobservables are commonly introduced. By considering the nonparametric index functions that depend on $t$, we also avoid other strong assumptions on parametric functional forms or time homogeneity.
A multi-period experiment with imperfect compliance is one motivating example of this paper's setup. Multi-period experiments are common in clinical trials, such as in the Fast Track Prevention Program (\citet*{conduct1992developmental}), the Elderly Program randomized trial for the Systolic Hypertension (the1988rationale), and the AIDS Clinical Trial Group\footnote{The AIDS Clinical Trials Group (https://actgnetwork.org) is one of the largest HIV clinical trials organizations in the world.}; also see the biostatistics literature referenced in the introduction for other examples. For instance, the Fast Track Prevention Program is a randomized trial to prevent conduct disorders and drug use in children at risk. Interventions are taken place at the end of each semester starting from first grade, by means of home visits and teacher consultations. In household visits, for example, it is reported that assignment deviation occurs for nearly 50% of the intervention children. murphy2001marginal focus on the effect of treatment had there been no deviation, i.e., the intention-to-treat parameters. In this paper, we recover the average treatment effect parameters allowing for this type of imperfect compliance.
Based on to these clinical trials, we consider the following stylized example for the structural model of this paper. A clinical research organization is interested in improving patients' symptoms ($Y_{t}$), and runs an experiment of randomly assigning treatments at each $t$ ($Z_{t}$). Based on the assignment, each patient decides whether or not to receive the treatment ($D_{t}$) by being a complier, defier, always-taker or never-taker. This information can be collected via a fidelity assessment as in the Fast Track Prevention Program. In making the compliance decision, the patient has a habit ($\boldsymbol{D}^{t-1}$) and takes into account her past symptoms ($\boldsymbol{Y}^{t-1}$). The current symptom ($Y_{t}$) is formed based on the past symptoms ($\boldsymbol{Y}^{t-1}$), the current and past treatment take-ups ($\boldsymbol{D}^{t}$), and other symptom-influencing factors ($X_{t}$) occurring at time $t$. As described in detail in the next section, we assume that patients cannot fully predict $X_{t}$ when making treatment decisions $D_{t}$. For patients with potential respiratory diseases, temporal variation in air quality can be such a variable. In the Fast Track Prevention Program, the average performance measure of non-risk peers randomly assigned every academic year can be a candidate.
We first identify the ARSF's, i.e., $E[Y_{t}(\boldsymbol{d})|\boldsymbol{X}^{t}]$ for every $\boldsymbol{d}$ and $t$, which will then be used to identify the ATE's and the optimal regimes $\boldsymbol{d}^{*}$ and $\boldsymbol{d}^{\dagger}$. We maintain the following assumptions on $(\boldsymbol{Z},\boldsymbol{X})$ and $(\boldsymbol{U}(\boldsymbol{d}),\boldsymbol{V})$ for every $\boldsymbol{d}$. These assumptions are written for the identification of $E[Y_{T}(\boldsymbol{d})|\boldsymbol{X}]$, and are sufficient but not necessary for the identification of $E[Y_{t}(\boldsymbol{d})|\boldsymbol{X}^{t}]$ for $t\le T-1$.
Assumption C is a regularity condition to ensure the smoothness of relevant conditional probabilities. Assumption SX imposes strict exogeneity, which is a simple sufficient condition for necessary requirements we need for identification; see Remark (ref). It is implicit that the independence is conditional on the covariates suppressed in the model. Just as the treatments $\boldsymbol{D}$, these covariates may be correlated with the individual effects contained in $(\boldsymbol{U}(\boldsymbol{d}),\boldsymbol{V})$. The variable $Z_{t}$ denotes the standard excluded instruments, which is allowed to be binary. A leading example is a sequence of randomized treatment assignments. Other examples include sequential policy shocks. In addition to $Z_{t}$, we introduce exogenous variables $X_{t}$ in the outcome equation (ref), that are excluded from the selection equation (ref). We make a behavioral/information assumption that there are outcome-determining factors that the agent cannot fully anticipate when making a treatment decision. Continuing with the stylized example in Section (ref), when $D_{t}$ is a compliance choice that a patient makes at the $t$-th visit to the clinical facility, $Y_{t-1}$ may be the symptom measured prior to the decision during the same visit. Then $Y_{t}$ is the symptom measured upon the next visit, which may create enough time gap to prevent the patient from predicting $X_{t}$.\footnote{In a static scenario, Han2018 motivate this reverse exclusion restriction using the notion of externalities. In their setting where multiple treatments are strategically chosen (e.g., firms' entry decisions), factors that determine the outcome (e.g., pollution) are assumed not to appear in the firms' payoff functions. } Note that $(Z_{t},X_{t})$ are assumed to be excluded from the outcome and treatment equations of all other periods as well. Next, we introduce a sequential version of the rank similarity assumption (chernozhukov2005iv):
Rank invariance (i.e., $\{\boldsymbol{U}(\boldsymbol{d})\}_{\boldsymbol{d}}$ being equal to each other) is particularly restrictive in the multi-period context, because it requires that the same rank be realized across $2^{T}$ different treatment states. Significantly weaker than the rank invariance would be a joint rank similarity assumption that $\boldsymbol{U}(\boldsymbol{d})$'s are identically distributed across $2^{T}$ states (conditional on the observables and treatment unobservables). This allows an individual to have different realized ranks across different $\boldsymbol{d}$'s. Assumption RS, which we call sequential rank similarity, relaxes this even further by only requiring that $\boldsymbol{U}(1,\boldsymbol{d}_{-t})$ and $\boldsymbol{U}(0,\boldsymbol{d}_{-t})$ are identically distributed instead. That is, the assumption requires that, within individuals with the same observed characteristics and history of the treatment unobservables, the joint distributions of the ranks are identical between just two states that differ by $d_{t}=1$ and $0$.\footnote{In fact, we can further relax Assumption RS by allowing $U_{t}(d_{t})$ to be a function of $x_{t}$ from the outset; see Remark (ref).}
Now, we are ready to derive a period-specific result. Define the following period-specific quantity directly identified from the data, i.e., from the distribution of $(\boldsymbol{Y},\boldsymbol{D},\boldsymbol{X},\boldsymbol{Z})$:
for $t\ge1$, where $(\boldsymbol{Z}^{0},\boldsymbol{X}^{0},\boldsymbol{D}^{0},Y_{0})$ is understood to mean that there is no conditioning.
Without relying on further assumptions, the sign of $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},1,x_{t})-\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},0,\tilde{x}_{t})$ itself is already useful for calculating bounds on the ARSF's and thus on the ATE's; we discuss the partial identification in Section (ref).
For the analysis of this paper which deals with a dynamic model, it is convenient to define the $\boldsymbol{U}$-set and $\boldsymbol{V}$-set, namely the sets of histories of the unobservable variables that determine the outcomes and treatments, respectively. To focus our attention on the dependence of the potential outcomes on the unobservables, we iteratively define the potential outcome given $(\boldsymbol{d},\boldsymbol{x})$ as
for $t\ge2$, with $Y_{1}(d_{1},x_{1})=1\{\mu_{1}(0,d_{1},x_{1})\ge U_{1}(d_{1})\}$. Now, define the set of $\boldsymbol{U}^{t}(\boldsymbol{d}^{t})$ as
for $t\ge1$. Then, $\boldsymbol{Y}^{t}=\boldsymbol{y}^{t}$ if and only if $\boldsymbol{U}^{t}(\boldsymbol{d}^{t})\in\mathcal{U}^{t}(\boldsymbol{d}^{t},\boldsymbol{y}^{t};\boldsymbol{x}^{t})$, conditional on $(\boldsymbol{D}^{t},\boldsymbol{X}^{t})=(\boldsymbol{d}^{t},\boldsymbol{x}^{t})$. The $\boldsymbol{V}$-set $\mathcal{V}^{t}(\boldsymbol{d}^{t},\boldsymbol{u}^{t-1})\equiv\mathcal{V}^{t}(\boldsymbol{d}^{t},\boldsymbol{u}^{t-1};\boldsymbol{z}^{t},\boldsymbol{x}^{t-1})$ is similarly defined within the proof of Lemma (ref) in the Appendix. Then, $\boldsymbol{D}^{t}=\boldsymbol{d}^{t}$ if and only if $\boldsymbol{V}^{t}\in\mathcal{V}^{t}(\boldsymbol{d}^{t},\boldsymbol{U}^{t-1}(\boldsymbol{d}^{t-1}))$, conditional on $(\boldsymbol{Z}^{t},\boldsymbol{X}^{t-1})=(\boldsymbol{z}^{t},\boldsymbol{x}^{t-1})$. Given these sets, what we show in the proof of this lemma is that, under Assumptions C and SX,
the sign of which identifies the sign of $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},1,x_{t})-\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},0,\tilde{x}_{t})$ by Assumption RS. For example, when this quantity is zero, then $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},1,x_{t})-\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},0,\tilde{x}_{t})=0$.
For the point identification of the ARSF's, the final assumption we introduce concerns the variation of the exogenous variables $(\boldsymbol{Z},\boldsymbol{X})$. Define the following sets:
where (ref) is related to the sufficient variation of $X_{t}$ and (ref) is related to the rectangular variation of $(X_{t},Z_{t})$.
This assumption requires that $X_{t}$ varies sufficiently to achieve $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t},x_{t})=\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},\tilde{d}_{t},\tilde{x}_{t})$, while holding $Z_{t}$ to be $z_{t}$ and $\tilde{z}_{t}$, respectively, conditional on $(\boldsymbol{X}_{-t},\boldsymbol{Z}_{-t})$. This is a dynamic version of the support assumption found in VY07.\footnote{In our setting, it is possible that $\mathcal{X}_{t}(\boldsymbol{d}^{t};\boldsymbol{x}_{-t},\boldsymbol{z}_{-t})$ is nonempty even when $Z_{t}$ is discrete, as long as $X_{t}$ contains continuous elements with sufficient support (VY07). In all these works, including the present one, the support requirement is conditional on the exogenous variables in other periods; see also cameron1998life.} Although Assumption SP requires sufficient rectangular variation in $(X_{t},Z_{t})$, it clearly differs from the large variation assumptions in, e.g., heckman2007dynamic and heckman2016dynamic. These papers employ identification-at-infinity arguments in each period that the support of explained variation (i.e., $\mu_{t}(\cdot)$ in our notation) is no smaller than the support of unobservables. On the other hand, Assumption SP only requires the existence of variation that equates $\mu_{t}(\cdot)$ for two different values of $D_{t}$. Apparently, this is trivially satisfied with the former assumption of large support. Note that even though Assumption SP seems to be written in terms of the unknown object $\mu_{t}(\cdot)$, it is testable because the sets defined above have empirical analogs, according to Lemma (ref). Let $\mathcal{X}_{t}(\boldsymbol{d}^{t};\boldsymbol{x}_{-t})\equiv\{x_{t}:x_{t}\in\mathcal{X}_{t}(\boldsymbol{d}^{t};\boldsymbol{x}_{-t},\boldsymbol{z}_{-t})\text{ for some }\boldsymbol{z}_{-t}\in\text{Supp}(\boldsymbol{Z}_{-t}|\boldsymbol{x}_{-t})\}$ and $\mathcal{X}(\boldsymbol{d})\equiv\{\boldsymbol{x}:x_{t}\in\mathcal{X}_{t}(\boldsymbol{d}^{t};\boldsymbol{x}_{-t})\text{ for some }(x_{t+1},...,x_{T}),\text{ for }t\ge1\}$, which sequentially collect $x_{t}\in\mathcal{X}_{t}(\boldsymbol{d}^{t};\boldsymbol{x}_{-t},\boldsymbol{z}_{-t})$ for all $t$. We are now ready to state the main identification result.
Based on Theorem (ref), we can identify the ATE's. Since the identification of all $E[Y_{t}(\boldsymbol{d})|\boldsymbol{x}^{t}]$'s can be shown analogously to Theorem (ref), we can identify the optimal treatment regimes $\boldsymbol{d}^{*}(\boldsymbol{x})$ and $\boldsymbol{d}^{\dagger}(\boldsymbol{x})$ as well.
We sketch the identification analysis here; the full proof of Theorem (ref) is found in the Appendix. We consider the identification of $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z}]$, since $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x}]=E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z}]$ by Assumption SX.\footnote{When we are to identify the average potential outcome at $t$ instead, the conditioning variables we use are the vectors of exogenous variables up to $t$, i.e., $E[Y_{t}(\boldsymbol{d}^{t})|\boldsymbol{x}^{t},\boldsymbol{z}^{t}]$. Then the entire proof can be easily modified based on this expression.} As the first step of identifying $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z}]$ for given $\boldsymbol{d}=(d_{1},...,d_{T})$, $\boldsymbol{x}=(x_{1},...,x_{T})$ and $\boldsymbol{z}=(z_{1},...,z_{T})$, we apply the result of Lemma (ref). Fix $t\ge2$ and $\boldsymbol{y}^{t-1}\in\{0,1\}^{t-1}$. Suppose $x_{t}'$ is such that $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t},x_{t})=\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}',x_{t}')$ with $d_{t}'\neq d_{t}$ by applying Lemma (ref). The existence of $x_{t}'$ is guaranteed by Assumption SP, as $x_{t}\in\mathcal{X}_{t}(\boldsymbol{d}^{t},\boldsymbol{y}^{t-1};\boldsymbol{x}_{-t},\boldsymbol{z}_{-t})\subset\mathcal{X}_{t}(\boldsymbol{d}^{t};\boldsymbol{x}_{-t},\boldsymbol{z}_{-t})$. The implication of $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t},x_{t})=\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}',x_{t}')$ for relevant $\boldsymbol{U}$-sets is as follows: Analogous to the $\boldsymbol{U}$-set defined earlier, define
Then, by definition, $\boldsymbol{U}\in\mathcal{U}(\boldsymbol{d},y_{T};\boldsymbol{x})$ is equivalent to $\boldsymbol{U}\in\mathcal{U}^{T}(d_{t}',\boldsymbol{d}_{-t},y_{T};x_{t}',\boldsymbol{x}_{-t})$ conditional on $\boldsymbol{Y}^{t-1}(\boldsymbol{d}^{t-1},\boldsymbol{x}^{t-1})=\boldsymbol{y}^{t-1}$ for all $\boldsymbol{x}_{-t}$ and $\boldsymbol{d}_{-t}$.\footnote{The subsequent analysis is substantially simplified when $\mu_{t}(y_{t-1},d_{t},x_{t})=\mu_{t}(y_{t-1},d_{t}',x_{t}')$ is satisfied for all $y_{t-1}$, but this situation is unlikely to occur. Therefore, it is important to condition on $Y_{t-1}(\boldsymbol{d}^{t-1},\boldsymbol{x}^{t-1})=y_{t-1}$ in the analysis.} Based on this result, we equate the unobserved quantity $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z},\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}']$ with a quantity that partly matches the assigned treatment and the observed treatment as follows. First, we can show that
for $t\ge2$, by Assumption SX. Then, by Assumption RS and the discussion above, this quantity is shown to be equal to
by Assumption SX. Note that this last quantity is still unobserved, since $d_{s}$ for $s\ge t+1$ are not realized treatments; e.g., when $T=3$ and $t=2$,
The quantity, however, will be useful in the remaining proof where we use mathematical induction to recover $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z}]$; see the Appendix. Recall the abbreviations $\mathcal{V}^{t}(\boldsymbol{d}^{t-1},d_{t}',\boldsymbol{U}^{t-1}(\boldsymbol{d}^{t-1}))\equiv\mathcal{V}^{t}(\boldsymbol{d}^{t-1},d_{t}',\boldsymbol{U}^{t-1}(\boldsymbol{d}^{t-1});\boldsymbol{z}^{t},\boldsymbol{x}^{t-1})$ and $\mathcal{U}^{t-1}(\boldsymbol{d}^{t-1},\boldsymbol{y}^{t-1})$ $\equiv\mathcal{U}^{t-1}(\boldsymbol{d}^{t-1},\boldsymbol{y}^{t-1};\boldsymbol{x}^{t-1})$. That is, in the derivation of (ref), the key is to consider the average potential outcome for a group of individuals that is defined by the treatments at time $t$ or earlier and the lagged outcome, for which $x_{t}$ is excluded.
The proof of Theorem (ref) is constructive in that it provides a closed-form expression for $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x}]$ in an iterative manner, which can immediately be used for estimation. For concreteness, we provide an expression for $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x}]$ when $T=2$ and binary $Z_{t}$. Define
and
with $\lambda_{1}(x_{1})\equiv\lambda_{1}(x_{1};y_{0})$. By Lemma (ref), $x_{t}'$ satisfies $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t},x_{t})=\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}',x_{t}')$ if and only if $x_{t}'\in\lambda_{t}(x_{t};\boldsymbol{y}^{t-1})$. Then, our identification result suggests that
where
The aggregation with respect to $\boldsymbol{Z}=\boldsymbol{z}$ conditional on $\boldsymbol{X}=\boldsymbol{x}$ is to improve efficiency. In Appendix (ref), we discuss further estimation and inference strategies for the parameter $E[Y_{2}(\boldsymbol{d})|\boldsymbol{x}]$.
In fact, the identification strategy introduced in the previous section can tackle a more general problem. In this section, we extend the identification analysis of the ATE (Theorem (ref) and Corollary (ref)) and show identification of the transition-specific ATE. Given the vector $\boldsymbol{Y}(\boldsymbol{d})\equiv(Y_{1}(\boldsymbol{d}),...,Y_{T}(\boldsymbol{d}))$ of potential outcomes, let $\boldsymbol{Y}_{-}(\boldsymbol{d})\equiv(Y_{t_{1}}(\boldsymbol{d}),...,Y_{t_{L}}(\boldsymbol{d}))\in\mathcal{Y}_{-}\subseteq\{0,1\}^{L}$ be its $1\times L$ subvector, where $t_{1}<t_{2}<\cdots<t_{L}\le T-1$ and $L<T$. Then, the transition-specific ATE can be defined as $E[Y_{T}(\boldsymbol{d})|\boldsymbol{Y}_{-}(\boldsymbol{d})=\boldsymbol{y}_{-},\boldsymbol{X}=\boldsymbol{x}]-E[Y_{T}(\tilde{\boldsymbol{d}})|\boldsymbol{Y}_{-}(\tilde{\boldsymbol{d}})=\boldsymbol{y}_{-},\boldsymbol{X}=\boldsymbol{x}]$ for some sequences $\boldsymbol{d}$ and $\tilde{\boldsymbol{d}}$.
The proof of this theorem extends that of Theorem (ref); see the Appendix.\footnote{As before, the parameters in Theorem (ref) and Corollary (ref) below can be defined for any given period instead of the terminal period $T$. The identification analysis of such parameters is essentially the same, and thus omitted.} The transition-specific ATE defined in Theorem (ref) concerns a transition from a state that is specified by the value of the vector of previous potential outcomes, $\boldsymbol{Y}_{-}(\boldsymbol{d})$. When $Y_{T}(\boldsymbol{d})$ is binary, $E[Y_{T}(\boldsymbol{d})|\boldsymbol{Y}_{-}(\boldsymbol{d})=\boldsymbol{y}_{-},\boldsymbol{X}=\boldsymbol{x}]$ can be viewed as a generalization of the transition probability. As a simple example, with $L=T-1$, one may be interested in a transition to one state when all previous potential outcomes have stayed in the other state until $T-1$. When $L=1$ with $\boldsymbol{Y}_{-}(\boldsymbol{d})=Y_{T-1}(\boldsymbol{d})$, the transition-specific ATE becomes $\Pr[Y_{T}(\boldsymbol{d})=1|Y_{T-1}(\boldsymbol{d})=0]-\Pr[Y_{T}(\tilde{\boldsymbol{d}})=1|Y_{T-1}(\tilde{\boldsymbol{d}})=0]$ introduced in Section (ref). This is a particular example of the treatment effect on the transition probability. The treatment effects on transitions have been studied by, e.g., abbring2003nonparametric, heckman2007dynamic, fredriksson2008dynamic and vikstrom2018bounds.\footnote{The definition of the treatment effect on the transition probability in this paper differs from those defined in the literature on duration models, e.g., that in vikstrom2018bounds. Since vikstrom2018bounds's main focus is on $Y_{t}$ that is irreversible, they define a different treatment parameter that yields a specific interpretation under dynamic selection; see their paper for details. In addition, they assume sequential randomization and that treatments are assigned earlier than the transition of interest.} Let $Y_{t}(d_{t})\equiv\mu_{t}(Y_{t-1},d_{t},X_{t},U_{t}(d_{t}))$ be the period-specific potential outcome at time $t$. Since $Y_{t-1}=Y_{t-1}(\boldsymbol{D}^{t-1})$, the period-specific potential outcome can be expressed as $Y_{t}(d_{t})=Y_{t}(\boldsymbol{D}^{t-1},d_{t})$ using the usual potential outcome. As a corollary of the result above, we also identify a related parameter that specifies the previous state by the observed outcome: $E[Y_{T}(1)-Y_{T}(0)|Y_{T-1}=y_{T-1}]$.
The corollary is derived by observing that $Y_{T}(d_{T})=Y_{T}(\boldsymbol{D}^{T-1},d_{T})$, and thus
where each $E[Y_{T}(\boldsymbol{d}^{T-1},d_{T})|Y_{T-1}(\boldsymbol{d}^{T-1})=y_{T-1},\boldsymbol{d}^{T-1},\boldsymbol{x}]$ is identified from the iteration at $t=T-1$ in the proof of Theorem (ref) by taking $Y_{-}(\boldsymbol{d})=Y_{T-1}(\boldsymbol{d}^{T-1})$.
Suppose Assumption SP does not hold in that $X_{t}$ does not exhibit sufficient rectangular variation, or that there is no $X_{t}$ that is excluded from the selection equation at time $t$. In this case, we partially identify the ARSF's, ATE's and $\boldsymbol{d}^{*}(\boldsymbol{x})$ (or $\boldsymbol{d}^{\dagger}(\boldsymbol{x})$).
We briefly illustrate the calculation of the bounds on the ARSF $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x}]$ when the sufficient rectangular variation is not guaranteed; the case where $X_{t}$ does not exist at all can be dealt in a similar manner, and so is omitted. For each $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z},\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}']$ in the proof of Theorem (ref), we can calculate its upper and lower bounds depending on the sign of $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},1,x_{t})-\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},0,\tilde{x}_{t})$, which is identified in Lemma (ref). Note that, in the context of this section, $\tilde{x}_{t}$ does not necessarily differ from $x_{t}$. For example, for the lower bound on $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x}]=E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z}]$, suppose $\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t},x_{t})-\mu_{t}(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}',x_{t}')\ge0$ for given $(\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1})$, where $x_{t}'$ is allowed to equal $x_{t}$. Then, by the definition of the $\boldsymbol{U}$-set and under Assumption RS, it satisfies that $\mathcal{U}^{T}(\boldsymbol{d},y_{T};\boldsymbol{x})\supseteq\mathcal{U}^{T}(d_{t}',\boldsymbol{d}_{-t},y_{T};x_{t}',\boldsymbol{x}_{-t})$, conditional on $\boldsymbol{Y}^{t-1}(\boldsymbol{d}^{t-1},\boldsymbol{x}^{t-1})=\boldsymbol{y}^{t-1}$. Therefore, we have a lower bound on as $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z},\boldsymbol{y}^{t-1},\boldsymbol{d}^{t-1},d_{t}']$ as
Then, it is possible to calculate the lower bounds on $E[Y_{T}(\boldsymbol{d})|\boldsymbol{x},\boldsymbol{z}]$ using the iterative scheme introduced in the proof of Theorem (ref). That is, at each iteration, we take the previous iteration's lower bound as given, expand each main term in (ref) as before, and apply (ref) for necessary terms.
Lastly, depending on the signs of the ATE's, we can construct bounds on $\boldsymbol{d}^{*}(w_{0})$ (or $\boldsymbol{d}^{\dagger}(w_{0})$), which will be expressed as strict subsets of $\mathcal{D}$. The partial identification of the optimal regimes may not yield sufficiently narrow bounds unless there are a sufficient number of ATE's whose bounds are informative about their signs. In general, however, the informativeness of bounds truly depends on the policy questions. Note that $\mathcal{D}$ is a discrete set. Even though the bounds may not be informative about the optimal regime, they may still be useful from the planner's perspective if they can help her exclude a few suboptimal regimes, i.e., $\boldsymbol{d}^{\circ}$ such that $E[Y_{T}(\boldsymbol{d})|w_{0}]\ge E[Y_{T}(\boldsymbol{d}^{\circ})|w_{0}]$ for some $\boldsymbol{d}$.
An important extension of the model introduced in this paper is to the case where treatments do not appear in every period, while the outcomes are constantly observed. For example, institutionally, there may only be a one-shot treatment at the beginning of time or a few treatments earlier in the horizon, or there may be evenly spaced treatment decisions with a lower frequency than outcomes. A potential outcome that corresponds to this situation can be defined as a function of a certain subsequence $\boldsymbol{d}_{-}$ of $\boldsymbol{d}$. Let $\boldsymbol{d}_{-}\equiv(d_{t_{1}},...,d_{t_{K}})\in\mathcal{D}_{-}\subseteq\{0,1\}^{K}$ be a $1\times K$ subvector of $\boldsymbol{d}$, where $t_{1}<t_{2}<\cdots<t_{K}\le T$ and $K<T$. Then, the potential outcomes $Y_{t}(\boldsymbol{d}_{-})$ and the associated structural functions are defined as follows: Let $\boldsymbol{d}_{-}^{t_{k}}\equiv(d_{t_{1}},...,d_{t_{k}})$. A potential outcome in the period when a treatment exists is expressed using a switching regression model as
for $k\ge1$ with $Y_{t_{1}-1}(\boldsymbol{d}_{-}^{t_{0}})=Y_{t_{1}-1}$, and a potential outcome when there is no treatment is expressed as
for $t$ such that $t_{k}<t<t_{(k+1)}$ ($1\le k\le K-1$). Lastly, $Y_{t}(\boldsymbol{d}_{-})=Y_{t}=\mu_{t}(\boldsymbol{Y}^{t-1},U_{t})$ for $t<t_{1}$ and $Y_{t}(\boldsymbol{d}_{-})=Y_{t}(\boldsymbol{d}_{-}^{t_{K}})=\mu_{t}(\boldsymbol{Y}^{t-1}(\boldsymbol{d}_{-}^{t_{K}}),U_{t})$ for $t>t_{K}$. Each structural model at the time of no treatment is a plain dynamic model with a lagged dependent variable. Let $T=4$ and $\boldsymbol{d}_{-}=(d_{1},d_{3})$ for illustration. Then the sequence of potential outcomes can be expressed as
The selection equations are of the following form: For $k\ge1$,
where the lagged outcome and the latest treatment enter each equation. The observable variables are $(\boldsymbol{Y},\boldsymbol{D}_{-},\boldsymbol{X}_{-},\boldsymbol{Z}_{-})$.\footnote{It may be the case that $X_{t}$ is observed whenever $Y_{t}$ is observed, and thus is included in the $Y_{t}$-equations for $t\neq t_{k}$ as well. We ignore that case here.}
Now all the parameters introduced in Section (ref) can be readily modified by replacing $\boldsymbol{d}$ with $\boldsymbol{d}_{-}$ for some $\boldsymbol{d}_{-}$; we omit the definitions for the sake of brevity. Moreover, the identification analysis of Section (ref) can be easily modified in accordance with the extended setting. Let $\boldsymbol{U}_{-}(\boldsymbol{d}_{-})\equiv(U_{t_{1}}(d_{t_{1}}),...,U_{t_{K}}(d_{t_{K}}))$ and let $\boldsymbol{U}(\boldsymbol{d}_{-})$ be the vector of all the outcome unobservables that consists of $\boldsymbol{U}_{-}(\boldsymbol{d}_{-})$ and $\{U_{t}\}_{t\in\{1,...,T\}\backslash\{t_{1},...,t_{K}\}}$.\begin{asC2}The distribution of $(\boldsymbol{U}_{-}(\boldsymbol{d}_{-}),\boldsymbol{V}_{-})$ has strictly positive density with respect to Lebesgue measure on $\mathbb{R}^{2K}$.\end{asC2}\begin{asSX2}$(\boldsymbol{Z}_{-},\boldsymbol{X}_{-})$ and $(\boldsymbol{U}(\boldsymbol{d}_{-}),\boldsymbol{V}_{-})$ are independent.\end{asSX2}Let $\boldsymbol{d}_{-,-t_{k}}$ be $\boldsymbol{d}_{-}$ without the $t_{k}$-th element.\begin{asRS2}For each $t_{k}$ and $\boldsymbol{d}_{-,-t_{k}}$, $\{\boldsymbol{U}_{-}(d_{t_{k}},\boldsymbol{d}_{-,-t_{k}})\}_{d_{t_{k}}}$ are identically distributed conditional on $(\boldsymbol{U}^{t_{k}-1}(\boldsymbol{d}_{-}^{t_{(k-1)}}),\boldsymbol{V}_{-}^{t_{k}})$.\end{asRS2}Under these modified assumptions, Lemma (ref) is now only relevant for $t=t_{k}$. Restrict the definitions of $\mathcal{X}_{t}(d_{t};\boldsymbol{x}_{-t},\boldsymbol{z}_{-t})$ in (ref) and $\mathcal{X}_{t}(d_{t};\boldsymbol{x}_{-t})$ to hold only for $t=t_{k}$.\begin{asSP2}For each $t_{k}$ and $d_{t_{k}}$, $\Pr[X_{t_{k}}\in\mathcal{X}_{t_{k}}(d_{t_{k}};\boldsymbol{x}_{-,-t_{k}},\boldsymbol{z}_{-,-t_{k}})|\boldsymbol{x}_{-,-t_{k}},\boldsymbol{z}_{-,-t_{k}}]>0$ almost everywhere.\end{asSP2}Let $\mathcal{X}_{-}(\boldsymbol{d}_{-})\equiv\{\boldsymbol{x}_{-}:x_{t_{k}}\in\mathcal{X}_{t_{k}}(d_{t_{k}};\boldsymbol{x}_{-,-t_{k}})\text{ for some }(x_{t_{(k+1)}},...,x_{t_{K}}),\text{ for }k\ge1\}$.
In this paper, we consider identification in a nonparametric model for dynamic treatments and outcomes. We introduce a sequence of selection models, replacing the assumption of sequential randomization, which may be hard to justify under partial compliance or in observational settings. We consider treatment and outcome processes of general forms, and avoid making strong assumptions on distribution and functional forms, nor assumptions on rationality. We show that the treatment parameters and optimal treatment regimes are point identified under the two-way exclusion restriction and sequential rank similarity. We argue that the reverse exclusion restriction is a useful alternative tool for empirical researchers who seek identification in this type of nonseparable models with endogeneity. This source of variation may especially be easy to find and justify in a dynamic setting as in this paper. When the reverse exclusion restriction is violated, we show how to characterize bounds on these parameters.