Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
37,764 characters · 8 sections · 39 citation commands
Conditional Triple Difference-in-Differences
Triple difference-in-differences (TDID) designs are widely used to estimate causal effects, yet their theoretical foundations remain underdeveloped. In a survey of 66 well-cited TDID applications in top journals (see appendix for details), we find that over 70% include control variables, typically in a linear regression with a three-way interaction. While motivated by standard econometrics textbooks angrist2009mostly,cunningham2021causal, this practice lacks formal justification. This paper provides the first theoretical analysis of TDID with controls, highlighting potential biases and proposing a theoretically grounded framework.
We begin by developing a TDID identification framework which incorporates controls for identification.\footnote{Adding controls for precision is discussed in the conclusion.} Our approach synthesizes two strands of the literature. The first, the unconditional TDID framework of olden2022triple, assumes that biases in unconditional parallel trends are invariant across two groups, denoted A and B. The second, the DID framework with controls of callaway2021difference,sant2020doubly, requires that parallel trends hold conditional on covariates. Combining the two, the new identification assumption is that biases in conditional parallel trends are invariant across two groups. We also consider TDID under two scenarios for the treatment assignment mechanism: one in which only eligible units in group A are treated, and another in which eligible units are treated in both groups.
We derive two main identification results. First, we show that the standard approach, estimating a DID with controls separately for each group and differencing the results, fails to recover the target causal estimand when covariate distributions differ across groups. The target estimand, either the average treatment effect on the treated (ATT) for group A or the difference in ATTs between groups A and B, is determined by the treatment assignment mechanism. The intuition is straightforward: each group’s ATT equals its DID plus the average bias in conditional parallel trends. If covariates are distributed differently, the plausible case in most applications, the averaged bias terms do not cancel out, and the (triple) difference in DID no longer identifies the causal estimand.
Second, we show that an alternative approach, taking the difference in conditional DIDs under group A’s covariate distribution, identifies a causal estimand. Depending on the treatment assignment, this target is either the ATT of group A or the expected difference in conditional ATTs between groups A and B, evaluated at group A’s covariate distribution. While the estimand can be viewed as a practical adjustment—identifying what is feasible, it can also be motivated economically, as differences in treatment effects holding the covariate distribution constant may be the estimand of interest in many applications.
For estimation, we develop a double-robust (DR) procedure that extends both parametric and semi-parametric DID methods to the TDID-with-controls setting sant2020doubly, chang2020double. The estimator reweights group B’s DID using the covariate distribution of group A, following the identification result. We illustrate this framework using the canonical minimum wage study by card1993minimum.
To fix ideas, consider the following examples. A first example is muralidharan2017cycling, who study the impact of a bicycle program on girls’ 9th grade enrollment by comparing younger and older cohorts (first difference), boys and girls (second difference), and across two Indian states (third difference). The policy was implemented in only one state and targeted only girls. In our framework, TDID assumes that the bias in parallel trends between genders across cohorts, under the counterfactual without the policy, is equal across states conditional on covariates, such as caste, religion, or access to infrastructure. The standard approach computes a DID within each state and differences the two. This equals the ATT plus a bias term capturing differences in average conditional-trend bias across states. This bias is non-zero when covariate distributions differ across the two states. However, reweighting the control state’s DID to match the covariate distribution in the treated state identifies the ATT.
Second, kleven2019children examine the impact of parenthood on earnings by comparing individuals before and after childbirth (first difference), early and late age at first birth (second difference), and across mothers and fathers (third difference). Both genders become parents after their first child is born, making both groups treated in the post-childbirth period. A TDID design assumes that the bias in parallel trends in earnings across age-at-first-birth groups, under the counterfactual without children, is equal across genders conditional on covariates, such as education. The standard approach computes a DID separately for mothers and fathers and differences the two, yielding the difference in ATTs between mothers and fathers plus a bias term reflecting differences in average conditional-trend bias between genders. However, reweighting the DID for fathers to match the covariate distribution of mothers identifies the difference in conditional ATTs, averaged over the mothers' covariate distribution. This estimand is policy-relevant, as it isolates gender differences in the impact of parenthood while holding covariate distributions fixed.
This paper contributes to the econometric literature on TDID and DID. We build on olden2022triple, who formalize TDID as a single bias-in-parallel-trends assumption, and extend recent work on conditional parallel trends in DID settings zimmert2018efficient, chang2020double, sant2020doubly. The paper makes four contributions. First, we lay the theoretical foundations for identification in TDID designs with covariates. Second, we show that the standard TDID empirical strategy is biased when covariate distributions differ across groups. Third, we develop a framework that identifies a causal target estimand, and we clarify this estimand is determined by the treatment assignment mechanism. Lastly, we develop estimators for the identifiable estimand. An accompanying R package for implementation code is available at \href{https://github.com/dorleventer/tdid}{tdid}.
The remainder of the paper is structured as follows. Section (ref) introduces the notation and basic setup. Section (ref) presents the identification framework and results. Section (ref) discusses the estimation framework. Section (ref) illustrates the results in an empirical application. Section (ref) concludes.
We consider a 2-by-2-by-2 TDID setup, where outcomes are observed across two time periods, for units in two treatment cohorts and two distinct groups. TDID designs differ in their treatment assignment mechanisms, which in turn determine the identified causal estimand (discussed below in Section (ref)). Returning to the examples above, in muralidharan2017cycling only units in one group were treated, while in kleven2019children units from both groups were treated.
We now formalize a general framework that captures both cases by explicitly distinguishing between treatment assignment mechanisms. Let $i \in \{1,\dots,n\}$ index individuals observed in periods $t \in \{1,2\}$. To allow for the possibility that only certain units are eligible for treatment, we separate eligibility from treatment. Let $E_i \in \{2, \infty\}$ denote the period in which unit $i$ becomes eligible for treatment, with $E_i = \infty$ for those never eligible. Let $W_{i,t} \in \{0,1\}$ denote treatment status at time $t$, and let $G_i \in \{a,b\}$ denote group membership (e.g., girls and boys, or mothers and fathers). We assume no units are treated in period 1, i.e., $W_{i,1} = 0$ for all $i$. Treatment assignment in period 2 is governed by the following:
Assumption (ref) generalizes treatment assignment structures in TDID designs. The policy in muralidharan2017cycling corresponds to case (i); kleven2019children corresponds to case (ii).
Under the stable unit treatment value assumption (SUTVA) rubin1980randomization, we denote the potential outcome when unit $i$ remains untreated in both periods as $Y_{i,t}(0)$, and when not treated in the first period and treated in the second period as $Y_{i,t}(1)$. Observed and potential outcomes are linked by the consistency assumption, $Y_{i,t} = Y_{i,t}(W_{i,2})$.\footnote{This potential outcome notation follows the 2-by-2 potential outcome notation in roth2023s for DID.} Finally, let $X_i$ denote a set of time-invariant covariates with support $\mathcal{X}$.
This section starts with articulating several causal estimands and formulated the TDID with controls framework is formulated. Finally, the two identification results are presented. All proofs are given in Appendix (ref). An analytical example, which numerically shows the results, is presented in Appendix (ref).
Let $$ATT(g)=\mathbb{E}[Y_{2}(1)-Y_{2}(0)\mid G=g,E=2]$$ be the ATT of group $g$ at the second time period, and $$CATT(g,x)=\mathbb{E}\left[Y_{2}\left(1\right)-Y_{2}\left(0\right)\mid G=g,E=2,X=x\right]$$ be the ATT conditional (CATT) on covariate value $X=x$ of group $g$ at the second time period. As we show below, the identified causal estimand depends on Assumption (ref). Under Assumption (ref)(i), the target estimand is $ATT(a)$; under Assumption (ref)(ii), one can target either $ATT(a) - ATT(b)$ or $$\mathbb{E}[CATT\left(a,X\right)-CATT\left(b,X\right)\mid G=a,E=2].$$ This last estimand captures the difference in conditional treatment effects between groups $a$ and $b$, averaged over the covariate distribution of group $a$. By holding the covariate distribution fixed, it isolates differences in treatment effects. Returning to the example of kleven2019children, this estimand corresponds to the difference in the conditional effect of parenthood between mothers and fathers, evaluated at the covariate distribution of mothers. In this application, the last estimand may be of particular interest, as it isolates gender-based differences in the effect of parenthood.
The first identification assumption, commonly referred to as no anticipation callaway2021difference, equates the expected value of the untreated and treated potential outcomes in the pre-treatment period.
The second component concerns parallel trends. Standard DID with controls assumes a conditional parallel trends condition: the trend in untreated potential outcomes is equal between treated and never-treated units, conditional on covariates. Define the bias in conditional parallel trends for group $g$ as
The conditional parallel trends assumption of callaway2021difference corresponds to setting $\delta_{\mathrm{CPT}}(g, x) = 0$ for $g \in \{a, b\}$. In contrast, the TDID framework with covariates allows for non-zero $\delta_{\mathrm{CPT}}(g, x)$, but assumes that the bias is equal across groups. This is analogous to olden2022triple, who assume equal bias in unconditional parallel trends.
To compare or reweight conditional expectations across groups, we also require an assumption often termed overlap. Let $p\left(g,e,x\right)=P\left(\left(G,E\right)=\left(g,e\right)\mid X=x\right)$ denote the probability of belonging to group $g$ with eligibility status $e$ given covariates $X = x$. (callaway2021difference term this a "generalizied" propensity score). We assume the following:
Assumption (ref) is the key identifying assumption. Since treatment is not randomly assigned, it should be justified using contextual knowledge and economic modeling. To guide the assessment of its plausibility, we suggest considering two components. First, whether the two groups (e.g., mothers and fathers) select into treatment (e.g., when to have children) based on similar factors. Second, whether, conditional on these factors, the bias in untreated potential outcomes would evolve similarly across groups. Finally, because the assumption is conditional on covariates, the choice of covariates is critical. The assumption will only hold if $X$ captures the relevant dimensions of selection. Justifying it therefore requires careful consideration of the relationship between the covariates and the institutional setting.
Consider the following empirical strategy: run a DID regression with controls separately for each group, then take the difference. This corresponds to the typical TDID empirical strategy angrist2009mostly,cunningham2021causal. The result below shows that, without further assumptions, the descriptive estimand\footnote{We differentiate between descriptive estimands, which are functions of observed data, and causal estimands, which depend on both observed data and potential outcomes, following abadie2020sampling.} corresponding to this strategy does not identify the causal parameter of interest. Let
denote the DID descriptive estimand for group $g\in\{a,b\}$, conditional on covariate value $X=x$. We now turn to the first main result.
Theorem (ref) shows that the typical TDID regression estimand, the difference between groups of the conditional DIDs averaged over the covariate distribution of its respective group (the left-hand side expression) is equal to a causal estimand plus a bias term. The causal estimand depends on the treatment assignment mechanism: it is $ATT(a)$ under Assumption (ref)(i) or $ATT(a)-ATT(b)$ under Assumption (ref)(ii). The bias term is equal to the difference between the two groups in the average bias calculated of conditional parallel trends within each group.
The implication is that, under the TDID design, simply differencing DIDs across groups does not identify a causal estimand unless an additional condition holds. Specifically, identification would require that the average bias in conditional parallel trends is equal across groups. This condition holds if the covariate distributions of groups A and B are identical. However, the rationale for Assumption (ref) is precisely that biases in unconditional parallel trends differ, hence the need to condition on covariates. As a result, if Assumption (ref) is justified, the bias term in Theorem (ref) is unlikely to be zero, and the causal estimand is not identified by the difference in group-specific DIDs within the conditional TDID framework.
The previous result was negative: it showed that the standard empirical strategy in TDID does not identify a causal estimand. The next result is positive, showing that a well-defined causal estimand is identifiable within the conditional TDID framework.
Theorem (ref) shows that a causal estimand is identified by taking the difference between the DID with controls for group A and a reweighted DID for group B, where group B is weighted to match the covariate distribution of group A. This result requires no additional assumptions beyond those already stated.
The identification result in Theorem (ref) motivates estimation of the two expectations $\mathbb{E}[D(a,X)\mid G=a, E=2]$ and $\mathbb{E}[D(b,X)\mid G=a, E=2]$. We begin by establishing equality between these terms and doubly robust (DR) descriptive estimands, building on sant2020doubly. We then discuss an estimation procedure and inference. A simulation study is provided in Appendix (ref).
First, we define weights and outcome regressions.
Next, we define score functions for outcome regression (OR), inverse probability weighted (IPW) and DR.
where $O_i=(Y_{i,2},Y_{i,1},E_i,G_i,X_i)$ is the observed data and $\theta$ are the nuisance parameters, which in this context are the four generalized propensity scores, $p\left(a,2,x\right),p\left(a,\infty,x\right),p\left(b,2,x\right)$ and $p\left(b,\infty,x\right)$, and three outcome regressions, $m\left(a,\infty,x\right),m\left(b,2,x\right)$ and $m\left(b,\infty,x\right)$. From sant2020doubly,callaway2021difference or slight configuration of the lemma below, it follows that under the TDID design (abusing notation by omitting $O_i$ and $\theta$ for compactness)
After establishing identification of the first expectation using a DR approach, we now turn to the second term, $\mathbb{E}[D(b,X)\mid G=a,E=2]$. To differentiate between the score functions, we add a W (short for weighted) to the notation. Let
We formally establish identification in the following lemma (where we again abuse notation by omitting $O_i$ and $\theta$).
The proof is given in Appendix (ref), and loosely follows the structure of the proof in callaway2021difference. To our knowledge, this is a novel result establishing double-robust identification for a reweighted DID estimand in a TDID setting. It enables identification of TDID estimands under group-specific reweighting and is key to estimating the causal effect defined in Theorem (ref). Specifically, combining (ref), Lemma (ref), and Theorem (ref) implies that $\tau_{\mathrm{T}2}=\mathbb{E}\big[\psi_{\mathrm{DR}}(a)\big]-\mathbb{E}\big[\psi_{\mathrm{WDR}}\big]$.
Let $\widehat{\tau}_{\mathrm{T}2}$ be an estimator for $\tau_{\mathrm{T}2}$, for which we discuss a procedure below. Inference hinges on a central limit theorem $\sqrt{n}(\widehat{\tau}_{\mathrm{T}2}-\tau_{\mathrm{T}2})\rightarrow N(0,V)$. This can be achieved by two different approaches on estimated nuisance parameters, denoted $\widehat{\theta}$. First, as discussed in sant2020doubly, one can use parametric models. If these models are correctly specified and regularity conditions hold asymptotic normality of $\widehat{\tau}_{\mathrm{T}2}$ follows. For this approach, using ordinary least squares (OLS) to model the outcome regression ($m(.)$) and logistic regression to model the propensity score ($p(.)$) are straightforward choices. Second, as discussed in chang2020double, one can implement non-parametric models in a de-biased machine learning framework chernozhukov2018double. This requires algorithms with sufficient convergence rates and the use of cross fitting.
For a given choice of models to estimate the nuisance parameters, estimation can be carried out as follows. First, estimate $\widehat{\theta}$. Second, estimate $\widehat{\psi}_{\mathrm{DR}}(a;O_i,\widehat{\theta})$ and $\widehat{\psi}_{\mathrm{WDR}}(O_i,\widehat{\theta})$, which are estimators of $\psi_{\mathrm{DR}}(a;O_i,\theta)$ and $\psi_{\mathrm{WDR}}(O_i,\theta)$, respectively, constructed by substituting parameters in $\theta$ with fitted models in $\widehat{\theta}$ and expectations with sample means. Third, calculate $\widehat{\tau}_{\mathrm{T}2}=\mathbb{E}_n\big[\widehat{\psi}_{\mathrm{DR}}(a;O_i,\widehat{\theta})-\widehat{\psi}_{\mathrm{WDR}}(O_i,\widehat{\theta})\big]$, where $\mathbb{E}_n[.]$ denotes the sample mean. Finally, estimate the asymptotic variance of $\widehat{\tau}_{\mathrm{T}2}$ using $\widehat{V}=\frac{1}{n}\sum_i\eta \left(O_{i},\widehat{\theta},\widehat{\tau}_{\mathrm{T}2}\right)^2$, where $$\eta(O_{i},\theta,\tau) = \psi_{\mathrm{DR}}(a;O_i,\theta)-\psi_{\mathrm{WDR}}(O_i,\theta) - w_{\mathrm{T}}(a,2)\tau_{\mathrm{T}2},$$ and calculate the standard errors of the estimator using $\sqrt{\widehat{V}/n}$.
We illustrate our framework using the canonical minimum wage study by card1993minimum.\footnote{This application is also discussed in cunningham2021causal as a TDID example.} In November 1992, New Jersey raised its minimum wage from \$4.25 to \$5.05, while Pennsylvania’s remained unchanged. The original analysis used a two-by-two DID design comparing employment in restaurants before and after the reform across the two states.
We revisit the study to ask a different question: did the reform have a different impact on low- vs. high-wage restaurants? We illustrate how this question can be addressed using the TDID design. Since both groups are treated if located in New Jersey in the post-reform period, this aligns with Assumption (ref)(ii). Suppose that average potential employment under the lower minimum wage would have followed different trends in New Jersey and Pennsylvania—that is, standard DID fails due to bias in parallel trends. The TDID identification assumption is that this bias is constant across wage groups, conditional on covariates $X$ (Assumption (ref)). Because wage groups may differ systematically, this conditional version is more plausible than an unconditional assumption. The resulting estimand is the difference in conditional ATTs, averaged over the covariate distribution of low-wage restaurants ($\tau_{\mathrm{T}2}$ in Theorem (ref)), which directly adresses the research question.
Let $Y_{i,t}$ denote employment at restaurant $i$ in period $t \in \{\text{pre}, \text{post}\}$, and let $E_i \in \{\text{NJ}, \text{PA}\}$ indicate the state. Restaurants are grouped based on their pre-treatment average starting wage: group A includes those at or below the median (\$4.50), and group B includes those above it. Let $G_i \in \{a, b\}$ denote this group indicator. Treatment assignment follows $W_{i,\text{pre}} = 0$ and $W_{i,\text{post}} = 1_{\{E_i = \text{NJ}\}}$. Covariates $X_i$ include soda price, number of managers, and opening hour, measured in the pre-treatment period. After dropping observations with missing values, the sample size is 695.
We implement the following estimators. First, we estimate DID and TDID using standard linear regressions, both with and without controls, based on two-way interactions for DID and three-way interactions for TDID.\footnote{For completeness, we write the regressions explicitly. The (group-specific) DID regression is $$g\in\{a,b\},\quad Y_{i,t}=\alpha_{0}^g+\alpha_{1}^g1_{\{E_{i}=\text{NJ}\}}+\alpha_{2}^g1_{\{t=\text{post}\}}+\alpha_{3}^g1_{\{E_{i}=\text{NJ},t=\text{post}\}}+\phi^gX_{i}+u_{i}$$ with target coefficients $\alpha^a_3$ and $\alpha^b_3$, and the TDID regression is
with target cofficient $\beta_7$. A regression "without controls" means omitting $X_i$ from the above equations. } Controls are added linearly with no interactions. Second, we apply an OR approach, since it transparently illustrates covariate reweighting. Specifically, we estimate separate OLS outcome models of $Y$ on covariates $X$ for each combination of time period ($t$), state ($E$), and group ($G$), yielding eight models in total. To compute the DID for group A, we use the four models trained on group A to predict outcomes for treated units of group A under all $(t, W)$ combinations, calculate unit-level treatment effects, and average them. This corresponds to an estimator of $\mathbb{E}\big[\psi_{OR}(a)\big]$ (Section (ref)). We repeat the same procedure for group B and estimate $\mathbb{E}\big[\psi_{OR}(b)\big]$. To estimate the weighted DID for group B under the covariate distribution of group A, we predict outcomes for treated units from group A using the models trained on group B, and calculate the DID using these predictions. This corresponds to an estimator of $\mathbb{E}\big[\psi_{WOR}\big]$.
The results, presented in Table (ref), illustrate differences between conventional estimators and those motivated by the identification framework. In the OLS specifications, the triple difference (column 4) is modest, 1.35 without controls and 1.56 with controls. The OR estimator without controls yields similar conclusions. When controls are included, differencing the DID of group A (column 1) from that of group B (column 2) produces a larger estimate of 5.42. However, this estimator does not recover a valid causal estimand (Theorem (ref)). In contrast, differencing the DID of group A from the weighted DID of group B (column 3), which applies group A’s covariates to group B’s outcome model, yields an estimate of 4.23 (column 5). This estimator is unbiased (identification-wise) for the average difference in conditional ATTs under the covariate distribution of group A (Theorem (ref)). However, none of the estimates are statistically significant, likely due to the small sample size, and should be viewed as illustrative rather than conclusive.
This paper provides a theoretical analysis of the triple difference (TDID) design with time-invariant covariates. Our main contributions are as follows. First, we lay the theoretical foundations for identification in TDID designs when conditioning on covariates. Second, we show that the standard TDID empirical strategy can be biased if covariate distributions differ across groups. Third, we show that re-weighed by the covariates of one group identifies a well-defined causal estimand. Lastly, we construct double-robust estimators that recover this estimand under standard assumptions.
While our setup focuses on a simple 2-by-2-by-2 panel structure, several natural extensions are possible. One is to allow for multiple time periods, in the spirit of callaway2021difference. Another is to adapt the framework to repeated cross-section data. Additional extensions could draw on recent developments in the DID literature, including time-varying covariates caetano2022difference, continuous treatments callaway2024difference, and settings with compositional changes sant2023difference. Our results underscore that TDID presents distinct identification challenges relative to standard DID, motivating further research on TDID designs.
We finish with several remarks of practical relevance.
\printbibliography