Efficient Difference-in-Differences Estimation with High-Dimensional Common Trend Confounding

\thispagestyle{empty}

JEL classification: C14, C21\\ \\ Keywords: Semiparametric difference-in-differences, machine-learning, semiparametric efficiency bound, high-dimensional data, employment protection.\\ \\ Acknowledgements: Financial support from the Swiss National Science Foundation (SNSF) is gratefully acknowledged. The study is part of the project "Causal Analysis with Big Data" of the Swiss National Research Program "Big Data" (NRP 75).\\ I want to thank Michael Lechner, Michael Knaus, Bryan Graham, Anthony Strittmatter, Beatrix Eugster and Petyo Bonev for competent advice and Franziska Zimmert for motivating the research question. I also thank Gabriel Okasa for some very useful pieces of R code and Daniel Goller, Carina Steckenleiter and Jana Mareckova for helpful remarks and comments. Further, I want to thank all participants of the European Causal Inference Meeting 2019 in Bremen for constructive feedback and fruitful discussions.

\setcounter{footnote}{0} \setcounter{page}{1} \doublespacing \allowdisplaybreaks

Introduction

In difference-in-differences identification is ensured by the fact that the subpopulation that will be exposed to the treatment (treatment group) and the subpopulation that will not be exposed to the treatment (control group) would have developed equally in the absence of treatment.\footnote{For textbook treatments see for example Athey_Imbens_2017, Lechner_2010 or Imbens_Wooldridge_2009.} Identification often relies on the assumption that the common trend holds conditional on covariates. Any underlying factor differently shifting the potential outcomes under non-treatment for the treatment group and the control group needs to be controlled for. However, even if the researcher can credibly identify the factors that may lead to common trend confounding, it is still unclear in what form covariates should ultimately enter the statistical model for several reasons.\\ Crucially, the statistical model depends on assumptions about the relation between the treatment group identifier, time and observed covariates. With cross-sectional data covariates might be needed to account for imbalances between treatment and control group and across time. With some notable exceptions (Lechner_2010, Hong_2013, Stuart_Huskamp_Duckworth_Simmons_Song_Chernew_Barry_2014, Lu_Nie_Wager_2019) most studies in semiparametric difference-in-differences exclude time-varying treatment group compositions and covariates (e.g., Heckman_Ichimura_Todd_1997, Abadie_2005, SantAnna_Zhao_2020, Chang_2020). This paper investigates semiparametric difference-in-differences models under various assumptions on how covariates, time and treatment group composition are related. Efficient influence functions are derived under more or less restrictive assumptions and for different sampling schemes. We present various identification and efficiency results for low-dimensional semiparametric difference-in-differences models. Our results are sensitive to assumptions about how covariates enter the model. In particular, our results hint at a trade-off between the strength of the assumptions the researcher is willing to impose on the model and the efficiency bound that can be achieved under such assumptions. Further, our results suggest that there are cases where we might want to include covariates even if not needed for identification, as they could increase the precision of some of the estimators. A comparison of the efficiency bounds for cross-section and panel data allows to draw interesting conclusions about the efficiency loss when panel data is not available. We therefore contribute to the literature on semiparametric efficiency in causal inference settings (e.g., Hahn_1998, Firpo_2007, Froelich_2007, Chen_Hong_Tarozzi_2008, Cattaneo_2010, Graham_Pinto_Egel_2016, Lee_2018). Such an analysis is typically based on the approach developed by Newey_1990,Newey_1994 and Bickel_Klaassen_Ritov_Wellner_1993. Chamberlain_1987,Chamberlain_1992 contributes an alternative approach based on moment conditions. Graham_2011 establishes an equivalence result between the moment condition based approach and the approach of Bickel_Klaassen_Ritov_Wellner_1993 for the general missing data problem. In parallel work SantAnna_Zhao_2020 also consider efficiency theory for semiparametric difference-in-differences problems to derive efficient score functions. Their results crucially rely on a relatively strong stationarity assumption and are included in this paper as a special case. We also notice that a previous version of the present paper was the first that proposed efficiency bounds for the semiparametric difference-in-differences problem using Graham_2011's (Graham_2011) equivalence result. It turns out that for the panel case the moment conditions exhaust all the information in the identifying assumptions while for the cross-sectional case they do not.\footnote{See Zimmert_2018 on this. We do not follow Graham_2011's (Graham_2011) approach in this version. However, we consider the insufficiency of the first and second stage moment conditions to exhaust all information necessary to derive the efficiency bound for the cross-sectional difference-in-differences case as an interesting topic for further research.}\\ The efficient influence functions derived imply plug-in estimators that allow to combine semiparametric difference-in-differences models with very flexible first stage estimators. This is important because there might be many different covariates that are supposed to measure the same economic channel for common trend confounding and it might be unclear in what functional form the covariates should be included in the model. These issues might be especially prevalent in difference-in-differences models. Often covariates like geographic or industry classifications are available for different levels of aggregation -- making covariate selection an even more tedious task. For standard parametric models usually used for difference-in-differences estimation (e.g., Card_1990, Card_Krueger_1994, Eissa_Liebman_1996) or semiparametric models with parametric or nonparametric first stages (e.g., Abadie_2005, SantAnna_Zhao_2020) a high-dimensional covariate space will cause the estimator to break down. Advances in the supervised machine-learning literature\footnote{For an overview see e.g. Hastie_Tibshirani_Friedman_2009.} showed an immense potential to approach this problem by choosing a data-driven trade-off between the covariate dimension and the sample size at hand and were successfully integrated in causal inference settings (e.g., Belloni_Chen_Chernozhukov_Hansen_2012, Zhang_Zhang_2014, vandeGeer_Buehlmann_Ritov_Dezeure_2014, Belloni_Chernozhukov_Hansen_2014, Athey_Imbens_Wager_2018). This paper builds on the generic 'double machine-learning' framework developped in Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017. A major insight from this work is that 'single'-robust estimators based on the treatment mechanism (e.g., Horvitz_Thompson_1952, Hirano_Imbens_Ridder_2003, Hahn_Ridder_2013) or outcome based models (e.g., Hahn_1998) are inappropriate with machine-learning generated first stages while 'double'-robust estimators (Robins_Rotnitzky_Zhao_1994, Scharfstein_Rotnitzky_Robins_1999) maintain good statistical properties. Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017 show that the (rate) double robustness properties can be used such that there is no effect of first stage nuisance parameter estimation under relatively weak convergence conditions of the first stage parameters. We modify and extent the double machine-learning framework in this paper. In contrast to Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017, this paper does not rely on high-level conditions based on Gateaux differentiation to verify the rate double robustness properties. Alternatively, we provide easy-to-check conditions that cover a broad range of scores typically used in the causal inference literature. Focusing on a specific class of widely used score functions allows to derive generalizable convergence condition requirements. Crucially, this substantially reduces the computational burden when deriving the asymptotic properties of an estimator and requires fewer regularity conditions such as the existence of the derivative or the interchangeability of the derivative and the expectation operator. Further, some of the derived efficient influence functions imply a new class of plug-in estimators whose convergence conditions depend on the existence of higher-order moments of the outcome. These results do not trivially follow from existing theory (e.g., Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017). Our theoretical results on double machine-learning is therefore also useful beyond the scope of difference-in-differences estimation. Additionally, they allow us to derive first stage convergence conditions for different semiparametric difference-in-differences estimators. This enables to incorporate sophisticated supervised machine-learning algorithms that can cope with settings where the dimension of the covariate space is high. Plug-in estimators that follow from the derived efficient influence functions are shown to achieve the low-dimensional variance lower bound. Our results also indicate that for some cases there is a trade-off between estimation robustness and efficiency. Throughout the evolvement of this paper other related but independent work on semiparametric difference-in-differences estimation with machine-learning appeared. Chang_2020 also considers difference-in-differences estimation under the strong stationarity assumption by directly applying the double machine-learning results of Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017. His estimator does generally not attain the semiparametric efficiency bound. Lu_Nie_Wager_2019 propose estimators that are robust against the distortion of the stationarity assumption but focus on another parameter, consider alternative estimation methods and do not derive efficiency results.\\ Finally, we adopt the methods to a well-known application and investigate whether the efficiency-robustness trade-offs matter in practice and the value added when using machine-learning methods instead of standard parametric estimation methods.\\ The following section introduces the setting and required notation. The development of semiparametric theory for the difference-in-differences problem will be the starting point of our analysis in Section (ref). Section (ref) presents our results on estimation and inference. To assess the usability of the proposed methods, they are applied to real world data in Section (ref). The last section concludes. Most technical proofs are relegated to the Appendix.

Setting and notation

Random variables like $A$ are denoted by capital letters. They have realizations $A=a$ that are in the support of the random variable $\mathcal{A}$. $A=a$ has density $f_A(a)$. If $A$ is discrete we write $Pr(A=a)=f_A(a)$ as a shorthand. The cumulative distribution function is given by $F_A(a)$. Let $B$ be another random variable. Then independence between two random variables $A$ and $B$ is denoted by $A\perp B$. The expectation operator is defined by $\mathbb{E}$ and $\text{Var}$ is used as a shorthand for the variance. For a generic function $g=g(B)=g_{A}(B)$ we use $A$ in the subscript to remind us of the mapping $g: b\mapsto a$. The $L_p$ norm is denoted by $\left\lVert g(B)\right\rVert_p$. As a special case we use $\sup_{b\in\mathcal{B}}\lvert g(b)\rvert$ and $\lVert g(B)\rVert_{\infty}$ interchangeably to denote the supremum of the function. The infinum is $\inf_{b\in\mathcal{B}}\lvert g(b)\rvert$. Let $\beta$ be some parameter then we denote $\dot{g}(B,\beta)=\frac{\partial g(B,\beta)}{\partial\beta}$. Generically, $C>0$ denotes a constant.\\ Let $D$, $T$ and $G_{\tau}$ be binary indicator variables such that $d,t,g_{\tau}\in\{0,1\}$ where $\tau\in\mathcal{T}$ and either $\mathcal{T}=\{(d,t)\}$ or $\mathcal{T}=\{d\}$. In particular, $D=1$ for observations that belong to the treatment group, $T=1$ for observations that are observed in period 1 and $G_{d,t}=1$ if $D=d$ and $T=t$ and 0 otherwise and $G_d=1$ if $D=d$ and 0 otherwise.\footnote{Obviously for the latter case $G_d=D$. We introduce this notation to formulate results as general as possible throughout our exposition.} Denote the outcome variable by $Y$ and some further observed variables by $X$. We follow the established literature (e.g., Roy_1951, Rubin_1974) and let $Y^d(t)$ be the potential outcome variable that contains the potentially unobserved realizations of $Y$ for the state $D=d$ and $T=t$. The exposition additionally relies on the definition of some conditional expectations. In particular, we have $m_Y(d,t,x)=\mathbb{E}\left[Y|D=d,T=t,X=x\right]$ with $m_Y(x)=\sum_{d=0}^1\sum_{t=0}^1(-1)^{d+t}m_Y(d,t,x)$. Similarly, for $\Delta Y=Y(1)-Y(0)$ we have $m_{\Delta Y}(d,x)=\mathbb{E}\left[Y(1)-Y(0)|D=d,X=x\right]$ with $m_{\Delta Y}(x)=m_{\Delta Y}(1,x)-m_{\Delta Y}(0,x)$. Additionally, suppose that $A$ is a binary variable. Then we generically define the probabilities $p_{A=a}(b)=Pr(A=a|B=b)$, $p_{A}(b)=Pr(A=1|B=b)$, $p_{A=a}=Pr(A=a)$ and $p_{A}=Pr(A=1)$. For example we have $p_{D=d,T=t}(x)=Pr(D=d,T=t|X=x)$ and $p_{DT}=Pr(D=1,T=1)$. Note that the definition of $G_{\tau}$ allows us to flexibly write for example $m_Y(d,t,x)=m_Y(G_{d,t}=1,x)$ and similarly for the other parameters.\\ In difference-in-differences settings the researcher is generally interested in identifying the parameter $\theta=\mathbb{E}\left[Y^1(1)-Y^0(1)|D=1,T=1\right]$. It can be described as an average treatment effect on the treated (ATET) because the parameter is defined for those who actually receive the treatment ($D=1$, $T=1$). An average population effect cannot be identified because this would require a subpopulation for which the treatment vanishes between period $T=0$ and $T=1$ (for a discussion on this see Lechner_2010). We also note that under a strong stationarity conditions or when panel data is available Abadie_2005 shows that $\mathbb{E}\left[Y^1(1)-Y^0(1)|D=1\right]$ is identified. Notice that, without further assumptions, this parameter is not an ATET but an average treatment effect for the treatment group. Intuitively, panel data or a stationarity assumption ensures that the composition of the treatment group does not depend on $T$ and so $\mathbb{E}\left[Y^1(1)-Y^0(1)|D=1,T=0\right]=\mathbb{E}\left[Y^1(1)-Y^0(1)|D=1,T=1\right]$. As the ATET is identified under all assumptions made in this paper, we focus on this parameter but indicate whenever the treatment group effect equals the ATET.

Identification and efficiency bounds

Repeated cross-sections

Assumption (ref) describes the data-generating process (DGP) for the repeated cross-sections. It guarantees that we can use the pseudo-sample $W_i$ with observations $i=1,...,N$ and emphasizes that we have to cope with a merged sample problem where the sample sizes of $W(0)$ and $W(1)$, $N(0)$ and $N(1)$ obey $\frac{N(0)}{N(1)}\rightarrow C$ (Abadie_Imbens_2006, Graham_Pinto_Egel_2016). Also notice that the existence of the joint distribution implies that the dimension of $X$ is fixed. We will relax this condition in Section (ref).\\ In what follows we discriminate five settings (CS-1) to (CS-5) that describe different assumption sets on the relation between $D$, $T$ and $X$. In (CS-1) we do not make any further assumptions. It is the fully robust setting. Settings (CS-2)-(CS-5) are comprised in Assumption (ref).

(CS-2) allows for time varying $D$ and $X$ but requires that all time variation in $D$ is fully captured by $X$. (CS-3) does not allow for time-varying $X$ but $D$ may still follow a time trend. (CS-4) implies the strong stationarity assumption used by Abadie_2005, SantAnna_Zhao_2020 and Chang_2020. It excludes time variation in $D$ and $X$. (CS-5) is the `experimental' setting. Even though the outcome might depend on $X$, there are no imbalances neither between treatment and control group nor across time.\\ Since Assumption (ref) only contains conditions on the relation of observed random variables, it is in principal testable. Especially, a significant correlation between $D$ and $T$ rules out settings (CS-4) and (CS-5). Also notice that (CS-2) and (CS-3) are mutually exclusive and that the restrictiveness of the assumptions can be ordered as (CS-5), (CS-4), (CS-3)/(CS-2), (CS-1).

Identification

Assumption (ref) yields an identification result for $\theta$ with cross-sectional data. The Observational Rule underscores that we only observe $Y$ and not $Y(0)$ and $Y(1)$ for every observation. It rules out that observations in the treatment group can be part of the control group or that observations in period $T=0$ are again observed in $T=1$. Notice that since we consider a pseudo-sample, this does not rule out individuals from an actual population from being re-sampled in the second cross-section. Common Support is necessary to guarantee the existence of conditional expectations. Since we are only interested in the ATET, the propensity score only needs to be bounded away from zero for identification. The No Anticipation condition rules out an effect of the treatment in period $T=0$ for the treatment group. The Conditional Common Trends condition requires that conditional on the covariates the treatment and the control group would have developed equally in the absence of the treatment. To allow for a more compact representation of the results, some further notation is introduced. Let $q_{CS;D=d,T=t}(X)$ denote the conditional probability function $p_{D=d,T=t}(X)$ under some of the specific assumptions on the relation of $D$, $T$ and $X$ in (CS-1) to (CS-5). For example we have $q_{CS-2;D=1,T=1}(X)=p_D(X)p_T(X)$. Equivalently, denote by $q_{CS;DT}$ the unconditional probability $p_{DT}$ under some specific assumption (CS-1) to (CS-5).

Proof: Notice that

using Assumptions (ref). Further, for the conditional density function $f_{X|D=d,T=t}(x|d,t)$

For cases (CS-4) and (CS-5) the treatment group effect is identified.

Semiparametric efficiency bounds

Proof: see Appendix (ref).\\ Theorem (ref) is our first main result. Under each of the settings (CS-1)-(CS-5) an influence function with a different adjustment term $\psi^{*b}_{CS}(W)$ is derived implying different efficiency bounds for the different settings. An important implication is summarized in Corollary (ref).

Proof: see Appendix (ref).\\ Unambiguously, $\Delta_{CS-1,CS-2},\Delta_{CS-1,CS-3},\Delta_{CS-1,CS-4},\Delta_{CS-1,CS-5}>0$. Hence, knowing that one of the assumptions (CS-2)-(CS-5) is true results in lower efficiency bounds compared to not making any assumptions (CS-1) about the relation between $D$, $T$ and $X$. Since (CS-2) and (CS-3) contain (CS-4) and (CS-4) contains (CS-5) similar results can be shown for these cases. To conclude, more restrictive assumptions about the relation between $D$, $T$ and $X$ imply lower efficiency bounds. This hints at a trade-off regarding the robustness with respect to the model assumptions imposed and the variance lower bound that is asymptotically achievable.

Panel data

Assumption (ref) describes the DGP when panel data is available. In contrast to Assumption (ref), the sample is not merged and contains $i=1,...,N$ unique individuals. Also the outcomes $Y(0)$ and $Y(1)$ are directly observable in the sample for one observation and thus do not need to be inferred.\\ For panel data we discriminate two settings (PA-1) and (PA-2) that describe different assumption sets on the relation between $D$ and $X$. Since we do not need a time indicator to describe the sample, the analysis is limited to the relation between $D$ and $X$. In (PA-1) we do not make any further assumptions. It is the fully robust setting.

(PA-2) is more restrictive in the sense that imbalances between the treatment and the control group are ruled out.

Identification

Similarly to Assumption (ref), Assumption (ref) ensures the identification of $\theta$ when panel data is available. The Observational Rule guarantees that for both outcomes $Y(0)$ and $Y(1)$ an observation cannot be part of the treatment and the control group at the same time. The other assumptions are adapted in a straightforward manner from the cross-sectional setting.\\ Again we let $q_{PA;D=d}(X)$ denote the conditional probability function $p_{D=d}(X)$ under either (PA-1) or (PA-2).

Proof: The proof follows similar to Lemma (ref).\\ In the panel the treatment group effect is identified.

Efficiency bounds

Proof: see Appendix (ref).\\ Theorem (ref) is our second main result. Under (PA-1) and (PA-2) an influence function with a different adjustment term $\psi^{*b}_{PA}(W)$ is derived implying different efficiency bounds for (PA-1) and (PA-2). An important implication is summarized in Corollary (ref).

Proof: The proof follows similarly to Corollary (ref).\\ Again, since $\Delta_{PA-1,PA-2}>0$, the more restrictive assumption (PA-2) is associated with a lower efficiency bound. Hence, the robustness-efficiency trade-off also materializes for panel data.\\ Having derived results for both cross-sectional and panel data, it might be of interest to compare the variance lower bounds under both sampling schemes. Corollary (ref) summarizes the implications of Theorems (ref) and (ref) for the relative efficiency between panel and cross-sectional data.

Proof: see Appendix (ref).\\ Since $\Delta_{CS-1,PA-1}>0$, under no further condition on the relation between $D$, $T$ and $X$ the first result of the corollary shows that knowing the panel structure generally reduces the variance lower bound that is asymptotically achievable under comparable assumptions. This is an intuitive result because the information in a panel is unambiguously richer. $\Delta_{CS-1,PA-1}$ can therefore also be seen as the gain from the potentially more costly panel sampling scheme. Since the difference $\Delta Y$ is observed with panel data, the variance lower bound only contains propensity score reweighted conditional variances. For cross-sectional data the conditional variances are additionally reweighted by $p_{T=t}$ resulting in an efficiency loss. Further, notice that the gain from observing the panel $\Delta_{CS-1,PA-1}$ is higher (lower) when the correlation between $Y(0)$ and $Y(1)$ is positive (negative). This can be explained by the fact that the variance of $\Delta Y$ is lowest when $Y(0)$ and $Y(1)$ are positively correlated. Hence, the panel becomes more valuable when the observed difference $\Delta Y$ is less volatile. The second result of the corollary assesses the value of making assumptions relative to having access to panel versus cross-sectional data. It is hypothesized that the researcher knows that (CS-5) is correct and has panel data available where he uses the stronger than necessary setting (PA-1). From Theorem (ref) and Corollary (ref) we know that among all settings (CS-1)-(CS-5) the minimum variance lower bound with cross-sectional data is achieved when making assumption (CS-5). From Theorem (ref) and Corollary (ref) we know that among (PA-1) and (PA-2) the maximum variance lower bound with panel data is achieved when making assumption (PA-1). The difference among these settings when panel data becomes available $\Delta_{CS-5,PA-1}$ thus measures the value of making assumptions relative to having access to better data. Notice that $\Delta_{CS-5,PA-1}\lesseqgtr 0$. Hence, having access to panel data does not necessarily lead to lower efficiency bounds. Rather, the corollary shows the importance of making adequate assumptions on the relation between $D$, ($T$) and $X$.

Estimation and Inference

High-dimensional data and important building blocks

Denote by $\eta$ a set of nuisance parameters that are generally unknown and have to be estimated in a first stage. From Section (ref), $\eta$ consists of functions of $X$. By Assumptions (ref) and (ref) the results of Section (ref) only hold when the dimension of the covariates $\lambda_X$ is fixed. For estimation and inference we relax this condition and assume that $X\in\mathbb{R}^{\lambda_X}$ and potentially $\lambda_X\rightarrow\infty$ when $N\rightarrow\infty$. Conditions in the next subsection will describe the concrete growth rates of $\lambda_X$ in relation to $N$.\\ Let $\psi$ be a function of the observed variables $W$ and the nuisance parameters. $W$ contains the generic outcome variable $\tilde{Y}$ and $\eta$ contains some projection on $\tilde{Y}$ denoted by $m_{\tilde{Y}}(\cdot)$. In contrast to Section (ref), we now explicitly indicate that the function $\psi$ depends on $\eta$ and consider $\psi$ to be of the general form

In particular, the function $\psi(W,\eta)$ is a sum over the index $\tau$ of terms in the form

where $q_{G_{\tau}}(x)=Pr(G_{\tau}=1|X=x)$ under some specific assumption on the relation between $G_{\tau}$ and $X$. $q_1(X)$ and $q_1$ are shorthand symbols for $q_{G_{\tau}}(X)$ and $q_{G_{\tau}}$ with $\tau=(1,1)$ in the cross-sectional and $\tau=1$ in the panel case. We assume that $\mathbb{E}\left[\psi(W,\eta;\theta)\right]=0$ such that $\theta$ is identified as

A plug-in estimator uses the ratio of sample averages with estimated nuisance parameters. Following the suggestions of Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017 we apply a cross-fitting algorithm for the nuisance parameter estimation step in order to guarantee that the resulting estimators of $\psi(W,\eta)$ and $\psi^b(W,\eta)$ consist of independent observations. The details of the estimation strategy are outlined in the algorithm below.

Our conditions on the first stage nuisance parameter convergence rates should cover a wide range of estimators. We therefore rely on $L_2$ convergence rates. To reduce the notational burden, generically write $L_2$-rates for cross-fitted nuisance parameters as $\epsilon_{\eta}=\left\lVert\hat{\eta}_{-k}-\eta\right\rVert_2$.

Asymptotic results

In order to derive asymptotic results for the cross-fitting estimator described in the previous subsection, we have to make several assumptions.

Notice that Assumption (ref) at least requires the existence of the second moment of the outcome. We will later show that for some difference-in-differences estimators the existence of higher-order moments interacts with the requirements of first stage convergence conditions.

Assumption (ref) is stronger than the usual Common Support conditions made in Assumptions (ref) and (ref) to obtain the identification results in Section (ref). All conditional probabilities and their estimators are strictly bounded away from zero and one. The assumption precludes using for example the Linear Probability Model or the linear Lasso for estimating the propensity scores.

Assumption (ref) contains some further regularity conditions on the behaviour of the term $\psi^b(W,\eta)$ that are easily satisfied for the difference-in-differences estimators to be considered. Condition (i) is in principle redundant because if it would not hold then $\mathbb{E}\left[\psi(W,\eta;\theta)\right]=0$ would not be satisfied. We keep it, however, to remind us that the term cannot be of arbitrary form. Since the cross-fitted estimator $\hat{\theta}$ involves dividing by the sample plug-in average of $\psi^b(W,\eta)$, conditions (ii) are needed to guarantee that $\hat{\theta}$ is nicely behaved. The last condition is satisfied whenever the infinum of $\psi^b(W,\eta)$ is zero.

Assumption (ref) comprises the required coupled convergence conditions that are at the centre of our theoretical argument. The assumption ensures that the first stage nuisance parameter estimation has no effect on the asymptotic behaviour of the cross-fitted estimator. Conditions (i)-(iii) contain the convergence requirements requirements for term $\psi^b(W,\eta)$. Notice that they are trivially satisfied whenever $\psi^b=\psi^b(W)$. Hence, when the term does not depend on nuisance parameters but just on observed variables, conditions (i)-(iii) always hold. However, some of the efficient influence functions considered for the cross-sectional case in Section (ref) suggest that one may also use functions where $\psi^b=\psi^b(W,\eta)$. This represents the first extension to the work of Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017 who just consider problems of the latter type when deriving first stage convergence rate conditions. Condition (iv) requires that for all $\tau\in\mathcal{T}$ the nuisance parameters converge in $L_2$. Condition (v) requires that for all $\tau\in\mathcal{T}$ the conditional probability and the outcome nuisances jointly achieve $\sqrt{N}$-convergence. Conditions (iv) and (v) are easy-to-check conditions because they can be applied to all sorts of reweighting schemes. In particular, the specific convergence conditions for all $\tau\in\mathcal{T}$ can be directly retrieved from the conditions provided. This represents the second extension to the work of Chernozhukov_Chetverikov_Demirer_Duflo_Hansen_Newey_2017 who rely on Gateaux differentiation and implied regularity conditions to derive required convergence rate for specific settings. Our theory covers all functions $\psi(W,\eta;\theta)$ where $\psi(W,\eta)$ can be written as a sum over some index $\tau$. This includes the usual parameters of interest and settings in causal econometrics such as selection-on-observables with binary or multiple treatments, difference-in-differences and instrumental variables.\\ The $L_2$ rate conditions in Assumption (ref) (iv) and (v) can be shown to be satisfied for many supervised machine-learning algorithms under sparsity conditions. For example Belloni_Chernozhukov_2013 show that the predictive error of the Lasso is of order $O_p\left(\sqrt{\frac{s\log\max(\lambda_X,N)}{N}}\right)$ where $s$ is the unknown number of true coefficients in the oracle model. Assumption $\ref{ass:estinfjoint}$ (iv) and (v) then require that $\frac{s^2\log^2\max(\lambda_X,N)}{N}\rightarrow 0$ (if $s$ is the same in the propensity score and the outcome nuisance models). It follows that $\lambda_X\rightarrow\infty$ when $N\rightarrow\infty$ under the sparsity condition. Thus, the dimension of the covariates can be high in the sense that it can (slowly) grow with the sample size whenever the true model is sparse. Similar $L_2$ rate conditions can also be shown for non-linear models like Random Forests (Wager_Walther_2015), Honest Random Forests (Wager_Athey_2018) or forms of Deep Neural Nets (Farrell_Liang_Misra_2018).

Proof: see Appendix (ref).\\ Theorem (ref) is our third main result. The cross-fitted estimator is asymptotically normal with influence function $\psi(W,\eta;\theta)$. Notice that the result implies that whenever we use the sample plug-in estimator of the efficient influence function the cross-fitted estimator asymptotically attains the low-dimensional variance lower bound if Assumptions (ref)-(ref) are satisfied. The following subsections contain several corollaries of Theorem (ref) that summarize the convergence conditions and asymptotic behaviour of different cross-fitted semiparametric difference-in-differences estimators.

Plug-in estimators

Corollary (ref) summarizes the implications of Theorem (ref) for cross-sectional difference-in-differences estimators in settings (CS-1)-(CS-5) that use the efficient influence functions derived in Theorem (ref).

Proof: see Appendix (ref).\\ The joint convergence conditions are more sophisticated for settings that are less restrictive with respect to the model assumptions on the relation between $D$, $T$ and $X$ imposed. For (CS-1) we need six, for (CS-2) and (CS-3) we need five, for (CS-4) we need two and for (CS-5) we need zero joint convergence rates to be satisfied. This hints at a trade-off between the robustness of the estimators with respect to the assumption on $D$, $T$ and $X$ and the robustness of the estimators with respect to first stage nuisance parameter convergence requirements. Further, we notice that for settings (CS-2) and (CS-3) higher-order moments $r>2$ have to exist. For setting (CS-3) this is a mere regularity condition. For setting (CS-2) it has some implications on the joint convergence condition of the two propensity scores. The more moments of the outcome exist, the closer the joint convergence condition at the usual $\sqrt{N}$ condition. This implies that for a bounded outcome the condition becomes $\epsilon_{p_D(X)}\times\epsilon_{p_T(X)}=o_p\left(N^{-\frac{1}{2}}\right)$.\footnote{For most outcomes in labour market applications the outcome is bounded. The condition might, however, become relevant for distributions with `fat tails' typically present in financial econometrics.} If the researcher is only willing to assume that a second moment of $Y$ exists, this implies that for both propensity scores parametric convergence rates are needed. We additionally notice that for the experimental setting (CS-5) the efficient estimator is not the simple difference in means estimator but a residualized version of it.\\ Corollary (ref) summarizes the implications of Theorem (ref) for panel difference-in-differences estimators in settings (PA-1) and (PA-2) that use the efficient influence functions derived in Theorem (ref).

Proof: see Appendix (ref).\\ Compared to the cross-sectional case, the joint convergence condition are generally weaker for (PA-1) for two reasons. Firstly, the fact that we observe $\Delta Y$ only requires one projection on the difference of the outcomes instead of the difference between two projections on each single outcome. Secondly, the efficient influence function in Theorem (ref) implies that $m_{\Delta Y}(1,X)$ is redundant. This allows us to obtain efficient difference-in-differences estimators for the panel under relatively weak conditions.

Redundancy of nuisance parameters

For the cross-sectional difference-in-differences estimators considered so far some nuisance parameters are redundant. For example in setting (CS-1) for all $x\in\mathcal{X}$ we have $\sum_{d=0}^1\sum_{t=0}^1p_{D=d,T=t}(x)=1$. In principle, we could therefore infer one of the four propensity scores from the other three. A well-known problem is that if one estimates the three propensity scores without requiring that the implied four propensity scores sum to one, Assumption (ref) might be violated because the fourth, implied propensity score is not guaranteed to be strictly greater than zero. A solution to this could be to explicitly require that the four propensity scores estimators sum to one for all $x\in\mathcal{X}$. Multinomial versions of the standard Logit regression or the Logit Lasso are available. To the knowledge of the author, solutions do not exist for more sophisticated machine-learning algorithms like Random Forests or Neural Nets. However, notice that $p_{D=d,T=t}(X)=p_{D=d}(t,X)p_{T=t}(X)=p_{T=t}(d,X)p_{D=d}(X)$ and that the propensity score estimators obey $\sum_{d=0}^1\sum_{t=0}^1\hat{p}_{D=d}(t,x)_{-k}\hat{p}_{T=t}(x)_{-k}=\sum_{d=0}^1\sum_{t=0}^1\hat{p}_{T=t}(d,x)_{-k}\hat{p}_{D=d}(x)_{-k}=1$ for all $x\in\mathcal{X}$ by construction. Corollary (ref) summarizes the properties of the two implied estimators.

Proof: The proof follows similarly to Corollary (ref) (a).\\ Compared to Corollary (ref) (a), the joint convergence conditions are increased from six to seven and the efficiency result is maintained. The scores rely on either $p_{D=d}(t,X)$ or $p_{T=t}(d,X)$. We therefore recommend to use $\psi^{**}_{CS-1}(W,\eta;\theta)$ when $\lvert p_T-0.5\rvert<\lvert p_D-0.5\rvert$ and $\psi^{***}_{CS-1}(W,\eta;\theta)$ otherwise.\\ The result of Corollary (ref) (b)-(d) indicate that some of the outcome nuisances in settings (CS-2), (CS-3) and (CS-4) are redundant in the sense that they do not contribute to the joint convergence condition in Assumption (ref) (v). However, the outcome nuisances still need to converge in $L_2$ in order to satisfy Assumption (ref) (iv). The same applies for (CS-5) since the simple difference-in-means estimator represents an alternative estimator that does not rely on any first stage convergence conditions. We do not provide a separate result for case (CS-5). However, it is easy to see that the implied estimator without any outcome nuisances is just the difference-in-means estimator. Notice that as long as $Y$ is not independent of $X$ the difference-in-means estimator is not an efficient estimator in a semiparametric sense. This has some practical relevance as in difference-in-differences the credibility of the design is often assessed by using placebo tests in some pre-periods without relying on covariates. Not rejecting the null might, however, just due to a higher than necessary standard error when using the difference-in-means estimator. The same applies for setting (PA-2). Since extensions for (CS-3) are also trivial, we focus on settings (CS-2) and (CS-4). Corollaries (ref) and (ref) summarize the implications when scores are used that do not contain some of the outcome nuisances. Generally, the implied cross-fitted sample plug-in estimators do not attain the variance lower bound but have otherwise desirable asymptotic properties under weaker conditions. This hints at another trade-off between the robustness of the estimator towards first stage convergence requirements and semiparametric efficiency.

Proof: see Appendix (ref).\\ Corollary (ref) shows that for (CS-2) one may in principle get rid off the restrictive existence of higher-order moments requirement and the $L_2$ convergence condition for $m_Y(1,1,x)$ in Corollary (ref) (b). However, this results in an efficiency loss. Notice that from Corollary (ref) $\Delta_{CS-1,CS-2}$ represents the value of knowing that (CS-2) in Assumption (ref) is true relative to not making any assumptions. Hence, when using score $\psi'_{CS-2}(W;\theta)$ all efficiency gains from the stronger setting (CS-2) relative to (CS-1) are exchanged for relatively weak first stage convergence conditions.

Proof: see Appendix (ref).\\ Corollary (ref) provides the convergence conditions when we use a score where $m_Y(1,1,x)$ and $m_Y(1,0,x)$ are redundant. This results in convergence conditions similar to those of the score $\psi^{*}_{PA-1}(W;\theta)$. However, also the efficiency loss can be relatively huge.

Application

To illustrate the practical relevance of the proposed method, we revisit Angrist_Acemoglu_2001. The paper is concerned with the theoretically ambiguous effect of increased employment protection for disabled workers on weeks worked (for more details see the paper). An empirical evaluation of the Americans with Disabilities Act reform introduced in 1991 is used to test the theory using data from the Current Population Survey (CPS), a repeated cross-section.\\ As in the paper, we define $D$ as being disabled. $Y$ are the weeks worked in the respective year. The outcome is therefore bounded between 0 and 52. In the original paper several post-reform years are considered. Since the credibility of the common trend assumption might be questionable for years well after the reform, we focus on 1992 as the post-reform period ($T=1$). As in the original paper we use the years 1988-1990 for $T=0$. All CPS data is retrieved from Joshua Angrist's data archive\footnote{\href{https://economics.mit.edu/faculty/angrist/data1/data/aceang01}{https://economics.mit.edu/faculty/angrist/data1/data/aceang01}}. We consider three different sets of covariates. An overview of the different variable sets is provided in Table (ref).

The variable set labelled as `original' is constructed using the covariates that are also included in the specifications of Angrist_Acemoglu_2001. The variable sets `baseline' and `extended' use some further covariates available from the CPS. The `baseline' specification disaggregates the region variable used in the original dataset to control for geographically different common trends. Instead, we include state dummies and dummies indicating whether the individual lives in a metropolitain area. Additional controls on marital status, class of worker, industry and occupation should help to control for non-parallel trends between disabled and non-disabled. For example, if people who report a disability self-select more often into a certain sector or industry then any underlying structural change in this sector needs to be controlled for in order to guarantee the common trend assumption to hold. This might be a point that is of more general interest in empirical economics. In labour market applications the common trend is often valid only after conditioning on variables that are available at different aggregation levels e.g. geographic or sector dummies. The `extended' specification includes further covariates on employment history, social welfare payments and health insurance status.

Table (ref) shows that the merged sample consists of 276127 observations (for all specifications) and is highly imbalanced. We observe that the share of $D$ varies between $T=0$ and $T=1$. Given the large sample size, this indicates that assumptions (CS-4) and (CS-5) are likely to be violated.

Estimators considered

We consider different estimators using the different score functions outlined in Section (ref) with different first stage estimators.\\ Our preferred estimator is an Ensemble Learner that weights Lasso and Random Forest predictions by using out-of-sample MSE optimal weights. For the Lasso we allow for polynomials up to order four and all two way interactions. The Random Forest is an ensemble of regression trees and therefore implicitly contains higher-order terms. For both estimators we use the default settings in the glmnet and ranger R-packages. Using an ensemble of machine-learning estimators has several important advantages. Firstly, Lasso and Random Forests are designed for different DGPs. Whereas the Lasso allows for some form of smoothing, we expect a tree-based estimator to work well with strong non-linearities. Secondly, the Ensemble gives more weight to the single predictor that works best and therefore should be less dependent on the particular tuning parameter choices of the Lasso and the Random Forest. Thirdly, the strong imbalances in our sample indicate that using single propensity score estimators might result in extreme weights. By combining two predictors the likelihood of generating extreme weights is reduced.\\ We compare the Ensemble Learner to the single predictors Lasso and Random Forest. Due to computational constraints, we restrict this analysis to the `baseline' set of covariates. Further, we compare the Ensemble to parametric models for the propensity scores and the outcome nuisances. For the propensity scores we use Logit and for the outcome nuisances standard linear regression. We include covariates in levels. As a benchmark, we also consider Abadie_2005's (Abadie_2005) Inverse Probability Weighting (IPW) difference-in-differences estimator with Logit regression for the propensity score.

Placebo tests

As usual in the difference-in-differences literature, we run some placebo experiments for period 1988/89.

Table (ref) shows the results under (CS-5). Notice that the simple differences-in-means estimator does not hint at a violation of the specification that does not account for any imbalances. This is not because the effect is economically small but due to the comparatively high standard error. If the efficient score $\psi^*_{CS-5}(W,\eta;\theta)$ is used, the point estimators are in the same range but -- in line with theory in Section (ref) -- the standard errors are substantially decreased leading to significant results. This shows that using the efficient score function rather than the simple mean differences estimator may lead to opposite conclusions regarding the credibility of the design.\footnote{We notice that the asymptotic standard error might, however, be a more accurate approximation of the finite sample standard error in case of the difference-in-means estimator.} The result also highlights the importance of discriminating the two roles covariates can have in semiparametric difference-in-differences estimation. First of all, they may be included to improve the reliability of the Conditional Common Trends (see Assumptions (ref) and (ref)). Second of all, under some assumptions, they should be included to improve on the efficiency of the derived estimator. Further, we notice that the standard error for the Ensemble Learner decreases when more covariates are added to the model, whereas the standard error when using the linear model increases for the `extended' specification. The standard errors are also generally larger. This indicates that the Ensemble Learner is more effective in predicting the conditional outcomes leading to a stronger variance reduction of the residualized estimator.\\ From these results we conclude that incorporating covariates seems to be necessary in our example. Notice that due to the strong imbalances in our application, especially propensity scores for $D=1$, $T=1$ are hard to predict. Results of placebo tests for our estimators are reported in Table (ref). For estimators that rely on the Ensemble Learner none of the different specifications used hint at a violation of the conditional common trend assumption. Moreover, the standard errors are reduced when a higher number of covariates is included, indicating that the Ensemble is effective in extracting the additional information. When covariates in `baseline' and `extended' are used, estimators under (CS-2) and (CS-4) show substantially decreased standard errors compared to estimators that rely on (CS-1). This is in line with the theoretical argument in Corollary (ref). When using the same scores with the same set of variables most estimators that rely on parametric models hint at a potential violation of the conditional common trend assumption. With the exception of IPW, estimators implied by scores that only rely on the propensity scores $p_D(X)$ and $p_T(X)$ ((CS-2), (CS-4)) generally do not indicate a violation of the conditional common trend assumption. The same can be observed for the single nuisance predictors. We conclude that scores that rely on sophisticated convergence conditions are less robust to potential violations of these conditions in practice. However, it seems that the efficiency-estimation trade-off seems to be less of a concern when high-quality predictors like the Ensemble are used. Moreover, even when using a parametric model and a large number of covariates the double robust scores give results whereas IPW explodes for the `extended' specification.

Main results

The placebo tests for the different estimators suggest some trustworthiness of estimators that use the Ensemble Learner for first stage prediction. Table (ref) summarizes the main results for the difference-in-differences estimators considered. For some of the specifications for the single predictors and the linear estimators we obtain extreme results. In general the same conclusions as for the placebo test apply. We notice that the particular first stage estimator and the estimation-robustness of the score used can drastically shift the results. Whereas estimators for (CS-1)-(CS-3) that use the Ensemble Learner unambiguously give negative or insignificant effects of reasonable size, again some of the other estimators explode. In addition, the Ensemble Learner based estimators mostly become insignificant when more covariates are included in the model. In contrast, to estimators with parametric first stages, this is not due to an increase in standard errors if more covariates are included in the model. Rather, the decreased standard errors indicate that the Ensemble Learner allows to effectively exhaust the information of growing covariate sets. Lastly, we notice that for specifications `baseline' and `extended' the efficiency-robustness trade-off from Corollary (ref) becomes relevant. Scores under setting (CS-1) generally have higher standard errors compared to efficient scores in settings (CS-2), (CS-3) and (CS-4). While for (CS-4) using $\psi^{'}_{CS-4}(W,\eta;\theta)$ leads to a substantially higher standard error compared to the efficient score, for (CS-2) there is barely a difference. Also the point estimators are similar. This is expected since the score $\psi^{'}_{CS-2}(W,\eta;\theta)$ converges under similar conditions when the outcome is bounded.

Conclusion

Semiparametric difference-in-differences estimation is a non-trivial endeavour. In this study we highlight the importance of different assumptions in semiparametric difference-in-differences models. Our results show that efficiency bounds may strongly depend on the model assumptions imposed and the data that is available. In particular, we show that there is a trade-off between the strength of the assumptions imposed and the variance lower bound that can be achieved. For estimation we provide easy-to-check conditions to derive the required convergence rates for a broad class of estimation problems. Our theoretical results allow to integrate scores with sophisticated adjustment terms in the double machine-learning framework. Further, we show that the different semiparametric models imply estimators with different properties. Estimators that are more robust against the model assumptions imposed also rely on more sophisticated conditions for first stage prediction. Some of these conditions can be relaxed when we give up on asymptotically attaining the efficiency bound. An empirical example shows that our proposed estimators are useful in practice. However, estimation results might be highly sensitive regarding the choice of the first stage nuisance parameter predictor. Placebo tests indicate that, in contrast to other choices, our proposed Ensemble Learner performs well.\\ Some interesting problems are beyond the scope of this study and have to be left for further research. The performance of the estimators proposed might depend on the parameter $K$. Some theoretical results how to optimally choose this parameter would be helpful. Our theoretical results for the different estimators suggest that the finite sample performance of the point estimators and the coverage probabilities of the variance estimators might be relatively diverse and depend on the particular DGP considered. Monte Carlo simulations could shed some more light on this subject. Also this study is limited to two time periods and two groups. Some recent advances in the semiparametric difference-in-differences literature (e.g., Callaway_SantAnna_2018, GoodmanBacon_2018) comprise extensions to more complicated adoption patterns. Further, in practice panel and cross-sectional data are often combined (e.g. in rotating panels). The results in Section (ref) suggest that for these kind of data combination problems efficiency gains are possible if neither the panel structure is neglected nor the cross-sectional data is thrown away. A generalization of the efficiency theory provided in this study to these settings represents yet another avenue for further research. \printbibliography