Semiparametric Difference-in-Differences with Potentially Many Control Variables

Semiparametric Difference-in-Differences with Potentially Many Control Variables

Introduction

The difference-in-differences (DID) estimator has been widely used in empirical economics to evaluate causal effects when there exists a natural experiment with a treated group and an untreated group. By comparing the variation over time in an outcome variable between the treated group and the untreated group, the DID estimator can be used to calculate the effect of treatment on the outcome variable. Applications of DID include but are not limited to studies of the effects of immigration on labor markets \citep*{card1990impact}, the effects of minimum wage law on wages \citep*{david1994minimum}, the effect of tariffs liberalization on corruption \citep*{sequeira2016corruption}, the effect of household income on children's personalities \citep*{akee2018does}, and the effect of corporate tax on wages \citep*{fuest2018higher}.

The traditional linear DID estimator depends on a parallel trend assumption that in the absence of treatment, the difference of outcomes between treated and untreated groups remains constant over time. In many situations, however, this assumption may not hold because there are other individual characteristics that may be associated with the variations of the outcomes. The treatment may be taken as exogenous only after controlling these characteristics. To address this problem, \citet*{abadie2005semiparametric} proposed the semiparametric DID estimators. Compared to the traditional linear DID estimators, the advantages of Abadie's estimators are threefold. First, the characteristics are treated nonparametrically so that any estimation error caused by functional specification is avoided. Second, the effect of treatment is allowed to vary among individuals, while the traditional linear DID estimator does not allow this heterogeneity. Third, the estimation framework proposed in \citet*{abadie2005semiparametric} allows researchers to estimate how the effect of treatment varies with changes in the characteristics.

This paper is an extension of \citet*{abadie2005semiparametric}. \citet*{abadie2005semiparametric} considered the case where the number of control variables has to be limited. A practical difficulty empirical researchers encounter is choosing what variables to include when there is a rich data set. Although economic intuition can help us narrow down the choice set, it will not completely select all the important variables. This variable selection problem may lead to the chance of omitted variables in practice. In this paper, I consider the DID estimation with many control variables, potentially more than the sample size. The classical estimation methods which require a fixed number of variables do not work in this situation. One has to consider using ML methods such as Lasso, Logit Lasso, random forests, boosted trees, or various hybrids. However, by the well-known theory of inference of ML methods developed in \citet*{chernozhukov2018double}, if one directly applies ML methods to the conventional semiparametric DID estimators proposed in \citet*{abadie2005semiparametric}, the result will lead to significant bias and invalid inference. In particular, the regularization bias embedded in ML methods will result in the conventional semiparametric DID estimators failing to be $\sqrt{N}$-consistent.

I contribute to the literature by proposing three new DID estimators for three different data structures: repeated outcomes, repeated cross-sections, and multilevel treatment. These new estimators can relieve the impact of the regularization bias of ML methods and achieve $\sqrt{N}$-consistency. The key is to find the so-called Neyman-orthogonal scores \citep*{chernozhukov2018double} of \citet*{abadie2005semiparametric}'s estimands. The Neyman-orthogonal score is a function that identifies the parameter of interest, and its derivatives with respect to the nuisance parameters are zero. This property helps us remove the first-order bias caused by ML methods so that only the second-order bias remains, which is much smaller and easier to control than the first-order bias as in the conventional semiparametric DID estimators. Using the cross-fitting algorithm in \citet*{chernozhukov2018double}, I show that the new DID estimators can be $\sqrt{N}$-consistent and asymptotically normal when using ML methods. Figure 1 presents a Monte Carlo simulation that illustrates the negative effect of directly combining ML methods with Abadie's estimator and the benefit of using the newly proposed DID estimator.

The second contribution is concerned with the conventional semiparametric DID estimators with a limited number of control variables considered in \citet*{abadie2005semiparametric}. In this case, the conventional semiparametric DID estimators are able to achieve $\sqrt{N}$-consistency using kernel estimators, but they will require undersmoothing. Undersmoothing is a condition that requires the pointwise bias of the kernel estimators to converge to zero faster than the pointwise standard deviation. This condition will be violated if researchers use standard data-driven methods, such as cross-validation (CV), to choose the bandwidths of kernel estimators because those methods do not undersmooth.

In this paper, I show that the new estimators do not require undersmoothing to achieve $\sqrt{N}$-consistency. Specifically, I will show that the new estimators have the small bias property (SBP), in terms of \citet*{newey2004twicing}, meaning that the bias of the new estimators will converge to zero faster than the pointwise bias of the nonparametric estimator on which it is based. The SBP, as shown in \citet*{chernozhukov2016locally}, is a sufficient condition to remove the undersmoothing requirement. Figure 2 shows the Monte Carlo simulation results of Abadie's estimator and the new estimator with bandwidths chosen by CV. We can observe that Abadie's estimator is biased since CV does not undersmooth, and the newly proposed estimator can correct this bias.

As an empirical example, I study the effect of tariff reduction on corruption behavior using the trade data between South Africa and Mozambique during 2006 and 2014. The treatment is the large tariff reduction on certain commodities occurring in 2008. This natural experiment was previously studied by \citet*{sequeira2016corruption} using the traditional linear DID estimator. I apply my proposed semiparametric DID estimator and \citet*{abadie2005semiparametric}'s semiparemetric DID estimator on the same data set (Table 9 of \citet*{sequeira2016corruption}). In comparison to \citet*{sequeira2016corruption} that a decrease in tariff rate will decrease corruption behavior, the two semiparametric estimators consistently suggest that the effect is actually substantially larger than previously reported by \citet*{sequeira2016corruption}. A potential explanation for this difference is that the true data generating process violates the linear specification assumed in the traditional linear DID estimator. In addition, when compared to \citet*{abadie2005semiparametric}'s estimator, my proposed estimator shows that the effect is even larger.

The new estimators proposed in this paper heavily rely on the recent high-dimensional and ML literature: \citet*{belloni2012sparse}, \citet*{Belloni14restud}, \citet*{chernozhukov2015valid}, \citet*{belloni2017program}, and \citet*{chernozhukov2018double}; and the literature of the SBP in semiparametric estimation: \citet*{newey1998undersmoothing, newey2004twicing} and \citet*{chernozhukov2016locally}.

Plan of the paper. Section 2 describes the conventional semiparametric DID estimators and discusses their limitations when applying ML methods. Section 3 presents the new DID estimators and discusses their theoretical properties. Section 4 conducts Monte Carlo simulation to shed some light on the finite sample performance of the proposed estimators. Section 5 provides an application, and Section 6 concludes the paper.

The Conventional Semiparametric DID Estimators

Let $Y_{i}\left(t\right)$ be the outcome of interest for individual $i$ at time $t$ and $D_{i}\left(t\right)\in\left\{ 0,1\right\} $ the treatment status. The population is observed in a pre-treatment period $t=0$, and in a post-treatment period $t=1$. With potential outcome notations \citep*{rubin1974estimating}, we have $Y_{i}\left(t\right)=Y_{i}^{0}\left(t\right)+\left(Y_{i}^{1}\left(t\right)-Y_{i}^{0}\left(t\right)\right)D_{i}\left(t\right)$, where $Y_{i}^{0}\left(t\right)$ is the outcome that individual $i$ would attain at time $t$ in the absence of the treatment, and $Y_{i}^{1}\left(t\right)$ represents the outcome that individual $i$ would attain at time $t$ if exposed to the treatment. Since individuals are only exposed to treatment at $t=1$, we have $D_{i}\left(0\right)=0$ for all $i$. To reduce notation, I define $D_{i}\coloneqq D_{i}\left(1\right)$. Also, let $X_{i}\in\mathbb{R}^{d}$ be a vector of control variables with dimension $d$ potentially larger than the sample size $N$.

The traditional linear DID estimator is the parameter $\alpha$ in the following linear model \[ Y_{i}\left(t\right)=\mu+X_{i}'\pi\left(t\right)+\tau\cdot D_{i}+\delta\cdot t+\alpha\cdot D_{i}\left(t\right)+\varepsilon_{i}\left(t\right), \] where $\varepsilon_{i}\left(t\right)$ is an exogenous shock that has mean zero and $\left(\mu, \pi\left(t\right), \tau, \delta\right)$ are the corresponding parameters. Clearly, the linear specification assumed here is a strong assumption since the true data generating process may be nonlinear. In addition, \citet*{meyer1995workers} noticed that including control variables in this linear form may not be appropriate if the treatment has different effects for different groups in the population. To deal with these problems, \citet*{abadie2005semiparametric} proposed the semiparametric DID estimators which can identify average treatment effect on the treated (ATT) \[ \theta_{0}\coloneqq E\left[Y_{i}^{1}\left(1\right)-Y_{i}^{0}\left(1\right)\mid D_{i}=1\right]. \] According to the data, there are three particular cases.

Case 1: Random sample with repeated outcomes

Consider the case that researchers can observe both pre-treatment and post-treatment outcomes for each individual of interest. That is, researchers observe $\left\{ Y_{i}\left(0\right),Y_{i}\left(1\right),D_{i},X_{i}\right\} _{i=1}^{N}$. In this case, the ATT can be identified under the following assumptions \citep*{abadie2005semiparametric}:

Assumption (2.1) is the conditional parallel trend assumption. It states that conditional on individual's characteristics, the average outcomes for treated and untreated groups would have followed parallel paths in the absence of treatment. With these two assumptions, the ATT is identified \citep*{abadie2005semiparametric} as \[ \theta_{0}=E\left[\frac{Y_{i}\left(1\right)-Y_{i}\left(0\right)}{P\left(D_{i}=1\right)}\frac{D_{i}-P\left(D_{i}=1\mid X_{i}\right)}{1-P\left(D_{i}=1\mid X_{i}\right)}\right].\tag{2.1} \]

Case 2: Random sample with repeated cross sections

Often times, researchers may not be able to observe both pre-treatment and post-treatment outcomes of the same individual. Instead, they observe repeated cross-section data sets. Let $T_{i}$ be a time indicator that takes value one if the observation belongs to the post-treatment sample. Researchers observe $\left\{ Y_{i},D_{i},T_{i},X_{i}\right\} _{i=1}^{N}$, where $Y_{i}=Y_{i}\left(0\right)+T_{i}\left(Y_{i}\left(1\right)-Y_{i}\left(0\right)\right)$.

Suppose Assumptions (2.1)-(2.3) hold, the ATT is identified \citep*{abadie2005semiparametric} as \[ \theta_{0}=E\left[\frac{T_{i}-\lambda_{0}}{\lambda_{0}\left(1-\lambda_{0}\right)}\frac{Y_{i}}{P\left(D_{i}=1\right)}\frac{D_{i}-P\left(D_{i}=1\mid X_{i}\right)}{1-P\left(D_{i}=1\mid X_{i}\right)}\right],\tag{2.2} \] where $\lambda_{0}\coloneqq P\left(T_{i}=1\right).$

Case 3: Multilevel treatments

In many cases, individuals can be exposed to different levels of treatment. Let $W\in\left\{0, w_{1},...,w_{J}\right\} $ be the level of treatment, where $W=0$ denotes the untreated individuals. Researchers observe $\left\{ Y_{i}\left(0\right),Y_{i}\left(1\right),W_{i},X_{i}\right\} _{i=1}^{N}$.

For $w\in\left\{ 0,w_{1},...,w_{J}\right\} $ and $t\in\left\{ 0,1\right\} $, let $Y^{w}\left(t\right)$ be the potential outcome for treatment level $w$ at period $t$. Denote the ATT for each level of treatment $w$ by \[ \theta_{0}^{w}\coloneqq E\left[Y^{w}\left(1\right)-Y^{0}\left(1\right)\mid W=w\right]. \] Suppose that Assumptions (2.1) and (2.2) hold for each level of treatment: \[ E\left[Y_{i}^{0}\left(1\right)-Y_{i}^{0}\left(0\right)\mid X_{i},W_{i}=w\right]=E\left[Y_{i}^{0}\left(1\right)-Y_{i}^{0}\left(0\right)\mid X_{i},W_{i}=0\right] \] for $w\in\left\{ w_{1},...,w_{J}\right\} $ and $P\left(W_{i}=w\right)>0$ and with probability one $P\left(W_{i}=w\mid X_{i}\right)<1$ for $w\in\left\{ w_{1},...,w_{J}\right\} $. Then we have \citep*{abadie2005semiparametric} \[ \theta_{0}^{w}=E\left[\frac{Y\left(1\right)-Y\left(0\right)}{P\left(W=w\right)}\frac{I\left(W=w\right)\cdot P\left(W=0\mid X\right)-I\left(W=0\right)\cdot P\left(W=w\mid X\right)}{P\left(W=0\mid X\right)}\right],\tag{2.3} \] where $I\left(\cdot\right)$ is an indicator function.

Let us focus on Case 1 in which researchers confront repeated outcomes data. To use the identification result (2.1), the first step is to estimate the two nuisance parameters: $P\left(D_{i}=1\right)\eqqcolon p_{0}$ and $P\left(D_{i}=1\mid X_{i}\right)\eqqcolon g_{0}\left(X_{i}\right)$. The estimator of $p_{0}$ is just a sample average $\hat{p}=N^{-1}\sum_{i=1}^{N}D_{i}$, while the propensity score $g_{0}$ is infinite-dimensional and needs to be estimated nonparametrically. Denote by $\hat{g}$ the estimator of $g_{0}$, then the plug-in estimator based on equation (2.1) is \[ \hat{\theta}=\frac{1}{N}\sum_{i=1}^{N}\frac{Y_{i}\left(1\right)-Y_{i}\left(0\right)}{\hat{p}}\frac{D_{i}-\hat{g}\left(X_{i}\right)}{1-\hat{g}\left(X_{i}\right)}. \] When $\hat{g}$ is estimated using classical nonparametric methods such as kernel or series estimators, the estimator $\hat{\theta}$ can be $\sqrt{N}$-consistent and asymptotically normal under certain conditions provided in the semiparametric estimation literature \citep*{newey1994asymptotic, newey1994large}.

When $\hat{g}$ is an ML estimator, however, the estimator $\hat{\theta}$ will fail to be $\sqrt{N}$-consistent in general. By the general theory of inference of ML methods developed in \citet*{chernozhukov2018double}, the reason is twofold : (1) the score function based on (2.1), $\varphi\left(W,\theta_{0},p_{0},g_{0}\right)\coloneqq\frac{Y\left(1\right)-Y\left(0\right)}{P\left(D=1\right)}\frac{D-g_{0}\left(X\right)}{1-g_{0}\left(X\right)}-\theta_{0}$, has a non-zero directional (Gateaux) derivative with respect to the propensity score $g_{0}$: $$\partial_{g} E\left[\varphi\left(W,\theta_{0},p_{0},g_{0}\right)\right]\left[g-g_{0}\right]\neq0,$$ where the directional (Gateaux) derivative is formally defined in Section 3; (2) ML estimators usually have a convergence rate slower than $N^{-1/2}$ due to regularization bias. Similarly, the estimators obtained by directly plugging ML estimators into (2.2) and (2.3) will not be $\sqrt{N}$-consistent in general. The Monte Carlo simulation in Section 4 supports this theoretical insight and reveals significant bias on the estimators based on (2.1)-(2.3) when using ML estimators in the first-stage nonparametric estimation.

The next section proposes three new score functions to relieve the regularization bias of the first-stage ML estimators. These three new score functions are derived under the same identification assumptions as those in \citet*{abadie2005semiparametric}, so that no extra assumption is made. Heuristically, a distinctive feature of the new score functions is that their derivatives with respect to their infinite-dimensional nuisance parameters are zero. This property can help us remove the first-order bias of the first-stage estimation so that the bias of the estimators based on these new score functions will be much smaller. In addition, I will use the cross-fitting algorithm to improve the over-fitting phenomena that frequently arise when using highly adaptive ML methods \citep*{chernozhukov2018double}.

The New DID Estimators

The Main Algorithm

Supposing Assumptions (2.1)-(2.3) hold, consider the following three new score functions.

Case 1: Random sample with repeated outcomes

The new score function for repeated outcomes is

with the unknown constant $p_{0}$ and the infinite-dimensional nuisance parameter \[ \eta_{10}=\left(P\left(D=1\mid X\right),E\left[Y\left(1\right)-Y\left(0\right)\mid X,D=0\right]\right)\eqqcolon\left(g_{0},\ell_{10}\right). \]

Case 2: Random sample with repeated cross sections

The new score function for repeated cross sections is \[ \psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)=\frac{T-\lambda_{0}}{\lambda_{0}\left(1-\lambda_{0}\right)}\frac{Y}{P\left(D=1\right)}\frac{D-P\left(D=1\mid X\right)}{1-P\left(D=1\mid X\right)}-\theta_{0}-c_{2},\tag{3.2} \] where the adjustment term is \[ c_{2}=\frac{D-P\left(D=1\mid X\right)}{\lambda_{0}\left(1-\lambda_{0}\right)\cdot P\left(D=1\right)\cdot\left(1-P\left(D=1\mid X\right)\right)}\times E\left[\left(T-\lambda_{0}\right)Y\mid X,D=0\right]. \] The nuisance parameters are the unknown constants $p_{0}$ and $\lambda_{0}$, and the infinite-dimensional parameter \[ \eta_{20}=\left(P\left(D=1\mid X\right),E\left[\left(T-\lambda\right)Y\mid X,D=0\right]\right)\eqqcolon\left(g_{0},\ell_{20}\right). \]

Case 3: Multilevel treatment

For each $w\in\left\{ w_{1},...,w_{J}\right\} $, the new score function for multilevel treatment is

where the adjustment term is

The nuisance parameters are the unknown constant $p_{w0}\coloneqq P\left(W=w\right)$ and the infinite-dimensional parameter \[ \eta_{w0}=\left(P\left(W=w\mid X\right),P\left(W=0\mid X\right),E\left[Y\left(1\right)-Y\left(0\right)\mid X,I\left(W=0\right)=1\right]\right)\eqqcolon\left(g_{0w},g_{0z},\ell_{30}\right). \]

Notice that the above three new functions are equal to the original score functions (2.1)-(2.3) plus the adjustment terms, $\left(c_{1},c_{2},c_{w}\right)$, which have zero expectations. Thus, the new score functions (3.1)-(3.3) still identify the ATT in each case. I will use these new scores to construct new DID estimators.

To avoid repetition, I will focus on the estimation of ATT when data belongs to repeated outcomes and repeated cross sections. The estimation of multilevel treatment is provided in appendix. Now I combine the score functions described above with the cross-fitting estimation algorithm of \citet*{chernozhukov2018double}.

The estimators $\left(\hat{g}_{k},\hat{\ell}_{1k},\hat{\ell}_{2k}\right)$ can be constructed using any ML methods or classical estimators such as kernel or series estimators. For completeness, I present the Logit Lasso and Lasso estimators here.

Consider a class of approximating functions of $X_{i}$, \[ q_{i}\coloneqq\left(q_{i1}\left(X_{i}\right),...,q_{ip}\left(X_{i}\right)\right)'. \] For example, $q_{i}$ can be polynomials or B-splines. Let $\Lambda\left(u\right)\coloneqq1/\left(1+\exp\left(-u\right)\right)$ be the cumulative distribution function of the standard Logistic distribution, construct the estimator of the propensity score $g_{0}$ by \[ \hat{g}_{k}\left(x_{i}\right)\coloneqq\Lambda\left(q_{i}'\hat{\beta}_{k}\right),\tag{3.4} \] where \[ \hat{\beta}_{k}\coloneqq\arg\min_{\beta\in\mathbb{R}^{p}}\frac{1}{M}\sum_{i\in I_{k}^{c}}\left\{ -D_{i}(q_{i}'\beta)+\log\left(1+\exp\left(q_{i}'\beta\right)\right)\right\} +\lambda_{k}\parallel\beta\parallel_{1} \] is the Logit Lasso estimator and $M=N-n$ is the sample size of the auxiliary sample $I_{k}^{c}$. Next, define $I_{kz}^{c}\coloneqq I_{k}^{c}\cap\left\{ i:D_{i}=0\right\} $, $M_{k}$ the sample size of $I_{kz}^{c}$. Construct the estimators of $\ell_{10}$ and $\ell_{20}$ by \[ \hat{\ell}_{1k}\left(x_{i}\right)\coloneqq q_{i}'\hat{\beta}_{1k}, \] \[ \hat{\ell}_{2k}\left(x_{i}\right)\coloneqq q_{i}'\hat{\beta}_{2k}, \] where \[ \hat{\beta}_{1k}\in\arg\min_{\beta\in\mathbb{R}^{p}}\left[\frac{1}{M_{k}}\sum_{i\in I_{kz}^{c}}\left(Y_{i}\left(1\right)-Y_{i}\left(0\right)-q_{i}'\beta\right)^{2}\right]+\frac{\lambda_{1k}}{M_{k}}\parallel\hat{\Upsilon}_{1k}\beta\parallel_{1} \] and \[ \hat{\beta}_{2k}\in\arg\min_{\beta\in\mathbb{R}^{p}}\left[\frac{1}{M_{k}}\sum_{i\in I_{kz}^{c}}\left(\left(T_{i}-\hat{\lambda}_{k}\right)Y_{i}-q_{i}'\beta\right)^{2}\right]+\frac{\lambda_{2k}}{M_{k}}\parallel\hat{\Upsilon}_{2k}\beta\parallel_{1} \] are the modified Lasso estimators proposed in \citet*{belloni2012sparse}. The choices of the penalty levels and loadings $\left(\lambda_{1k},\lambda_{2k},\hat{\Upsilon}_{1k},\hat{\Upsilon}_{2k}\right)$ suggested by \citet*{belloni2012sparse} are provided in appendix.

Theoretical Properties

In this section, I discuss the theoretical properties of the new DID estimator $\tilde{\theta}$. In particular, I will show that the estimator $\tilde{\theta}$ can achieve $\sqrt{N}$-consistency and asymptotic normality as long as the first-stage estimators converge at rates faster than $N^{-1/4}$. This rate of convergence can be achieved by many ML methods, including Lasso and Logit Lasso. Further, I will show that when using kernel estimators in the first-stage estimation, the estimator $\tilde{\theta}$ has the SBP while the conventional semiparametric DID estimators do not.

The Neyman Orthogonality

The differences between the new DID estimators and the conventional semiparametric DID estimators in \citet*{abadie2005semiparametric} are the score functions on which they are based. The key property of the new score functions (3.1)-(3.3) is that their directional (or the Gateaux) derivatives with respect to their infinite-dimensional nuisance parameters are zero, while the scores based on (2.1)-(2.3) do not have this property. This property is the so-called Neyman orthogonality in \citet*{chernozhukov2018double}. The Neyman orthogonality enables us to remove the first-order bias of the first-stage estimation so that the estimators based on these Neyman-orthogonal scores can achieve $\sqrt{N}$-consistency under less restrictive conditions.

The definition of the Neyman-orthogonal score provided here is slightly different from \citet*{chernozhukov2018double} that instead of being orthogonal against all nuisance parameters, the Neyman-orthogonal score defined here is orthogonal against only those infinite-dimensional nuisance parameters. Formally, let $\theta_{0}\in\Theta$ be the low-dimensional parameter of interest, $\rho_{0}$ be the true value of the finite-dimensional nuisance parameter $\rho$, and $\eta_{0}$ the true value of the infinite-dimensional nuisance parameter $\eta\in\mathcal{T}$ . Suppose that $W$ is a random element taking values in a measurable space $\left(\mathcal{W},\mathcal{A}_{\mathcal{W}}\right)$ with probability measure $P$. Define the directional (or the Gateaux) derivative against the infinite-dimensional nuisance parameter $D_{r}:\tilde{\mathcal{T}}\rightarrow\mathbb{R}$, where $\tilde{\mathcal{T}}=\left\{ \eta-\eta_{0}:\eta\in\mathcal{T}\right\} $, \[ D_{r}\left[\eta-\eta_{0}\right]\coloneqq\partial_{r}\left\{ E_{P}\left[\psi\left(W,\theta_{0},\rho_{0},\eta_{0}+r\left(\eta-\eta_{0}\right)\right)\right]\right\} ,\eta\in\mathcal{T}, \] for all $r\in[0,1)$. For convenience, denote \[ \partial_{\eta}E_{P}\psi\left(W,\theta_{0},\rho_{0},\eta_{0}\right)\left[\eta-\eta_{0}\right]\coloneqq D_{0}\left[\eta-\eta_{0}\right],\eta\in\mathcal{T}. \] In addition, let $\mathcal{T}_{N}\subset\mathcal{T}$ be a nuisance realization set such that the estimator of $\eta_{0}$ take values in this set with high probability.

The score $\psi$ obeys the Neyman orthogonality condition at $\left(\theta_{0},\rho_{0},\eta_{0}\right)$ with respect to the nuisance parameter realization set $\mathcal{T}_{N}\subset\mathcal{T}$ if the directional derivative map $D_{r}\left[\eta-\eta_{0}\right]$ exists for all $r\in[0,1)$ and $\eta\in\mathcal{T}_{N}$ and vanishes at $r=0$: \[ \partial_{\eta}E_{P}\psi\left(W,\theta_{0},\rho_{0},\eta_{0}\right)\left[\eta-\eta_{0}\right]=0,\text{for all }\eta\in\mathcal{T}_{N}. \]

This property embedded in (3.1)-(3.3) will play the key role to make less restrictive assumptions in the following proofs of asymptotic distribution and the SBP.

Asymptotic Distribution

In the following, I will discuss the theoretical properties of the new estimator $\tilde{\theta}$ when data belongs to repeated outcomes and repeated cross sections. The results of multilevel treatment can be proven using the same arguments. Let $\kappa$ and $C$ be strictly positive constants, $K\geq2$ be a fixed integer, and $\varepsilon_{N}$ be a sequence of positive constants approaching zero. Denote by $\parallel \cdot \parallel_{P,q}$ the $L^{q}$ norm of some probability measure $P$: $\parallel f \parallel_{P,q} \coloneqq \left(\int \mid f\left(w\right) \mid^{q}dP\left(w\right)\right)^{1/q}$ and $\parallel f \parallel_{P,\infty} \coloneqq \sup_{w} \mid f\left(w\right) \mid $.

Let $P$ be the probability law for $\left(Y\left(0\right),Y\left(1\right),D,X\right)$. Let $D=g_{0}\left(X\right)+U$ and $Y\left(1\right)-Y\left(0\right)=\ell_{10}\left(X\right)+V_{1}$ with $E_{P}\left[U\mid X\right]=0$ and $E_{P}\left[V_{1}\mid X, D=0\right]=0$. Define $G_{1p0}\coloneqq E_{P}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]$ and $\Sigma_{10}\coloneqq E_{P}\left[\left(\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)+G_{1p0}\left(D-p_{0}\right)\right)^{2}\right]$. Suppose the following conditions hold: (a) $Pr\left(\kappa\leq g_{0}\left(X\right)\leq1-\kappa\right)=1$; (b) $\parallel UV_{1}\parallel_{P,4}\leq C$; (c) $E\left[U^{2}\mid X\right]\leq C$; (d) $E\left[V_{1}^{2}\mid X\right]\leq C$; (e) $\Sigma_{10}>0$; and (f) given the auxiliary sample $I_{k}^{c}$, the estimator $\hat{\eta}_{1k}=\left(\hat{g}_{k},\hat{\ell}_{1k}\right)$ obeys the following conditions. With probability $1-o\left(1\right)$, $\parallel\hat{\eta}_{1k}-\eta_{10}\parallel_{P,2}\leq\varepsilon_{N}$, $\parallel\hat{g}_{k}-1/2\parallel_{P,\infty}\leq1/2-\kappa$, and $\parallel\hat{g}_{k}-g_{0}\parallel_{P,2}^{2}+\parallel\hat{g}_{k}-g_{0}\parallel_{P,2}\times\parallel\hat{\ell}_{1k}-\ell_{10}\parallel_{P,2}\leq\left(\varepsilon_{N}\right)^{2}$.

Let $P$ be the probability law for $\left(Y,T,D,X\right)$. Let $D=g_{0}\left(X\right)+U$ and $\left(T-\lambda_{0}\right)Y=\ell_{20}\left(X\right)+V_{2}$ with $E_{p}\left[U\mid X\right]=0$ and $E_{p}\left[V_{2}\mid X, D=0\right]=0$. Define $G_{2p0}\coloneqq E_{P}\left[\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\right]$, $G_{2\lambda0}\coloneqq E_{P}\left[\partial_{\lambda}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\right]$, and $\Sigma_{20}\coloneqq E_{P}\left[\left(\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)+G_{2p0}\left(D-p_{0}\right)+G_{2\lambda0}\left(T-\lambda_{0}\right)\right)^{2}\right]$. Suppose the following conditions hold: (a) $Pr\left(\kappa\leq g_{0}\left(X\right)\leq1-\kappa\right)=1$; (b) $\parallel UV_{2}\parallel_{P,4}\leq C$; (c) $E\left[U^{2}\mid X\right]\leq C$; (d) $E\left[V_{2}^{2}\mid X\right]\leq C$; (e) $E_{P}\left[Y^{2}\mid X\right]\leq C$; (f) $\mid E_{P}\left[YU\right]\mid\leq C$; (g) $\Sigma_{20}>0$; and (h) given the auxiliary sample $I_{k}^{c}$, the estimators $\hat{\eta}_{2k}=\left(\hat{g}_{k},\hat{\ell}_{2k}\right)$ obeys the following conditions. With probability $1-o\left(1\right)$, $\parallel\hat{\eta}_{2k}-\eta_{20}\parallel_{P,2}\leq\varepsilon_{N}$, $\parallel\hat{g}_{k}-1/2\parallel_{P,\infty}\leq1/2-\kappa$, and $\parallel\hat{g}_{k}-g_{0}\parallel_{P,2}^{2}+\parallel\hat{g}_{k}-g_{0}\parallel_{P,2}\times\parallel\hat{\ell}_{2k}-\ell_{20}\parallel_{P,2}\leq\left(\varepsilon_{N}\right)^{2}$.

For repeated outcomes, suppose Assumptions (2.1), (2.2) and (3.1) hold. For repeated cross sections, suppose Assumptions (2.1)-(2.3) and (3.2) hold. If $\varepsilon_{N}=o\left(N^{-1/4}\right)$, then the new ATT estimator $\tilde{\theta}$ satisfies \[ \sqrt{N}\left(\tilde{\theta}-\theta_{0}\right)\rightarrow N\left(0,\Sigma\right) \] with $\Sigma=\Sigma_{10}$ for repeated outcomes and $\Sigma=\Sigma_{20}$ for repeated cross sections.

Construct the estimators of the asymptotic variances as \[ \hat{\Sigma}_{1}=\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\left(\psi_{1}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\eta}_{1k}\right)+\hat{G}_{1p}\left(D-\hat{p}_{k}\right)\right)^{2}\right]\tag{repeated outcomes} \] \[ \hat{\Sigma}_{2}=\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\left(\psi_{2}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\lambda}_{k},\hat{\eta}_{2k}\right)+\hat{G}_{2p}\left(D-\hat{p}_{k}\right)+\hat{G}_{2\lambda}\left(T-\hat{\lambda}_{k}\right)\right)^{2}\right]\tag{repeated cross sections} \] where $\mathbb{E}_{n,k}\left[f\left(W\right)\right]=n^{-1}\sum_{i\in I_{k}}f\left(W_{i}\right)$, $\hat{G}_{1p}=\hat{G}_{2p}=-\tilde{\theta}/\hat{p}_{k}$, and $\hat{G}_{2\lambda}$ is a consistent estimator of $G_{2\lambda0}$. If the assumptions of Theorem 1 hold, then $\hat{\Sigma}_{1}=\Sigma_{10}+o_{P}\left(1\right)$ and $\hat{\Sigma}_{2}=\Sigma_{20}+o_{P}\left(1\right).$

The interpretation of Theorem 1 and 2 is that the new DID estimator $\tilde{\theta}$ can achieve $\sqrt{N}$-consistency and asymptotic normality provided that the first-stage estimators of the infinite dimensional nuisance parameters converge at a rate faster than $N^{-1/4}$. This rate of convergence can be achieved by many ML methods. In particular, \citet*{van2008high} and \citet*{belloni2012sparse} provided detail conditions for Logit Lasso and the modified Lasso estimators to satisfy this rate of convergence. It is also worth noting that even when the first-stage estimators do not converge as fast as $N^{-1/4}$, the new estimator $\tilde{\theta}$ still has smaller bias than the original estimator because the Neyman orthogonality removes the first-order bias of the first-stage estimators.

The Small Bias Property

Consider the conventional semiparametric DID estimators with a limited number of control variables studied in \citet*{abadie2005semiparametric}. Let $\widehat{g}_{h}$ be the kernel estimator of $g_{0}$ with bandwidth $h\rightarrow0$ in (2.1) and (2.2). Under the standard assumptions of kernel estimation (Assumption (3.3) below), one can show that the pointwise bias of $\hat{g}_{h}$ is of order $O\left(h^{m}\right)$, where $m$ can be interpreted as the minimum number of derivatives of $g_{0}$; and the pointwise standard deviation is $sd\left(\hat{g}_{h}\left(x\right)\right)=O(\left(Nh^{d+2s}\right)^{-1/2})$. By Theorem 8.11 of \citet*{newey1994large}, one can show that the $\sqrt{N}$-consistency of the plug-in estimators based on (2.1) and (2.2) requires $\sqrt{N}h^{m}\rightarrow 0$. That is, the pointwise bias of the kernel estimator has to converge to zero faster than $N^{-1/2}$. Since the pointwise standard deviation converges to zero slower than $N^{-1/2}$, undersmoothing is required. In this case, standard data-driven bandwidth selection methods which do not undersmooth, such as cross-validation, are invalid.

To avoid undersmoothing, by the analysis of SBP in \citet*{newey1998undersmoothing, newey2004twicing}, the estimator of the parameter of interest needs to have smaller bias than the pointwise bias of the first-stage nonparametric estimators. That is, the SBP requires that the bias of the estimator of $\theta_{0}$ converges to zero faster than $h^{m}$.

In the following, I will show that the new DID estimator $\tilde{\theta}$ has the SBP. Let $\left(\hat{g}_{kh},\hat{\ell}_{1kh},\hat{\ell}_{2kh}\right)$ be the kernel estimators of $\left(g_{0},\ell_{10},\ell_{20}\right)$ using auxiliary sample $I_{k}^{c}$. I assume here that they have the same bandwidth $h$ and kernel $K\left(u\right)$ for convenience.

For repeated outcomes, suppose Assumptions (2.1), (2.2), (3.1), and (3.3) hold. For repeated cross sections, suppose Assumptions (2.1)-(2.3), (3.2), and (3.3) hold. Suppose that $\inf_{x\in\mathcal{X}}f_{0}\left(x\right)\neq 0$, $h=h\left(N\right)$ with $\log N/\left(\sqrt{N}h^{d+2s}\right)\rightarrow 0$. If $\sqrt{N}h^{2m}\rightarrow0$, then \[ \sqrt{N}\left(\tilde{\theta}-\theta_{0}\right)\rightarrow N\left(0,\Sigma\right) \] with $\Sigma=\Sigma_{10}$ for repeated outcomes and $\Sigma=\Sigma_{20}$ for repeated cross sections.

The interpretation of Theorem 3 is that the new estimator $\tilde{\theta}$ only requires $\sqrt{N}h^{2m}\rightarrow0$ to achieve $\sqrt{N}$-consistency, while the conventional semiparametric DID estimators require $\sqrt{N}h^{m}\rightarrow0$ under the same assumptions. With the Neyman orthogonality, the bias of $\tilde{\theta}$ is only of the second-order of the pointwise bias of the first-stage kernel estimators. The bias of $\tilde{\theta}$ is $h^{2m}$ instead of $h^{m}$. Hence, $\tilde{\theta}$ satisfies the SBP. In particular, the bandwidth $h$ such that $\log N/\left(\sqrt{N}h^{d+2s}\right)\rightarrow 0$ and $\sqrt{N}h^{2m}\rightarrow0$ exists only if $2m>d+2s$. Under this condition, the optimal bandwidth selected by minimizing mean-square errors (CV), $h=N^{-1/\left(d+2s+2m\right)}$, satisfies the conditions for $\sqrt{N}$-consistency.

Construct the estimators of the asymptotic variances as \[ \hat{\Sigma}_{1}=\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\left(\psi_{1}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\eta}_{1kh}\right)+\hat{G}_{1p}\left(D-\hat{p}_{k}\right)\right)^{2}\right] \tag{repeated outcomes} \] \[ \hat{\Sigma}_{2}=\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\left(\psi_{2}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\lambda}_{k},\hat{\eta}_{2kh}\right)+\hat{G}_{2p}\left(D-\hat{p}_{k}\right)+\hat{G}_{2\lambda}\left(T-\hat{\lambda}_{k}\right)\right)^{2}\right] \tag{repeated cross sections} \] where $\hat{G}_{1p}=\hat{G}_{2p}=-\tilde{\theta}/\hat{p}_{k}$ and $\hat{G}_{2\lambda}$ is a consistent estimator of $G_{2\lambda0}$. If the assumptions of Theorem 3 hold, then $\hat{\Sigma}_{1}=\Sigma_{10}+o_{P}\left(1\right)$ and $\hat{\Sigma}_{2}=\Sigma_{20}+o_{P}\left(1\right).$

Simulation

In this section, I present Monte Carlo simulation results of the conventional semiparametric DID estimators and the new DID estimator $\tilde{\theta}$ in three different data structures: repeated outcomes, repeated cross sections, and multilevel treatment. I use both ML methods and kernel estimators in the first-stage estimation. For ML estimation, I generate high-dimensional (HD) data and estimate the propensity score by Logit Lasso (Multi-Logit Lasso for multilevel treatment). To choose the penalty parameter for Logit Lasso (Multi-Logit Lasso), I use $K$-fold CV (as recommended by \citet*{van2008high}) with $K=10$. Alternatively, one could use a method developed in \citet*{belloni2018uniformly}. The other infinite-dimensional nuisance parameters are estimated by random forests with 500 regression trees. For kernel estimation, all the infinite-dimensional nuisance parameters are estimated using the standard Gaussian kernel.

Figure 3-20 in appendix show the simulation results. I find that the conventional semiparametric DID estimators are biased when using ML methods, while the new DID estimator $\tilde{\theta}$ can correct the bias. For kernel estimation, the conventional DID estimator with bandwidth selected by CV is biased, while the new DID estimator $\tilde{\theta}$ is centered at the true value. The data generating processes are presented in the following.

Repeated Outcomes

ML Estimation

Let $N\in \left\{200,500\right\}$ be the sample size and $p\in \left\{100,300\right\}$ the dimension of control variables, $X_{i}\sim N\left(0,I_{p\times p}\right)$. Also, let $\gamma_{0}=\left(1,1/2,1/3,1/4,1/5,0,...,0\right)\in\mathbb{R}^{p}$ and $D_{i}$ is generated by the propensity score \[ P\left(D=1\mid X\right)=\frac{1}{1+\exp\left(-X'\gamma_{0}\right)}\text{(Logistic)}. \] At $t=0$, the potential outcome is generated \[ Y_{i}^{0}\left(0\right)=X_{i}'\beta_{0}+\varepsilon_{1}, \] and at $t=1$, \[ Y_{i}^{0}\left(1\right)=Y_{i}^{0}\left(0\right)+1+\varepsilon_{2}, \] \[ Y_{i}^{1}\left(1\right)=\theta_{0}+Y_{i}^{0}\left(1\right)+\varepsilon_{3}, \] where $\beta_{0}=\gamma_{0}+0.5$ and $\theta_{0}=3$, and all error terms follow $N\left(0,0.1\right)$. Researchers observe $\left\{ Y_{i}\left(0\right),Y_{i}\left(1\right),D_{i},X_{i}\right\}$ for $i=1,...,N$, where $Y_{i}\left(0\right)=Y_{i}^{0}\left(0\right)$ and $Y_{i}\left(1\right)=Y_{i}^{0}\left(1\right)\left(1-D_{i}\right)+Y_{i}^{1}\left(1\right)D_{i}$. Figure 3-6 present the results.

Kernel Estimation

Let $N\in \left\{200,500\right\}$ be the sample size, $D_{i}\sim Bernoulli(0.5)$, and $X_{i}\mid D_{i}\sim N\left(D_{i},1\right)$. At $t=0$, the potential outcome is generated \[ Y_{i}^{0}\left(0\right)=\varepsilon_{1}, \] and at $t=1$, \[ Y_{i}^{0}\left(1\right)=Y_{i}^{0}\left(0\right)+X_{i}+\varepsilon_{2}, \] \[ Y_{i}^{1}\left(1\right)=\theta_{0}+Y_{i}^{0}\left(1\right)+\varepsilon_{3}, \] where $\theta_{0}=3$ and all error terms follow $N\left(0,0.1\right)$. Researchers observe $\left\{ Y_{i}\left(0\right),Y_{i}\left(1\right),D_{i},X_{i}\right\}$ for $i=1,...,N$, where $Y_{i}\left(0\right)=Y_{i}^{0}\left(0\right)$ and $Y_{i}\left(1\right)=Y_{i}^{0}\left(1\right)\left(1-D_{i}\right)+Y_{i}^{1}\left(1\right)D_{i}$. Figure 7-8 present the results.

Repeated Cross Sections

ML Estimation

Let $N\in \left\{200,500\right\}$ be the sample size and $p\in \left\{100,300\right\}$ the dimension of control variables, $X_{i}\sim N\left(0.3,I_{p\times p}\right)$. Also, let $\gamma_{0}=\left(1,1/2,1/3,1/4,1/5,0,...,0\right)\in\mathbb{R}^{p}$ and $D$ is generated by the propensity score \[ P\left(D=1\mid X\right)=\frac{1}{1+\exp\left(-X'\gamma_{0}\right)}\text{(Logistic)}. \] At $t=0$, the potential outcome is generated \[ Y_{i}^{0}\left(0\right)=1+\varepsilon_{1}, \] and at $t=1$, \[ Y_{i}^{0}\left(1\right)=Y_{i}^{0}\left(0\right)+1+\varepsilon_{2}, \] \[ Y_{i}^{1}\left(1\right)=\theta_{0}+Y_{i}^{0}\left(1\right)+\varepsilon_{3}, \] where $\beta_{0}=\gamma_{0}+0.5$ and $\theta_{0}=3$, and all error terms follow $N\left(0,0.1\right)$. Define $Y_{i}\left(0\right)=Y_{i}^{0}\left(0\right)$ and $Y_{i}\left(1\right)=Y_{i}^{0}\left(1\right)\left(1-D_{i}\right)+Y_{i}^{1}\left(1\right)D_{i}$. Let $T_{i}$ follow a Bernoulli distribution with parameter $0.5$. Researchers observe $\left\{ Y_{i},T_{i},D_{i},X_{i}\right\}$ for $i=1,...,N$, where $Y_{i}=Y_{i}\left(0\right)+T_{i}\left(Y_{i}\left(1\right)-Y_{i}\left(0\right)\right)$. Figure 9-12 present the results.

Kernel Estimation

Let $N\in \left\{200,500\right\}$ be the sample size, $D_{i}\sim Bernoulli(0.5)$, and $X_{i}\mid D_{i}\sim N\left(D_{i},1\right)$. At $t=0$, the potential outcome is generated \[ Y_{i}^{0}\left(0\right)=\varepsilon_{1}, \] and at $t=1$, \[ Y_{i}^{0}\left(1\right)=Y_{i}^{0}\left(0\right)+X_{i}+\varepsilon_{2}, \] \[ Y_{i}^{1}\left(1\right)=\theta_{0}+Y_{i}^{0}\left(1\right)+\varepsilon_{3}, \] where $\theta_{0}=3$ and all error terms follow $N\left(0,0.1\right)$. Let $Y_{i}\left(0\right)=Y_{i}^{0}\left(0\right)$ and $Y_{i}\left(1\right)=Y_{i}^{0}\left(1\right)\left(1-D_{i}\right)+Y_{i}^{1}\left(1\right)D_{i}$. Let $T_{i}\sim Bernoulli(0.5)$. Researchers observe $\left\{ Y_{i},T_{i},D_{i},X_{i}\right\}$ for $i=1,...,N$, where $Y_{i}=Y_{i}\left(0\right)+T_{i}\left(Y_{i}\left(1\right)-Y_{i}\left(0\right)\right)$. Figure 13-14 present the results.

Multilevel Treatment

ML Estimation

Suppose there are two levels of treatment so that $W\in\left\{ 0,1,2\right\}$. Let $N\in \left\{200,500\right\}$ be the sample size and $p\in \left\{100,300\right\}$ the dimension of control variables, $X_{i}\sim N\left(0,I_{p\times p}\right)$. Also, let $\gamma_{0}=\left(1,1/2,1/3,1/4,1/5,0,...,0\right)\in\mathbb{R}^{p}$ and $$\left(P\left(W=0\right),P\left(W=1\right),P\left(W=2\right)\right)=\left(0.3,0.3,0.4\right)$$ At $t=0$, the potential outcome is generated \[ Y_{i}^{0}\left(0\right)=X'\beta_{0}+\varepsilon_{1}, \] and at $t=1$, \[ Y_{i}^{0}\left(1\right)=Y_{i}^{0}\left(0\right)+1+\varepsilon_{2}, \] \[ Y_{i}^{1}\left(1\right)=\theta_{10}+Y_{i}^{0}\left(1\right)+\varepsilon_{3}, \] \[ Y_{i}^{2}\left(1\right)=\theta_{20}+Y_{i}^{0}\left(1\right)+\varepsilon_{4}, \] where $\beta_{0}=\gamma_{0}+0.5$ and $\theta_{10}=3$ and $\theta_{20}=6$, and all error terms follow $N\left(0,0.1\right)$. Researchers observe $\left\{ Y_{i}\left(0\right),Y_{i}\left(1\right),W_{i},X_{i}\right\}$ for $i=1,...,N$, where $Y_{i}\left(0\right)=Y_{i}^{0}\left(0\right)$ and $Y_{i}\left(1\right)=Y_{i}^{0}\left(1\right)I\left(W_{i}=0\right)+Y_{i}^{1}\left(1\right)I\left(W_{i}=1\right)+Y_{i}^{2}\left(1\right)I\left(W_{i}=2\right)$. I focus on the estimation of the second level ATT $\theta_{20}$. Figure 15-18 present the results

Kernel Estimation

Suppose there are two levels of treatment so that $W\in\left\{ 0,1,2\right\}$. Let $N$ be the sample size, $X_{i}\mid W_{i}\sim N\left(W_{i},1\right)$, and

\[ P\left(W_{i}=0\right)=P\left(W_{i}=1\right)=P\left(W_{i}=2\right)=\frac{1}{3}. \] At $t=0$, the potential outcome is generated \[ Y_{i}^{0}\left(0\right)=\varepsilon_{1}, \] and at $t=1$, \[ Y_{i}^{0}\left(1\right)=Y_{i}^{0}\left(0\right)+X_{i}+\varepsilon_{2}, \] \[ Y_{i}^{1}\left(1\right)=\theta_{10}+Y_{i}^{0}\left(1\right)+\varepsilon_{3}, \] \[ Y_{i}^{2}\left(1\right)=\theta_{20}+Y_{i}^{0}\left(1\right)+\varepsilon_{4}, \] where $\theta_{10}=3$, $\theta_{20}=6$, and all error terms follow $N\left(0,0.1\right)$. Let $Y_{i}\left(0\right)=Y_{i}^{0}\left(0\right)$ and $Y_{i}\left(1\right)=Y_{i}^{0}\left(1\right)I\left(W_{i}=0\right)+Y_{i}^{1}\left(1\right)I\left(W_{i}=1\right)+Y_{i}^{2}\left(1\right)I\left(W_{i}=2\right)$. Researchers observe $\{ Y_{i}\left(0\right),Y_{i}\left(1\right),W_{i},X_{i}\}$ for $i=1,...,N$. I focus on the estimation of the second level ATT $\theta_{20}$. Figure 19-20 present the results.

Empirical Example

In this example, I analyze the effect of tariffs reduction on corruption behaviors using the bribe payment data collected by \citet*{sequeira2016corruption} between South Africa and Mozambique. There have been theoretical and empirical debates on whether higher tariff rates increase incentives for corruption to occur \citep*{clotfelter1983tax,sequeira2014corruption} or lower tariffs encourage agents to pay higher bribes through an income effect \citep*{feinstein1991econometric,slemrod2002tax}. The former argues that an increase in the tariff rate makes it more profitable to evade taxes on the margin. The latter argues that an increased tariff rate makes the tax payer less wealthy and this, under the decreasing risk aversion of being penalized, tend to reduce evasion \citep*{allingham1972income}.

\citet*{sequeira2016corruption} collected primary data on the bribed payments between the ports in Mozambique and South Africa from 2007 to 2013. The treatment is the large reduction in the average nominal tariff rate (of 5 percent) occurring in 2008. Since not all products were on the tariff reduction list, a credible control group of products is available. This allows for a DID estimation.

This natural experiment between South Africa and Mozambique was previously studied by \citet*{sequeira2016corruption} by pooling the cross section data between 2007 and 2013, with sample size $N=1084$, and estimating the effect of treatment using the traditional linear DID. Here I focus on the specification of one of the main results (Table 9 of \citet*{sequeira2016corruption}):

where $y_{it}$ is the natural log of the amount of bribe paid for shipment $i$ in period $t$, conditional on paying a bribe. $TariffChangeCategory\in\left\{ 0,1\right\}$ denotes the treatment status of commodities, $POST\in\left\{ 0,1\right\}$ is an indicator for the years following 2008, and $BaselineTariff$ is the tariff rate before the tariff reduction. The specification also includes a vector of characteristics $\Gamma_{i}$, and time and individual fixed effects $p_{i}$, $w_{t}$, and $\delta_{i}$. The parameter $\gamma_{1}$ is the parameter of interest in the traditional linear DID estimation. \citet*{sequeira2016corruption} found that the amount of bribes paid dropped after the tariff reduction ($\hat{\gamma}_{1}=-2.928^{**}$).

I use the same data set but instead of using the traditional linear DID estimation, I estimate the ATT by \citet*{abadie2005semiparametric}'s DID estimator and my proposed DID estimator $\tilde{\theta}$. Since the data is repeated cross sections, I construct the estimators based on (2.2) and (3.2), respectively. The estimators with the first-stage kernel estimation contain one individual characteristic (the natural log of shipment value per ton), which is a significant characteristic in Table 9 of \citet*{sequeira2016corruption}. The estimators with the first-stage Lasso estimation contain a list of the significant characteristics in Table 9 of \citet*{sequeira2016corruption}, which includes product, shipment, firm-level characteristics, and their interaction terms. Table 1 below shows the results. I find that all these estimators consistently suggest that a decrease in tariff rate will lead to less bribes payment, but the effect of treatment may be actually substantially larger than previously reported by \citet*{sequeira2016corruption}.

Conclusion

In this article, I have introduced three new DID estimators based on the newly-derived Neyman-orthogonal scores. These new scores do not require any additional conditions other than the original conditions made in \citet*{abadie2005semiparametric}. The new DID estimators will be particularly appropriate when researchers would like to use ML methods in the first-stage nonparametric estimation. When using kernel estimators in the first-stage estimation , the new DID estimators do not require undersmoothing to achieve $\sqrt{N}$-consistency. Hence, researchers can use standard data-driven methods, such as CV, to select bandwidths.

Similarly, I use the cross-fitting algorithm \citep*{chernozhukov2018double}.

The estimator $\hat{g}_{wk}$ and $\hat{g}_{zk}$ can be constructed by Multi-Logit Lasso.

The following is suggested by \citet*{belloni2012sparse}. Let $y_{i}$ denote $Y_{i}\left(1\right)-Y_{i}\left(0\right)$ or $\left(T_{i}-\hat{\lambda}_{k}\right)$, $\lambda_{k}$ denote $\lambda_{1k}$ or $\lambda_{2k}$, and $\hat{\Upsilon}_{k}$ denote $\hat{\Upsilon}_{1k}$ or $\hat{\Upsilon}_{2k}$. For $k\in\left[K\right]$, the loading $\hat{\Upsilon}_{k}$ is a diagonal matrix with entries $\hat{\gamma}_{kj}$, $j=1,...,p$, constructed by the following steps:

\[ \text{Initial }\hat{\gamma}_{kj}=\sqrt{\frac{1}{M_k}\sum_{i\in I_{kz}^{c}}q_{ij}^{2}\left(y_{i}-\bar{y}_{k}\right)^{2}},\lambda_{k}=2c\sqrt{M_{k}}\Phi^{-1}\left(1-\gamma/2p\right), \] \[ \text{Refined }\hat{\gamma}_{kj}=\sqrt{\frac{1}{M_{k}}\sum_{i\in I_{kz}^{c}}q_{ij}^{2}\hat{\varepsilon}_{i}^{2}},\lambda_{k}=2c\sqrt{M_{k}}\Phi^{-1}\left(1-\gamma/2p\right), \] where $\bar{y}_{k}=M^{-1}\sum_{i\in I_{k}^{c}}y_{i}$, $c>1$ and $\gamma\rightarrow0$. The empirical residual $\hat{\varepsilon}_{i}$ is calculated by the modified Lasso estimator $\beta_{k}^{*}$ in the previous step: $\hat{\varepsilon}_{i}=y_{i}-q_{i}'\beta_{k}^{*}$. Repeat the second step $B>0$ times to obtain the final loading.

The Gateaux derivative of (3.1) in the direction $\eta_{1}-\eta_{10}=\left(g-g_{0},\ell_{1}-\ell_{10}\right)$ is

where the second inequality follows from the law of iterated expectations, the third from the definition of $\ell_{10}\left(X\right)$ and $E_{P}\left[D-g_{0}\left(X\right)\mid X\right]=0$.

Define $\partial_{\eta_{2}}E_{P}\left[\psi_{20}\right]\left(\eta_{2}-\eta_{20}\right)\coloneqq \partial_{\eta_{2}}E_{P}\left[\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\right]\left(\eta_{2}-\eta_{20}\right)$. Similar to the proof of repeated outcomes, the Gateaux derivative of (3.2) in the direction $\eta_{2}-\eta_{20}=\left(g-g_{0},\ell_{2}-\ell_{20}\right)$ is

where $p_{0}'\coloneqq p_{0}\lambda_{0}\left(1-\lambda_{0}\right)$.

Let $\Delta_{w}=g_{w}-g_{w0}$, $\Delta_{z}=g_{z}-g_{z0}$, and $\Delta_{\ell3}=\ell_{3}-\ell_{30}$. The Gateaux derivative of (3.3) in the direction $\eta_{w}-\eta_{w0}=\left(g_{w}-g_{w0},g_{z}-g_{z0},\ell_{3}-\ell_{30}\right)$ is

by the law of iterated expectation on each terms.

The proofs of Theorem 1 and 2 follow the general framework proposed in \citet*{chernozhukov2018double}.

The proof proceeds in five steps. In Step 1, I show the main result using the auxiliary results (A.1)-(A.4). In Step 2-5, I prove the auxiliary results. \[ \sup_{\eta_{1}\in\mathcal{T}_{N}}\left(E\left[\parallel\psi_{1}\left(W,\theta_{0},p_{0},\eta_{1}\right)-\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.1} \]

\[ \sup_{r\in\left(0,1\right),\eta_{1}\in\mathcal{T}_{N}}\parallel\partial_{r}^{2}E\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}+r\left(\eta_{1}-\eta_{10}\right)\right)\right]\parallel\leq\left(\varepsilon_{N}\right)^{2},\tag{A.2} \]

\[ \sup_{\eta_{1}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{1}\right)-\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.3} \] \[ \sup_{p\in\mathcal{P}_{N},\eta_{1}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},p,\eta_{1}\right)-\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.4} \] where $\mathcal{T}_{N}$ is the set of all $\eta_{1}=\left(g,\ell_{1}\right)$ consisting of $P$-square-integrable functions $g$ and $\ell_{1}$ such that \[ \parallel\eta_{1}-\eta_{10}\parallel_{P,2}\leq\varepsilon_{N}, \] \[ \parallel g-1/2\parallel_{P,\infty}\leq1/2-\kappa, \] \[ \parallel g-g_{0}\parallel_{P,2}^{2}+\parallel g-g_{0}\parallel_{P,2}\times\parallel\ell_{1}-\ell_{10}\parallel_{P,2}\leq\left(\varepsilon_{N}\right)^{2}, \] and $\mathcal{P}_{N}$ is the set of all $p>0$ such that $\mid p-p_{0}\mid\leq N^{-1/2}$. By the regularity condition (3.1) and $\mid \hat{p}_{k}-p_{0}\mid=O_{P}\left(N^{-1/2}\right)$, $\hat{\eta}_{1k}\in \mathcal{T}_{N}$ and $\hat{p}_{k}\in \mathcal{P}_{N}$ with probability $1-o\left(1\right)$.

Step 1. Observe that we have the decomposition

where $\bar{p}_{k}\in\left(\hat{p}_{k},p_{0}\right)$. For term (a), by the triangle inequality, we have \[ \mid\mathbb{E}_{n,k}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\right]-E_{p}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\mid\leq J_{1,k}+J_{2,k}, \] where \[ J_{1,k}=\mid\mathbb{E}_{n,k}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\right]-\mathbb{E}_{n,k}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\mid, \] \[ J_{2,k}=\mid\mathbb{E}_{n,k}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]-E_{p}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\mid. \] To bound $J_{2,k}$, we have

where the last inequality follows from the regularity condition (3.1). By Chebyshev's inequality, $J_{2,k}=O_{P}\left(n^{-1/2}\right)=o_{P}\left(1\right)$. Next, we bound $J_{1,k}$. Conditional on the auxiliary sample $I_{k}^{c}$, $\hat{\eta}_{1k}$ can be treated as fixed. Under the event that $\hat{\eta}_{1k}\in\mathcal{T}_{N}$, we have

by (A.3). By Lemma A.1, $J_{1,k}=O_{P}\left(\varepsilon_{N}\right)=o_{P}\left(1\right)$. Together, we have \[ \mathbb{E}_{n,k}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\right]\stackrel{p}{\rightarrow}E_{p}\left[\partial_{p}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]=G_{1p0}. \] For term (b), by the triangle inequality, we have \[ \mid\mathbb{E}_{n,k}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},\bar{p}_{k},\hat{\eta}_{1k}\right)\right]-E_{p}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\mid\leq J_{3,k}+J_{4,k}, \] where \[ J_{3,k}=\mid\mathbb{E}_{n,k}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},\bar{p}_{k},\hat{\eta}_{1k}\right)\right]-\mathbb{E}_{n,k}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\mid, \] \[ J_{4,k}=\mid\mathbb{E}_{n,k}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]-E_{p}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\mid. \] To bound $J_{4,k}$, we have

where the last inequality follows from the regularity conditions. By Chebyshev's inequality, $J_{4,k}=O_{P}\left(n^{-1/2}\right)=o_{P}\left(1\right)$. Conditional on $I_{k}^{c}$, both $\bar{p}_{k}$ and $\hat{\eta}_{1k}$ can be treated as fixed. Under the event that $\hat{p}_{k}\in\mathcal{P}_{N}$ (thus $\bar{p}_{k}\in\mathcal{P}_{N}$) and $\hat{\eta}_{1k}\in\mathcal{T}_{N}$ , we have

by (A.4). By Lemma A.1, $J_{3,k}=O_{P}\left(\varepsilon_{N}\right)=o_{P}\left(1\right)$. Hence, $\mathbb{E}_{n,k}\left[\partial_{p}^{2}\psi_{1}\left(W,\theta_{0},\bar{p}_{k},\hat{\eta}_{1k}\right)\right]=O_{P}\left(1\right)$.

Combine the above results with $\hat{p}_{k}-p_{0}=\mathbb{E}_{n,k}\left[D-p_{0}\right]$ and $\left(\hat{p}_{k}-p_{0}\right)^{2}=O_{P}\left(N^{-1}\right)$, the decomposition of $\tilde{\theta}$ becomes

where

If we can show that $\sqrt{N}R_{N}=o_{P}\left(1\right)$, then we are done.

This part is essentially identical to Step 3 in the proof of Theorem 3.1 (DML2) in \citet*{chernozhukov2018double}. I reproduce it here for reader's convenience. Since $K$ is a fixed integer, which is independent of $N$, it suffices to show that for any $k\in\left[K\right]$, \[ \mathbb{E}_{n,k}\left[\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\right]-\frac{1}{n}\sum_{i\in I_{k}}\psi_{1}\left(W_{i},\theta_{0},p_{0},\eta_{10}\right)=o_{P}\left(N^{-1/2}\right). \] Define the empirical process notation: \[ \mathbb{G}_{n,k}\left[\phi\left(W\right)\right]=\frac{1}{\sqrt{n}}\sum_{i\in I_{k}}\left(\phi\left(W_{i}\right)-\int\phi\left(w\right)dP\right), \] where $\phi$ is any $P$-integrable function on $\mathcal{W}$. By the triangle inequality, we have \[ \parallel\mathbb{E}_{n,k}\left[\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\right]-\frac{1}{n}\sum_{i\in I_{k}}\psi_{1}\left(W_{i},\theta_{0},p_{0},\eta_{10}\right)\parallel\leq\frac{I_{1,k}+I_{2,k}}{\sqrt{n}}, \] where \[ I_{1,k}\coloneqq\parallel\mathbb{G}_{n,k}\left[\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\right]-\mathbb{G}_{n,k}\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\parallel, \] \[ I_{2,k}\coloneqq\sqrt{n}\parallel E_{P}\left[\psi_{1}\left(W,\theta_{0},p_{0},\hat{\eta}_{1k}\right)\mid\left(W_{i}\right)_{i\in I_{k}^{c}}\right]-E_{P}\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]\parallel. \] To bound $I_{1,k}$, note that conditional on $\left(W_{i}\right)_{i\in I_{k}^{c}}$ the estimator $\hat{\eta}_{1k}$ is nonstochastic. Under the event that $\hat{\eta}_{1k}\in\mathcal{T}_{N}$, we have

by (A.1). Hence, $I_{1,k}=O_{P}\left(\varepsilon_{N}\right)$ by Lemma A.1. To bound $I_{2,k}$, define the following function \[ f_{k}\left(r\right)=E_{P}\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}+r\left(\hat{\eta}_{1k}-\eta_{10}\right)\right)\mid\left(W_{i}\right)_{i\in I_{k}^{c}}\right]-E\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right],r\in[0,1). \] By Taylor series expansion, we have \[ f_{k}\left(1\right)=f_{k}\left(0\right)+f'_{k}\left(0\right)+f''_{k}\left(\tilde{r}\right)/2,\text{for some }\tilde{r}\in\left(0,1\right). \] Note that $f_{k}\left(0\right)=0$ since $E\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\mid\left(W_{i}\right)_{i\in I_{k}^{c}}\right]=E\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right]$. Further, on the event $\hat{\eta}_{1k}\in\mathcal{T}_{N}$, \[ \parallel f_{k}'\left(0\right)\parallel=\parallel\partial_{\eta_{1}}E_{P}\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\left[\hat{\eta}_{1k}-\eta_{10}\right]\parallel=0 \] by the orthogonality of $\psi_{1}$. Also, on the event $\hat{\eta}_{1k}\in\mathcal{T}_{N}$, \[ \parallel f_{k}''\left(\tilde{r}\right)\parallel\leq\sup_{r\in\left(0,1\right)}\parallel f_{k}''\left(r\right)\parallel\leq\left(\varepsilon_{N}\right)^{2} \] by (A.2). Thus, \[ I_{2,k}=\sqrt{n}\parallel f_{k}\left(1\right)\parallel=O_{P}\left(\sqrt{n}\left(\varepsilon_{N}\right)^{2}\right). \] Together with the result on $I_{1,k}$, we have

by $n=O\left(N\right)$ and $\varepsilon_{N}=o\left(N^{-1/4}\right)$. Hence, $\sqrt{N}R_{N}=o_{P}\left(1\right)$.

Step 2. In this step, I present the proof of (A.1). We have the following decomposition:

Thus, we have

Given $\kappa\leq g_{0}\left(X\right)\leq1-\kappa$ and $\kappa\leq g\left(X\right)\leq1-\kappa$,

By $\kappa\leq g_{0}\left(X\right)\leq1-\kappa$ and $\kappa\leq g\left(X\right)\leq1-\kappa$ again, we can obtain

Thus, by $E_{P}\left[U^{2}\mid X\right]\leq C$ and $E_{P}\left[V_{1}^{2}\mid X\right]\leq C$,

Step 3. In this step, I present the proof of (A.2). Define $$f\left(r\right)=E_{P}\left[\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}+r\left(\eta_{1}-\eta_{10}\right)\right)\right].$$ Then its second-order derivative is

It follows that \[ \mid\partial_{r}^{2}f\left(r\right)\mid\leq O\left(\parallel\left(g-g_{0}\right)\parallel_{P,2}^{2}+\parallel\left(g-g_{0}\right)\parallel_{P,2}\times\parallel\left(\ell_{1}-\ell_{10}\right)\parallel_{P,2}\right)\leq\left(\varepsilon_{N}\right)^{2}. \]

Step 4. Notice that

then we have

by Step 2.

Step 5. Notice that

then we have

where $\bar{p}\in\left(p,p_{0}\right)$. Thus,

The term in the second line is bounded by

by $\parallel UV_{1}\parallel_{P,2}\leq \parallel UV_{1}\parallel_{P,4}\leq C$ , $E_{P}\left[U^{2}\mid X\right]\leq C$, $E_{P}\left[V_{1}^{2}\mid X\right]\leq C$, and the conditions on the rates of convergence. Together with Step 2, we obtain

where I assume that $\varepsilon_{N}$ converges to zero no faster than $N^{-1/2}$.

In step 1, I show the main result with the following auxiliary results: \[ \sup_{\eta_{2}\in\mathcal{T}_{N}}\left(E\left[\parallel\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{2}\right)-\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.5} \]

\[ \sup_{r\in\left(0,1\right),\eta_{2}\in\mathcal{T}_{N}}\parallel\partial_{r}^{2}E\left[\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}+r\left(\eta_{2}-\eta_{20}\right)\right)\right]\parallel\leq\left(\varepsilon_{N}\right)^{2}.\tag{A.6} \] \[ \sup_{\eta_{2}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{2}\right)-\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.7} \] \[ \sup_{\eta_{2}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{\lambda}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{2}\right)-\partial_{\lambda}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.8} \] \[ \sup_{p\in\mathcal{P}_{N},\eta_{2}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{p}^{2}\psi_{2}\left(W,\theta_{0},p,\lambda_{0},\eta_{2}\right)-\partial_{p}^{2}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.9} \] \[ \sup_{p\in\mathcal{P}_{N},\lambda\in\Lambda_{N},\eta_{2}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{\lambda}^{2}\psi_{2}\left(W,\theta_{0},p,\lambda,\eta_{2}\right)-\partial_{\lambda}^{2}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.10} \] \[ \sup_{p\in\mathcal{P}_{N},\eta_{2}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\partial_{\lambda}\partial_{p}\psi_{2}\left(W,\theta_{0},p,\lambda_{0},\eta_{2}\right)-\partial_{\lambda}\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.11} \] where $\mathcal{T}_{N}$ is the set of all $\eta_{2}=\left(g,\ell_{2}\right)$ consisting of $P$-square-integrable functions $g$ and $\ell_{2}$ such that \[ \parallel\eta_{2}-\eta_{20}\parallel_{P,2}\leq\varepsilon_{N}, \] \[ \parallel g-1/2\parallel_{P,\infty}\leq1/2-\kappa, \] \[ \parallel\left(g-g_{0}\right)\parallel_{P,2}^{2}+\parallel\left(g-g_{0}\right)\parallel_{P,2}\times\parallel\left(\ell_{2}-\ell_{20}\right)\parallel_{P,2}\leq\left(\varepsilon_{N}\right)^{2}, \] $\mathcal{P}_{N}$ and $\Lambda_{N}$ are the sets consisting all $p>0$ and $\lambda>0$ such that $\mid p-p_{0}\mid\leq N^{-1/2}$ and $\mid\lambda-\lambda_{0}\mid\leq N^{-1/2}$, respectively. By the regularity condition (3.2), $\mid\hat{p}_{k}-p_{0}\mid=O_{P}\left(N^{-1/2}\right)$, and $\mid\hat{\lambda}_{k}-\lambda_{0}\mid=O_{P}\left(N^{-1/2}\right)$, we have $\hat{\eta}_{2k}\in \mathcal{T}_{N}$, $\hat{p}_{k}\in\mathcal{P}_{N}$, and $\hat{\lambda}_{k}\in\Lambda_{N}$ with probability $1-o\left(1\right)$.

In Step 2-4, I show the above auxiliary results.

Step 1. Notice that

where the term $o_{P}\left(1\right)$, by the same arguments for the term $b$ in repeated outcomes and the auxiliary results (A.9)-(A.11), contains the second-order terms \[ \sqrt{N}\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\partial_{p}^{2}\psi_{2}\left(W,\theta_{0},\bar{p}_{k},\lambda_{0},\hat{\eta}_{2k}\right)\right]\left(\hat{p}_{k}-p_{0}\right)^{2}, \] \[ \sqrt{N}\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\partial_{\lambda}^{2}\psi_{2}\left(W,\theta_{0},\hat{p}_{k},\bar{\lambda}_{k},\hat{\eta}_{2k}\right)\right]\left(\hat{\lambda}_{k}-\lambda_{0}\right)^{2}, \] \[ \sqrt{N}\frac{1}{K}\sum_{k=1}^{K}\mathbb{E}_{n,k}\left[\partial_{\lambda}\partial_{p}\psi_{2}\left(W,\theta_{0},\bar{p}_{k},\lambda_{0},\hat{\eta}_{2k}\right)\right]\left(\hat{\lambda}_{k}-\lambda_{0}\right)\left(\hat{p}_{k}-p_{0}\right), \] where $\bar{p}_{k}\in\left(\hat{p}_{k},p_{0}\right)$ and $\bar{\lambda}_{k}\in\left(\hat{\lambda}_{k},\lambda_{0}\right)$. On the other hand, by the same arguments for the term $a$ in repeated outcomes and the auxiliary results (A.7)-(A.8), we have \[ \mathbb{E}_{n,k}\left[\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\hat{\eta}_{2k}\right)\right]\stackrel{p}{\rightarrow}E_{p}\left[\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\right]=G_{2p0}, \] \[ \mathbb{E}_{n,k}\left[\partial_{\lambda}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\hat{\eta}_{2k}\right)\right]\stackrel{p}{\rightarrow}E_{p}\left[\partial_{\lambda}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\right]=G_{2\lambda0}. \] Hence, since $\hat{p}_{k}-p_{0}=\mathbb{E}_{n,k}\left[D-p_{0}\right]$ and $\hat{\lambda}_{k}-\lambda_{0}=\mathbb{E}_{n,k}\left[T-\lambda_{0}\right]$, we have

where

Using (A.5)-(A.6) and the same arguments as the step 1 in repeated outcomes, one can show that $\sqrt{N}R_{N}'=o_{P}\left(1\right)$. Hence, it remains to prove the auxiliary results (A.5)-(A.11).

Step 2. Recall that $p_{0}'= p_{0}\lambda_{0}\left(1-\lambda_{0}\right)$. For (A.5), notice that

The decomposition becomes

Given that $\kappa\leq g_{0}\left(X\right)\leq1-\kappa$, $\kappa\leq g\left(X\right)\leq1-\kappa$, we have

By $\kappa\leq g_{0}\left(X\right)\leq1-\kappa$, $\kappa\leq g\left(X\right)\leq1-\kappa$ again, we obtain

Given $E_{P}\left[U^{2}\mid X\right]\leq C$, $E_{P}\left[V_{2}^{2}\mid X\right]\leq C$, and the conditions on the rates of convergence,

For (A.6), let $f\left(r\right)=E_{P}\left[\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}+r\left(\eta_{2}-\eta_{20}\right)\right)\right]$. Then the second-order derivative is

It follows that \[ \mid\partial_{r}^{2}f\left(r\right)\mid\leq O\left(\parallel\left(g-g_{0}\right)\parallel_{P,2}^{2}+\parallel\left(g-g_{0}\right)\parallel_{P,2}\times\parallel\left(\ell_{2}-\ell_{20}\right)\parallel_{P,2}\right)\leq\left(\varepsilon_{N}\right)^{2}. \]

Step 3. For (A.7), notice that

then we have

by the proof of (A.5).

For (A.8), notice that

Define $\partial_{\lambda}\psi_{20}\coloneqq\partial_{\lambda}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)$, then

by (A.5) and $E_{P}\left[Y^{2}\mid X\right]\leq C$.

Step 4. For (A.9), notice that we have \[ \partial_{p}^{2}\psi_{2}\left(W,\theta,p,\lambda,\eta_{2}\right)=\frac{2}{p^{3}\lambda\left(1-\lambda\right)}\frac{D-g\left(X\right)}{1-g\left(X\right)}\left(\left(T-\lambda\right)Y-\ell_{2}\left(X\right)\right). \] Define $\partial_{p}^{2}\psi_{20}\coloneqq\partial_{p}^{2}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)$, then we have

where $\bar{p}\in\left(p,p_{0}\right)$. Hence, we have

By (A.5), we have $\parallel\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{2}\right)-\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)\parallel_{P,2}=O\left(\varepsilon_{N}\right)$. The term in the second line is bounded by

by $\parallel UV_{2}\parallel_{P,2}\leq \parallel UV_{2}\parallel_{P,4} \leq C$, $E_{P}\left[U^{2}\mid X\right]\leq C$, and $E_{P}\left[V_{2}^{2}\mid X\right]\leq C$. Thus, we obtain

where I assume that $\varepsilon_{N}$ converges to zero no faster than $N^{-1/2}$.

For (A.10), notice that we have

where $c_{1}$ is a constant depending on $\lambda$. Define $\partial_{\lambda}^{2}\psi_{20}\coloneqq\partial_{\lambda}^{2}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right)$, we have

where $\bar{p}\in\left(p,p_{0}\right)$ and $\bar{\lambda}\in\left(\lambda,\lambda_{0}\right)$. By the triangle inequality, we have

The norm term is the second line is bounded by

by $E_{P}\left[Y^{2}\mid X\right]\leq C$ and $D\in\left\{ 0,1\right\} $. The two high-order terms are bounded by

and

where $c_{2}$ and $c_{3}$ are constants depending on $\lambda$. Using the same arguments in (A.9), one can show that \[ \parallel\frac{D-g\left(X\right)}{1-g\left(X\right)}\left(\left(T-\lambda\right)Y-\ell_{2}\left(X\right)\right)\parallel_{P,2}\leq O\left(1\right), \] \[ \parallel\frac{D-g\left(X\right)}{1-g\left(X\right)}\left(\left(T-\bar{\lambda}\right)Y-\ell_{2}\left(X\right)\right)\parallel_{P,2}\leq O\left(1\right). \] Also, we have

by $\parallel UY\parallel_{P,2}\leq C$ and $E_{P}\left[Y^{2}\mid X\right]\leq C$.

Finally, we obtain

where I assume that $\varepsilon_{N}$ converges to zero no faster than $N^{-1/2}$.

For (A.11), notice that the derivative is

Define $\partial_{\lambda}\partial_{p}\psi_{20}\coloneqq\partial_{\lambda}\partial_{p}\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right),$ then we have

where $\bar{p}\in\left(p,p_{0}\right)$. By the triangle inequality, we obtain

Using the same arguments in (A.9) and (A.10), one can show that the high-order term is bounded by

Together with (A.8), we obtain

where I assume that $\varepsilon_{N}$ converges to zero no faster than $N^{-1/2}$.

In Step 1, I show the main result using the auxiliary results \[ \sup_{p\in\mathcal{P}_{N},\eta_{1}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\bar{\psi}_{1}\left(W,\theta_{0},p,\eta_{1}\right)-\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\parallel^{2}\right]\right)^{1/2}\leq\varepsilon_{N},\tag{A.12} \]

\[ \left(E_{P}\left[\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)^4\right]\right)^{1/4}\leq C_{1}, \tag{A.13} \] where $\mathcal{P}_{N}$ and $\mathcal{T}_{N}$ are specified in the proof of Theorem 1, $C_{1}$ is a constant, and \[ \bar{\psi}_{1}\left(W,\theta,p,\eta_{1}\right)\coloneqq\frac{1}{p}\frac{D-g\left(X\right)}{1-g\left(X\right)}\left(Y\left(1\right)-Y\left(0\right)-\ell_{1}\left(X\right)\right)-\frac{D\theta}{p}. \] In fact, we have $E_{P}\left[\left(\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\right)^{2}\right]=\Sigma_{10}$. In Step 2, I show the auxiliary results (A.12) and (A.13).

Step 1. Notice that

where the second equality follows from $\hat{G}_{1p}=-\tilde{\theta}/\hat{p}_{k}$.

Since $K$ is fixed, which is independent of $N$, it suffices to show that for each $k\in \left[k\right]$, \[ I_{k}\coloneqq\mid\mathbb{E}_{n,k}\left[\bar{\psi}_{1}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\eta}_{1k}\right)^{2}\right]-E_{P}\left[\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)^{2}\right]\mid=o_{P}\left(1\right). \] By the triangle inequality, we have \[ I_{k}\leq I_{3,k}+I_{4,k}, \] where \[ I_{3,k}\coloneqq\mid\mathbb{E}_{n,k}\left[\bar{\psi}_{1}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\eta}_{1k}\right)^{2}\right]-\mathbb{E}_{n,k}\left[\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)^{2}\right]\mid, \] \[ I_{4,k}\coloneqq\mid\mathbb{E}_{n,k}\left[\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)^{2}\right]-E_{P}\left[\bar{\psi}_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)^{2}\right]\mid. \] To bound $I_{4,k}$, we have

where the last inequality follows from (A.13). By Chebyshev's inequality, $I_{4,k}=O_{P}\left(n^{1/2}\right)$.

Next, we bound $I_{3,k}$. This part is essentially identical to the proof of Theorem 3.2 in \citet*{chernozhukov2018double}, I reproduce it here for reader's convenience. Observe that for any number $a$ and $\delta a$, \[ \mid\left(a+\delta a\right)^{2}-a^{2}\mid\leq2\left(\delta a\right)\left(a+\delta a\right). \] Denote $\psi_{i}=\bar{\psi}_{1}\left(W_{i},\theta_{0},p_{0},\eta_{10}\right)$ and $\hat{\psi}_{i}=\bar{\psi}_{1}\left(W_{i},\tilde{\theta},\hat{p}_{k},\hat{\eta}_{1k}\right)$, and $a\coloneqq\psi_{i}$, $a+\delta a\coloneqq\hat{\psi}_{i}$. Then

Thus, \[ I_{3,k}^{2}\lesssim S_{N}\times\left(\frac{1}{n}\sum_{i\in I_{k}}\parallel\bar{\psi}_{1}\left(W_{i},\theta_{0},p_{0},\eta_{10}\right)\parallel^{2}+S_{N}\right), \] where \[ S_{N}\coloneqq\frac{1}{n}\sum_{i\in I_{k}}\parallel\bar{\psi}_{1}\left(W_{i},\tilde{\theta},\hat{p}_{k},\hat{\eta}_{1k}\right)-\bar{\psi}_{1}\left(W_{i},\theta_{0},p_{0},\eta_{10}\right)\parallel^{2}. \] Since $\frac{1}{n}\sum_{i\in I_{k}}\parallel\bar{\psi_{1}}\left(W_{i},\theta_{0},p_{0},\eta_{0}\right)\parallel^{2}=O_{P}\left(1\right)$, it suffices to bound $S_{N}$. We have the decomposition

where $\bar{\theta}\in\left(\tilde{\theta}-\theta_{0}\right)$. The first term is bounded by

Also, notice that conditional on $\left(W_{i}\right)_{i\in I_{k}^{c}}$, both $\hat{p}_{k}$ and $\hat{\eta}_{1k}$ can be treated as fixed. Under the event that $\hat{p}_{k}\in\mathcal{P}_{N}$ and $\hat{\eta}_{1k}\in\mathcal{T}_{N}$, we have

by (A.12). It follows that $S_{N}=O_{P}\left(N^{-1}+\left(\varepsilon_{N}\right)^{2}\right)$. Therefore, we obtain \[ I_{k}=O_{P}\left(N^{-1/2}\right)+O_{P}\left(N^{-1/2}+\varepsilon_{N}\right)=o_{P}\left(1\right). \]

Step 2. It remains to prove (A.12) and (A.13). By Taylor series expansion,

where $\bar{p}\in\left(p,p_{0}\right)$. Then we have

By (A.1), we have $\parallel\psi_{1}\left(W,\theta_{0},p_{0},\eta_{1}\right)-\psi_{1}\left(W,\theta_{0},p_{0},\eta_{10}\right)\parallel_{P,2}=O\left(\varepsilon_{N}\right)$. The term in the second line is bounded by

where I use $\parallel UV_{1}\parallel_{P,2}\leq \parallel UV_{1}\parallel_{P,4}\leq C$ , $E_{P}\left[U^{2}\mid X\right]\leq C$, $E_{P}\left[V_{1}^{2}\mid X\right]\leq C$, and

by $\mid E_{P}\left[UV_{1}\right]\mid\leq\parallel UV_{1}\parallel_{P,4}\leq C$. Thus, we obtain

where I assume that $\varepsilon_{N}$ converges to zero no faster than $N^{-1/2}$.

For (A.13),

since $\parallel UV_{1}\parallel_{P,4}\leq C$.

In Step 1, I show the main result with the auxiliary results: \[ \sup_{p\in\mathcal{P}_{N},\lambda\in\Lambda_{N},\eta_{2}\in\mathcal{T}_{N}}\left(E_{P}\left[\parallel\bar{\psi}_{2}\left(W,\theta_{0},p,\lambda,G_{2\lambda0},\eta_{2}\right)-\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)\parallel^{2}\right]\right)^{2}\leq\varepsilon_{N},\tag{A.14} \]

\[ \left(E_{P}\left[\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)^{4}\right]\right)^{1/4}\leq C_{2}, \tag{A.15} \] where $\left({P}_{N}, \Lambda_{N}, \mathcal{T}_{N} \right)$ are specified in the proof of Theorem 1, $C_{2}$ is a constant, and \[ \bar{\psi}_{2}\left(W,\theta,p,\lambda,G_{2\lambda},\eta_{2}\right)\coloneqq\frac{1}{\lambda\left(1-\lambda\right)p}\frac{D-g\left(X\right)}{1-g\left(X\right)}\left(\left(T-\lambda\right)Y-\ell_{2}\left(X\right)\right)-\frac{D\theta}{p}+G_{2\lambda}\left(T-\lambda\right). \] In fact, we have $E_{P}\left[\left(\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda_{0}},\eta_{20}\right)\right)^{2}\right]=\Sigma_{20}$. In Step 2, I prove (A.14) and (A.15).

Step 1. Notice that

where the second inequality follows from $\hat{G}_{2p}=-\tilde{\theta}/\hat{p}_{k}$ .

Since $K$ is fixed, which is independent of $N$, it suffices to show that \[ J_{k}\coloneqq\mid\mathbb{E}_{n,k}\left[\bar{\psi}_{2}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\lambda}_{k},\hat{G}_{2\lambda},\hat{\eta}_{2k}\right)^{2}\right]-E_{P}\left[\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)^{2}\right]\mid=o_{P}\left(1\right). \] By the triangle inequality, we have \[ J_{k}\leq J_{5,k}+J_{6,k}, \] where \[ J_{5,k}\coloneqq\mid\mathbb{E}_{n,k}\left[\bar{\psi}_{2}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\lambda}_{k},\hat{G}_{2\lambda},\hat{\eta}_{2k}\right)^{2}\right]-\mathbb{E}_{n,k}\left[\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)^{2}\right]\mid, \]

\[ J_{6,k}\coloneqq\mid\mathbb{E}_{n,k}\left[\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)^{2}\right]-E_{P}\left[\bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)^{2}\right]\mid. \] By the same arguments for $I_{4,k}$ in the proof of repeated outcomes and (A.15), we can show $J_{6,k}=o_{P}\left(1\right)$. Also, by the same arguments for $I_{3,k}$ in the proof of repeated outcomes, we have \[ J_{5,k}^{2}\lesssim S_{N}'\times\left(\frac{1}{n}\sum_{i\in I_{k}}\parallel\bar{\psi_{2}}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)\parallel^{2}+S_{N}'\right), \] where \[ S_{N}'\coloneqq\frac{1}{n}\sum_{i\in I_{k}}\parallel\bar{\psi_{2}}\left(W,\tilde{\theta},\hat{p}_{k},\hat{\lambda}_{k},\hat{G}_{2\lambda},\hat{\eta}_{2k}\right)-\bar{\psi_{2}}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)\parallel^{2}. \] Since $\frac{1}{n}\sum_{i\in I_{k}}\parallel\bar{\psi_{2}}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)\parallel^{2}=O_{P}\left(1\right)$, it remains to bound $S_{N}'$. Define $\bar{\psi}_{20}\coloneqq\bar{\psi_{2}}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)$. By the triangle inequality, we have

where $\bar{\theta}\in\left(\tilde{\theta},\theta_{0}\right)$ and $\overline{G_{2\lambda}}\in\left(\widehat{G_{2\lambda}},G_{2\lambda0}\right)$. Then we have

The two terms in the last line are bounded by \[ \left(\frac{1}{n}\sum_{i\in I_{k}}\left(\frac{D_{i}}{\hat{p}_{k}}\right)^{2}\right)\times\parallel\tilde{\theta}-\theta_{0}\parallel^{2}=O_{P}\left(1\right)\times O_{P}\left(N^{-1}\right) \] and \[ \left(\frac{1}{n}\sum_{i\in I_{k}}\left(T_{i}-\hat{\lambda}_{k}\right)^{2}\right)\times\parallel\hat{G}_{2\lambda}-G_{2\lambda0}\parallel^{2}=O_{P}\left(1\right)\times\left(o_{P}\left(1\right)\right)^{2}. \] Conditional on the auxiliary sample $I_{k}^{c}$, $\left(\hat{p}_{k},\hat{\lambda}_{k},\hat{\eta}_{2k}\right)$ can be treated as fixed. Also, under the event that $\hat{p}_{k}\in\mathcal{P}_{N}$, $\hat{\lambda}_{k}\in\Lambda_{N}$, and $\hat{\eta}_{2k}\in\mathcal{T}_{N}$, we have

by (A.14). It follows that $S_{N}'=O_{P}\left(N^{-1}+\varepsilon_{N}^{2}\right)+\left(o_{P}\left(1\right)\right)^{2}$ so that \[ J_{k}=o_{P}\left(1\right)+O_{P}\left(N^{-1/2}+\varepsilon_{N}\right)+o_{P}\left(1\right)=o_{P}\left(1\right). \]

Step 2. It remains to show (A.14) and (A.15). Define $\bar{\psi}_{20}\coloneqq \bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{20}\right)$. By the triangle inequality and \[ \bar{\psi}_{2}\left(W,\theta_{0},p_{0},\lambda_{0},G_{2\lambda0},\eta_{2}\right)-\bar{\psi}_{20}=\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{2}\right)-\psi_{2}\left(W,\theta_{0},p_{0},\lambda_{0},\eta_{20}\right), \] we have

where $\bar{p}\in\left(p,p_{0}\right)$ and $\bar{\lambda}\in\left(\lambda,\lambda_{0}\right)$. The term in the second line is bounded by

by the same arguments in (A.9)-(A.11) and

since $\mid E_{P}\left[UV_{2}\right]\mid\leq\parallel UV_{2}\parallel_{P,4}\leq C$ and $\mid E_{P}\left[YU\right]\mid\leq C$. Also, we have

by the same arguments in (A.9)-(A.11) and

since $\mid E_{P}\left[UV_{2}\right]\mid\leq\parallel UV_{2}\parallel_{P,4}\leq C$. Together with (A.5), we have

where I assume that $\varepsilon_{N}$ converges to zero no faster than $N^{-1/2}$.

For (A.15), we have

since $\parallel UV_{2}\parallel_{P,4}\leq C$.

By the same arguments in the proof of Theorem 1, we can have \[ \sqrt{N}\left(\tilde{\theta}-\theta_{0}\right)=\frac{1}{\sqrt{N}}\sum_{i=1}^{N}\psi_{1}\left(W_{i},\theta_{0},p_{0},\eta_{10}\right)+G_{1p0}\left(D-p_{0}\right)+O_{P}\left(\varepsilon_{N}+\sqrt{N}\left(\varepsilon_{N}\right)^{2}\right) \] for repeated outcomes and

for repeated cross sections. The term $\varepsilon_{N}$ is the rate of convergence of the kernel estimators $\hat{g}_{kh}$, $\hat{\ell}_{1kh}$, and $\hat{\ell}_{2kh}$. It remains to show $ \parallel\hat{g}_{kh}-g_{0}\parallel_{P,2}=o_{P}\left(N^{-1/4}\right) $, $ \parallel\hat{\ell}_{1kh}-\ell_{10}\parallel_{P,2}=o_{P}\left(N^{-1/4}\right) $, and $ \parallel\hat{\ell}_{2kh}-\ell_{20}\parallel_{P,2}=o_{P}\left(N^{-1/4}\right) $.

Here I use the standard result of kernel estimation in \citet*{newey1994large}. Let $\hat{\gamma}_{kh}\left(x\right)$ denote the kernel estimator of $\gamma_{0}\left(x\right)=f_{0}\left(x\right)E_{P}\left[z\mid x\right]$ using the auxiliary sample $I_{k}^{c}$, where $z\in \left\{1,D,Y\left(1\right)-Y\left(0\right)\mid D=0,\left(T-\lambda_{0}\right)Y\mid D=0\right\}$. By Assumption (3.3) and Lemma 8.10 of \citet*{newey1994large}, we have

by the conditions on $h$. Let $\hat{f}_{kh}\left(x\right)$ denote $\hat{\gamma}_{k}\left(x\right)$ with $z=1$ and $\hat{m}_{kh}\left(x\right)$ denote $\hat{\gamma}_{k}\left(x\right)$ with $z=D$. Then \[ \hat{g}_{kh}\left(x\right)=\frac{\hat{m}_{kh}\left(x\right)}{\hat{f}_{kh}\left(x\right)}=\frac{\hat{m}_{kh}\left(x\right)/f_{0}\left(x\right)}{\hat{f}_{kh}\left(x\right)/f_{0}\left(x\right)}. \] For the denominator, we have \[ \sup_{x\in\mathcal{X}}\mid\frac{\hat{f}_{kh}\left(x\right)}{f_{0}\left(x\right)}-1 \mid\leq \frac{\sup_{x\in\mathcal{X}}\mid\hat{f}_{kh}\left(x\right)-f_{0}\left(x\right) \mid}{\inf_{x\in\mathcal{X}}f_{0}\left(x\right)}=o_{P}\left(N^{-1/4}\right) \] given $\inf_{x\in\mathcal{X}}f_{0}\left(x\right)\neq 0$. For the numerator, let $m_{0}\left(x\right)$ denote $\gamma_{0}\left(x\right)$ with $z=D$, we have \[ \sup_{x\in\mathcal{X}}\mid\frac{\hat{m}_{kh}\left(x\right)}{f_{0}\left(x\right)}-g_{0}\left(x\right) \mid \leq \frac{\sup_{x\in\mathcal{X}}\mid\hat{m}_{kh}\left(x\right)-m_{0}\left(x\right) \mid}{\inf_{x\in\mathcal{X}}f_{0}\left(x\right)}=o_{P}\left(N^{-1/4}\right) \] given $\inf_{x\in\mathcal{X}}f_{0}\left(x\right)\neq 0$. The above two inequalities imply that uniformly over $x\in \mathcal{X}$,

\[ \hat{g}_{kh}\left(x\right)=\frac{g_{0}\left(x\right)+o_{P}\left(N^{-1/4}\right)}{1+o_{P}\left(N^{-1/4}\right)}= g_{0}\left(x\right)+o_{P}\left(N^{-1/4}\right). \] That is, $\sup_{x\in\mathcal{X}}\mid \hat{g}_{kh}\left(x\right)-g_{0}\left(x\right)\mid=o_{P}\left(N^{-1/4}\right)$. Using the same arguments, one can also show that $\sup_{x\in\mathcal{X}}\mid \hat{\ell}_{1kh}\left(x\right)-\ell_{0}\left(x\right)\mid=o_{P}\left(N^{-1/4}\right)$ and $\sup_{x\in\mathcal{X}}\mid \hat{\ell}_{2kh}\left(x\right)-\ell_{0}\left(x\right)\mid=o_{P}\left(N^{-1/4}\right)$. Since uniform convergence implies $L^{2}$-norm convergence, we complete the proof.

The proof is the same as the proof in Theorem 2 provided the assumptions in Theorem 3 hold.

Let $\left\{ X_{m}\right\} $ and $\left\{ Y_{m}\right\} $ be sequences of random vectors. (i) If for $\epsilon_{m}\rightarrow0$, $\text{Pr}\left(\parallel X_{m}\parallel>\epsilon_{m}\mid Y_{m}\right)\stackrel{p}{\rightarrow}0$, then $\text{Pr}\left(\parallel X_{m}\parallel>\epsilon_{m}\right)\rightarrow0$. This occurs if $E\left[\parallel X_{m}\parallel^{q}/\epsilon_{m}^{q}\mid Y_{m}\right]\stackrel{p}{\rightarrow}0$ for some $q\geq1$, by Markov's inequality. (ii) Let $\left\{ A_{m}\right\} $ be a sequence of positive constants. If $\parallel X_{m}\parallel=O_{P}\left(A_{m}\right)$ conditional on $Y_{m}$, namely, that for any $\ell_{m}\rightarrow\infty$, $\text{Pr}\left(\parallel X_{m}\parallel>\ell_{m}A_{m}\mid Y_{m}\right)\stackrel{p}{\rightarrow}0$, then $\parallel X_{m}\parallel=O_{P}\left(A_{m}\right)$ unconditionally, namely, that for any $\ell_{m}\rightarrow\infty$, $\text{Pr}\left(\parallel X_{m}\parallel>\ell_{m}A_{m}\right)\stackrel{}{\rightarrow}0$.

PROOF: This lemma is the Lemma 6.1 in \citet*{chernozhukov2018double}.