A Unified Framework for Efficient Estimation of General Treatment Models

Supplemental Material for “A Unified Framework for Efficient Estimation of General Treatment Models ”

\setstretch{1.3}

\listoftables

Complete Simulation Results

In this supplemental material, we present complete simulation results that are partly omitted in the main paper Ai_Motegi_Zhang_cts_treat. See Section (ref) for simulation designs and Section (ref) for results.

Simulation Design

We consider four data generating processes (DGPs):

For each DGP, $\xi_{i} \stackrel{i.i.d.}{\sim} N(0, 9)$, $\epsilon_{i} \stackrel{i.i.d.}{\sim} N(0, 25)$, and $\boldsymbol{X}_{i} = [X_{1i}, X_{2i}, X_{3i}, X_{4i}]^{T} \stackrel{i.i.d.}{\sim} N(0, \boldsymbol{\Sigma})$, where the diagonal elements of $\boldsymbol{\Sigma}$ are all 1 and the off-diagonal elements are all $\rho \in \{ 0.0, 0.2, 0.4 \}$. We draw $J = 1000$ Monte Carlo samples with sample size $N \in \{ 100, 500, 1000 \}$.

Our DGPs have similar structures with Fong_Hazlett_Imai_2018. The characteristics of the four DGPs can be summarized as follows.

In the main paper Ai_Motegi_Zhang_cts_treat, we discuss only DGP-1 and DGP-4 in order to save space. (DGP-4 here is called “DGP-2” in the main paper since DGP-2 and DGP-3 here are skipped.) As shown in the main paper, a parametric version of Fong, Hazlett, and Imai's Fong_Hazlett_Imai_2018 covariate balancing generalized propensity score (CBGPS) estimators produces no bias under DGP-1 and considerable bias under DGP-4.\footnote{ A non-parametric version of the CBGPS estimators is also biased under a data generating process which is similar to DGP-4, and the magnitude of the bias is as large as the parametric version Fong_Hazlett_Imai_2018. } Here we also consider DGP-2 and DGP-3, which serve as intermediate cases where either $T_{i}$ is nonlinear in $\boldsymbol{X}_{i}$ or $Y_{i}$ is nonlinear in $\boldsymbol{X}_{i}$.

Since $Y_{i}$ is linear in $T_{i}$ under all four DGPs, the link function is always of the form $\mathbb{E} [Y^*(t)] = \beta_{1} + \beta_{2} t$. It is straightforward to see that the true coefficients are $(\beta_{1}, \beta_{2}) = (1, 1)$ for each DGP. $\beta_{2}$ is of greater interest than $\beta_{1}$ since $\beta_{2}$ measures the average treatment effect.

To estimate $(\beta_{1}, \beta_{2})$ via our proposed method, we should decide which variables to include in polynomials with respect to $T_{i}$ and $\boldsymbol{X}_{i}$. For each DGP, we use $u_{K_{1}} (t_{i}) = [1, t_{i}, t_{i}^{2}]^{T}$ (i.e. $K_{1} = 3$) and $u_{K_{2}} (\boldsymbol{X}_{i}) = [1, X_{1i}, X_{2i}, X_{3i}, X_{4i}, X_{1i}^{2}, X_{2i}^{2}, X_{3i}^{2}, X_{4i}^{2}]^{T}$ (i.e. $K_{2} = 9$) or $u_{K_{2}} (\boldsymbol{X}_{i}) = [1, X_{1i}, X_{2i}, X_{3i}, X_{4i}, X_{1i}^{2}, X_{2i}^{2}, X_{3i}^{2}, X_{4i}^{2}, X_{1i} X_{2i}, X_{1i} X_{3i}, X_{1i} X_{4i}, X_{2i} X_{3i}, X_{2i} X_{4i}, X_{3i} X_{4i}]^{T}$ (i.e. $K_{2} = 15$). In the literature of non-parametric estimation of treatment effects, it is common to include the first and second moments of $\boldsymbol{X}_{i}$ with or without cross terms chan2016globally. Hence we consider two cases with and without the cross terms of $\boldsymbol{X}_{i}$.

We compare our estimators with the parametric version of Fong, Hazlett, and Imai's Fong_Hazlett_Imai_2018 CBGPS estimator. The non-parametric version of CBGPS is omitted since the parametric and non-parametric versions exhibit similar performance according to the simulation results of Fong_Hazlett_Imai_2018. Computation of the parametric CBGPS estimator involves two steps. The first step is the estimation of stabilized weights, and the second step is the estimation of average treatment effects. Our covariates are chosen to be $\boldsymbol{X}_{i} = [X_{1i}, X_{2i}, X_{3i}, X_{4i}]^{T}$ for the first step and $\boldsymbol{Z}_{i} = [1, T_{i}, \boldsymbol{X}_{i}^{T}]^{T}$ for the second step.\footnote{ In extra simulations not reported here, we added some irrelevant covariates to the model, as Fong_Hazlett_Imai_2018 did, in order to see whether the finite sample performance is sensitive to redundant covariates. Since the results were not so sensitive to the inclusion of irrelevant covariates, we only report results without the irrelevant covariates here. }

Simulation Results

In this section we present our simulation results. See Tables (ref)-(ref) for DGP-1; Tables (ref)-(ref) for DGP-2; Tables (ref)-(ref) for DGP-3; Tables (ref)-(ref) for DGP-4. For each table, we report the bias, standard deviation, root mean squared error, and coverage probability based on the 95% confidence bands.

As discussed in the main paper Ai_Motegi_Zhang_cts_treat, both of our stabilized-weight (SW) estimator and Fong, Hazlett, and Imai's Fong_Hazlett_Imai_2018 CBGPS estimator are unbiased under DGP-1. The latter often has smaller bias and standard deviation across all sample sizes $N \in \{ 100, 500, 1000 \}$. That is not a surprising result since DGP-1 satisfies the linearity assumption underpinning the CBGPS estimator. The coverage probability, however, is sometimes comparable between the two estimators. See, for example, Table (ref) ($\beta_{2}$) with $\rho = 0.2$ and $N = 500$, where the coverage probability is 0.926 for the SW estimator with $K_{2} = 9$ and 0.890 for the CBGPS estimator. The simulation results are not sensitive to the correlation coefficient $\rho \in \{ 0.0, 0.2, 0.4 \}$. The SW estimators with and without the cross terms of $\boldsymbol{X}_{i}$ lead to roughly similar performance.

Similar implications hold for DGP-2. The CBGPS estimator performs well arguably due to a negligibly small degree of nonlinearity. There is a nonlinear term $(X_{1i} + 0.5)^{2}$ in the DGP of $T_{i}$, but the DGP of $Y_{i}$ is kept to be linear. Hence the CBGPS estimator maintains the sharp performance as in DGP-1.

Under DGP-3, the SW estimator begins to be more comparable with the CBGPS estimator. See, for example, Table (ref) ($\beta_{2}$) with $\rho = 0.4$ and $N = 1000$, where the bias is 0.003 for the SW estimator with $K_{2} = 9$ and 0.004 for the CBGPS estimator; the standard deviation is 0.174 for SW and 0.090 for CBGPS; the coverage probability is 0.932 for SW and 0.892 for CBGPS. Under DGP-3, there are two nonlinear terms $0.75 X_{1i}^{2}$ and $0.2 (X_{2i} + 0.5)^{2}$ in the DGP of $Y_{i}$. It is likely that the CBGPS estimator is negatively affected by those nonlinear features while the SW estimator is more robust against nonlinearity.

As discussed in the main paper Ai_Motegi_Zhang_cts_treat, our SW estimator dominates the CBGPS estimator under DGP-4. See, for example, Table (ref) ($\beta_{1}$) with $\rho = 0.4$ and $N = 1000$, where the bias is 0.016 for the SW estimator with $K_{2} = 9$ and -0.182 for the CBGPS estimator; the standard deviation is 0.605 for SW and 0.194 for CBGPS; the coverage probability is 0.923 for SW and 0.800 for CBGPS. Also see Table (ref) ($\beta_{2}$) with $\rho = 0.4$ and $N = 1000$, where the bias is 0.038 for SW and 0.156 for CBGPS; the standard deviation is 0.170 for SW and 0.080 for CBGPS; the coverage probability is 0.916 for SW and 0.304 for CBGPS.

Under DGP-4, the CBGPS estimator keeps producing negative bias in $\beta_{1}$ and positive bias in $\beta_{2}$. The bias is considerably large, and it does not vanish as sample size $N$ increases. Our estimator, in contrast, produces virtually no bias for any sample size $N \in \{ 100, 500, 1000 \}$. It also achieves accurate coverage probability for larger sample sizes $N \in \{ 500, 1000 \}$. The relative advantage of our estimator stems from the strong degree of nonlinearity contained in DGP-4, where both $T_{i}$ and $Y_{i}$ have nonlinear terms.

In summary, our estimator is by construction robust against the functional form of underlying DGPs, while Fong, Hazlett, and Imai's Fong_Hazlett_Imai_2018 estimator is sensitive to model misspecification. In fact, our approach produces unbiased estimates under all DGPs considered, while Fong, Hazlett, and Imai's Fong_Hazlett_Imai_2018 approach produces biased estimates under DGP-4. In reality, the functional form of an underlying DGP is unknown to the researcher. It is therefore of practical use to employ our estimator in order to accomplish correct inference for dose-response curves.

Assumptions

Efficiency Bound

Proof of Theorem 3.1

Without loss of generality, we only consider the distribution of $(T,\boldsymbol{X},Y)$ to be absolutely continuous with respect to Lebesgue measure, i.e., there exists a density function $f_{T,X,Y}(t,\boldsymbol{x},y)$ such that $dF_{T,X,Y}(t,\boldsymbol{x},y)=f_{T,X,Y}(t,\boldsymbol{x},y)dtd\boldsymbol{x}dy$. For discrete cases, the proof can be established by using a similar argument.\\

We follow the approach of Bickel_Klaassen_Ritov_Wellner_1993 to derive the variance bound of $\boldsymbol{\beta}_0$, see also tchetgen2012semiparametric. Let $\left\{f^{\alpha}_{Y,T,X}(y,t,\boldsymbol{x})\right\}_{\alpha \in\mathbb{R}}$ denote a one dimensional regular parametric submodel with $ f^{\alpha=0}_{Y,T,X}(y,t,\boldsymbol{x})=f_{Y,T,X}(y,t,\boldsymbol{x})$. By definition, $\boldsymbol{\beta}_0$ solves following equation:

By Assumption (ref), (ref) is equivalent to

Therefore, the parameter $\boldsymbol{\beta}(\alpha)$ induced by the submodel $f^{\alpha}_{Y,T,X}(y,t,\boldsymbol{x})$ satisfies:

where $\mathbb{E}^{\alpha}\left[\cdot|T=t,\boldsymbol{X}=\boldsymbol{x}\right]$ denotes taking expectation with respect to the submodel $f^{\alpha}_{Y|T,X}(\cdot|t,\boldsymbol{x})$. \\

Differentiating both sides of (ref) with respect to $\alpha$, evaluating at $\alpha = 0$ and using the condition $Y^*(t)\perp T|\boldsymbol{X}$, we can deduce that

Since $H_0=- \nabla_{\beta}\left\{\int_{\mathcal{T}} \mathbb{E}[L'(Y^*(t)-g(t;\boldsymbol{\beta}))]\cdot m(t;\boldsymbol{\beta})f_{T}(t) dt\right\}\Bigg|_{\boldsymbol{\beta}=\boldsymbol{\beta}_0}$ is invertible by Assumption (ref), we get

The efficient influence function of $\boldsymbol{\beta}_0$, denoted by $S_{eff}(Y,T,\boldsymbol{X};\boldsymbol{\beta}_0)$, is a unique function satisfying the following equation:

Therefore, to justify our theorem, it suffices to substitute $S_{eff}(Y,T,\boldsymbol{X};\boldsymbol{\beta}_0) = H_0 ^{-1} \psi(Y,T,\boldsymbol{X};\boldsymbol{\beta}_0)$ into (ref) and check the validity. Note that

For the term (ref), we have

For the term (ref), we have

where the first equality holds in accordance with the definition of $\int_{\mathcal{Y}}L'(y-g(t;\boldsymbol{\beta}_0))f_{Y|X,T}(y|\boldsymbol{x},t)dy=:\varepsilon(t,\boldsymbol{x};\boldsymbol{\beta}_0)$. \\

For the term (ref), we have

We have proved (ref) holds, hence $S_{eff}$ is the efficient influence function of $\boldsymbol{\beta}_0$.

Particular Case I: Binary Treatment Effects

In this section, we show that when $T\in\{0,1\}$, $g(t;\boldsymbol{\beta})=\beta_0+\beta_1\cdot t$ and $L(v)=v^2$, our general efficiency bound derived in Theorem 3.1 reduces to the well-known efficiency bound for average treatment effects in robins1994estimation and hahn1998role. In accordance with our identification condition, $\beta_0$ and $\beta_1$ are identified by minimizing the following loss function $$\sum_{t\in\{0,1\}}\mathbb{E}[(Y^*(t)-\beta_0-\beta_1\cdot t)^2]\cdot \mathbb{P}(T=t).$$ The solutions are given by $$\beta_0=\mathbb{E}[Y^*(0)], \ \beta_1=\mathbb{E}[Y^*(1)-Y^*(0)].$$ Here $\beta_1$ is the average treatment effects.

Particular Case II: Multiple Treatment Effects

In this section, we show that when $T\in\{0,1,...,J\}$, $J\in\mathbb{N}$, $g(t;\boldsymbol{\beta})=\sum_{j=0}^J \beta_j \cdot I(t=j)$ and $L(v)=v^2$, our general efficiency bound derived in Theorem 3.1 reduces to the efficiency bound of multi-level treatment effects given in cattaneo2010efficient. In accordance with our proposed identification condition, $\{\beta_j\}_{j=0}^J$ are identified by minimizing the following loss function $$\sum_{j=0}^J\mathbb{E}\left[(Y^*(j)-\beta_j)^2\right]\cdot \mathbb{P}(T=j).$$ The solutions are $\beta_j=\mathbb{E}[Y^*(j)]$ for $j\in\{0,...,J\}$.

Particular Case III: Quantile Treatment Effects

In this section, we show that when $T\in\{0,1\}$ is a binary treatment variable, $L(v)=v(\tau-I(v\leq 0))$ is the check function with $\tau\in(0,1)$, and $g(t;\boldsymbol{\beta}_0)=\beta_0\cdot (1-t)+\beta_1\cdot t$, where $\boldsymbol{\beta}_0=(\beta_0,\beta_1)$, our general efficiency bound derived in Theorem 3.1 reduces to the efficiency bound of quantile treatment effects given in Firpo2007Efficient. In accordance with our identification condition, $\beta_0$ and $\beta_1$ are identified by minimizing the following loss function

The solutions are $\beta_0=\inf\{q: \mathbb{P}(Y^*(0)\leq q)\geq \tau\}$ and $\beta_1=\inf\{q: \mathbb{P}(Y^*(1)\leq q)\geq \tau\}$, which are the $\tau^{th}$ quantiles of potential outcomes.

Convergence Rate of Estimated Stabilized Weights

In this section, we establish the convergence rate of estimated stabilized weights $\hat{\pi}_K(T,\boldsymbol{X})$. Let $G_{K_1\times K_2}^*$, $\Lambda_{K_1\times K_2}^*$ and $\pi_K^*(t,\boldsymbol{x})$ be the theoretical counterparts of $\hat{G}_{K_1\times K_2}$, $\hat{\Lambda}_{K_1\times K_2}$ and $\hat{\pi}_K(t,\boldsymbol{x})$ respectively:

As discussed in Appendix A.3, we assume the sieve basises $u_{K_1}(T)$ and $v_{K_2}(\boldsymbol{X})$ are orthonormalized, i.e.,

Let

We also recall the following property satisfied by $\pi_0(T,\boldsymbol{X})$: for any integrable functions $u(t)$ and $v(\boldsymbol{X})$,

Lemma $\ref{lemma_pi^*}$

The first lemma states that $\pi_K^*(t,\boldsymbol{x})$ is arbitrarily close to the true stabilized weights $\pi_0(t,\boldsymbol{x})$.

Lemma $\ref{lemma_pi^hat}$

Corollary $\ref{cor:pi^-pi*}$

The next corollary states that $\hat{\pi}_K(t,\boldsymbol{x})$ is arbitrarily close to ${\pi}^*_K(t,\boldsymbol{x})$.

Efficient Estimation

Proof of Theorem 5.17

Since we have proved $\|\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}_0\|\xrightarrow{p}0$ in Theorem 10, we now begin to prove the asymptotic efficiency of $\hat{\boldsymbol{\beta}}$. By Assumption (ref), $\hat{\boldsymbol{\beta}}$ is a unique solution of the following equation:

with probability approaching to one. Note that $L'(\cdot)$ may be a non-differentiable function, e.g. $L'(v)=\tau-I(v\leq 0)$ in quantile regression, we cannot simply apply Mean Value Theorem on (ref) to obtain the expression for $\sqrt{N}(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}_0)$. To solve this problem, we define

which is a differentiable function in $\boldsymbol{\beta}$ and by definition $f(\boldsymbol{\beta}_0)=0$. Using Mean Value Theorem, we can obtain that

where $\tilde{\boldsymbol{\beta}}$ lies on the line joining $\hat{\boldsymbol{\beta}}$ and $\boldsymbol{\beta}_0$. Because $\nabla_{\beta}f(\boldsymbol{\beta})$ is continuous in $\boldsymbol{\beta}$ at $\boldsymbol{\beta}_0$, and $\|\hat{\boldsymbol{\beta}}-{\boldsymbol{\beta}}_0\|\xrightarrow{p}0$, then we have

Define the empirical process:

and by Assumption (ref) we can have

By Assumption (ref) and (ref), Theorems 4 and 5 of andrews1994empirical, we can conclude that $\mu_N(\cdot)$ is stochastically equicontinuous, which implies $\mu_N(\hat{\boldsymbol{\beta}}) -\mu_N(\boldsymbol{\beta}_0)\xrightarrow{p}0$. We also note that $\mathbb{E}[\pi_0 (T,\boldsymbol{X})L'(Y -g(T;\boldsymbol{\beta}_0 ))m(T;\boldsymbol{\beta}_0)]=0$, then

We next claim the following important Lemma (ref), and leave its proof to Section (ref).

Lemma (ref) is the most important step for establishing the efficiency of our proposed estimator. A key technique in proving Lemma (ref) is a use of a weighted least square projection of $L'(Y-g(T;\boldsymbol{\beta}_0))$ onto the space linearly spanned by the approximation basis $\{u_{K_{1}}(T),v_{K_{2}}(\boldsymbol{X})\}$.

Combining (ref) and Lemma (ref), we can obtain the asymptotic expression for $\sqrt{N}(\hat{ \boldsymbol{\beta }}-\boldsymbol{\beta }_{0})$:

which leads to our Theorem 5.17.

Proof of Lemma (ref)

Before proving Lemma (ref), we prepare some preliminary notation and results that will be used later. Since $\hat{\Lambda}_{K_1\times K_2}$ is a unique maximizer of the concave function $\hat{G}_{K_1\times K_2}$, then

Using Mean Value Theorem, we can have

where $\tilde{\Lambda}_{K_1\times K_2}$ lies on the line joining from $\hat{\Lambda}_{K_1\times K_2}$ to ${\Lambda}^*_{K_1\times K_2}$. We define the following notation:

and

In light of (ref) we have $$\left\|A^*_{K_1\times K_2}\right\|= O_p\left(\sqrt{\frac{K}{N}}\right). $$ From (ref), $A^*_{K_1\times K_2}$ can also be written as

We now start to prove Lemma (ref). We decompose $\frac{1}{\sqrt{N}}\sum_{i=1}^N\hat{\pi}_K(T_i,\boldsymbol{X}_i)\left\{Y_i-g\left(T_i;\boldsymbol{\beta}_0\right)\right\}m(T_i;\boldsymbol{\beta}_0)$ as follows:

where $\hat{A}_{K_1\times K_2}$ and $A_{K_1\times K_2}^*$ are defined in (ref) and (ref)}. We show that the terms (ref)-(ref) are all of $o_p(1)$, while the term (ref) is asymptotically normal.\\

For term (ref):

Denoting (ref) by $W_K$ and applying Mean Value Theorem twice, we can obtain

where $\tilde{A}_{K_1\times K_2}$ is defined in (ref), and{

} and $\xi_3(t,\boldsymbol{x})$ lies between $u_{K_1}(t)\tilde{\Lambda}_{K_1\times K_2}^{\top}v_{K_2}(\boldsymbol{x})$ and $u_{K_1}(t){\Lambda}^*_{K_1\times K_2}v_{K_2}(\boldsymbol{x})$. \\

For the term $W_{1K}$, we denote its $k^{th}$ component by $W_{1K,k}$, $k=1,\ldots,p$, and we also let $m_k(T_i;\boldsymbol{\beta}_0)$ be the $k^{th}$ component of $m(T_i;\boldsymbol{\beta}_0)$, i.e.,

where

We compute the second moment of $U_{K_2\times K_1}(k)$ to get that

where $\dps a_3 := \sup_{\gamma \in \Gamma_1} |\rho''(\gamma)|^2 < +\infty$, the second inequality follows from this definition and the fact that $u_{K_1}^{\top}(t){\Lambda}^*_{K_1\times K_2}v_{K_2}(\boldsymbol{x}) \in \Gamma_1,\ \forall (t,\boldsymbol{x}) \in \mathcal{T}\times\mathcal{X}$ when $K$ is large enough; the third inequality follows from Assumption (ref); the forth inequality follows from Assumption (ref) and the facts

Then in light of Chebyshev's inequality, Lemma (ref) and Assumption (ref), we have $$|W_{1K,k}|\leq \|U_{K_2\times K_1}\|\|\hat{A}_{K_1\times K_2}\| =O_p(\sqrt{K})O_p\left(\sqrt{\frac{K}{N}}\right)=O_p\left(\sqrt{\frac{K^2}{N}}\right) ,$$ which implies

For the term $W_{3K}$, since $\xi_3(t,\boldsymbol{x})$ lies between $u_{K_1}(t)^{\top}\Lambda_{K_1\times K_2}^*v_{K_2}(\boldsymbol{x})$ and $u_{K_1}(t)^{\top}\tilde{\Lambda}_{K_1\times K_2}^*v_{K_2}(\boldsymbol{x})$, which implies $\xi_3(t,\boldsymbol{x})$ lies between $u_{K_1}(t)^{\top}\Lambda_{K_1\times K_2}^*v_{K_2}(\boldsymbol{x})$ and $u_{K_1}(t)^{\top}\hat{\Lambda}_{K_1\times K_2}^*v_{K_2}(\boldsymbol{x})$. Then in light of (ref) and (ref), we have $\mathbb{P}\left(\xi_3(t,\boldsymbol{x})\in \Gamma_2(\epsilon),\ \forall (t,\boldsymbol{x})\in \mathcal{T}\times \mathcal{X}\right)>1-\epsilon$, therefore,

With (ref), (ref), the fact $\|\tilde{A}_{K_1\times K_2}\|\leq \|\hat{A}_{K_1\times K_2}\|$, Lemma (ref), and Assumption (ref), we can derive that{

}

For the term $W_{2K}$, we can deduce that {

} where the fourth inequality follows from the fact that{

} Therefore, we can obtain that

Finally, it follows that the term (ref) is of $o_p(1)$ in light of Assumption (ref).\\

For term (ref): Note that

where the last equality follows from Lemma (ref). Then by Chebyshev's inequality, we can claim that the term (ref) is of $O_p(\zeta(K)K^{-\alpha})$.\\

\noindentFor term (ref): By Lemma (ref) and Assumption (ref), we can deduce that

For term (ref): By Mean Value Theorem and the definition of $\hat{A}_{K_1\times K_2}$ in (ref), the term (ref) is exactly equal to zero. \\

For term (ref): We can telescope (ref) as follows:

For the term (ref), by Mean Value Theorem, {

} which is $O_p\left(\sqrt{\frac{K^2}{N}}\right)$ from (ref).\\

For the term (ref), we first compute the probability order of $\|A^*_{K_1\times K_2}-\hat{A}_{K_1\times K_2}\|$. Using (ref)}, the fact $\rho''(v)=-\rho'(v)$ and Mean Value Theorem, we have{

} For the term (ref), by (ref) we can write $\hat{A}_{K_1\times K_2}$ as $$ \hat{A}_{K_1\times K_2}=\mathbb{E}_{T,\boldsymbol{X}}\left[\pi_0(T,\boldsymbol{X})u_{K_1}(T)u_{K_1}(T)^{\top}\hat{A}_{K_1\times K_2}v_{K_2}(\boldsymbol{X})v^{\top}_{K_2}(\boldsymbol{X})\right], $$ where $\mathbb{E}_{T,\boldsymbol{X}}[\cdot]$ denotes taking expectation with respect to $(T,\boldsymbol{X})$. We telescope (ref) as follows:

For the term (ref), by Lemmas (ref) and (ref), we have that{

} For the term (ref), define the linear map $\mathcal{J}(\cdot):\mathbb{R}^{{K_1\times K_2}}\rightarrow \mathbb{R}$ by {

} then $ \eqref{A_diff_1_2}=\mathcal{J}(\hat{A}_{K_1\times K_2})$. For any fixed $M \in \mathbb{R}^{{K_1\times K_2}}$, by (ref) and $M=\mathbb{E}[\pi_0(T,\boldsymbol{X})u_{K_1}(T)u_{K_1}(T)^{\top}M$ $\cdot v_{K_2}(\boldsymbol{X})v^{\top}_{K_2}(\boldsymbol{X})]$, then we have

Using Chebyshev's inequality we have $$ |\mathcal{J}(M)| =\|M\| O_p\left(\zeta(K)\sqrt{\frac{K}{N}}\right)\ ,$$ then in light of Lemma (ref), $$\eqref{A_diff_1_2}=\mathcal{J}(\hat{A}_{K_1\times K_2})=\|\hat{A}_{K_1\times K_2}\| O_p\left(\zeta(K)\sqrt{\frac{K}{N}}\right)= O_p\left(\zeta(K)\frac{K}{N}\right)\ .$$ Therefore, $$\eqref{A_difference_1}=\eqref{A_diff_1_1}+\eqref{A_diff_1_2}= O_p\left(N^{-\frac{1}{2}}\zeta(K)\cdot K^{\frac{3}{2}-\alpha}\right)+O_p\left(\zeta(K)\frac{K}{N}\right) \ .$$

For the term (ref), we can deduce that

where the fourth inequality follows from (ref) and Lemma (ref). Now, we can obtain

Using (ref), Assumptions (ref) and (ref), for large enough $N$, we have {

} where the second inequality holds since by using the same argument of establishing (ref), we have $$\int_{\mathcal{T}\times\mathcal{X}}\left(u_{K_1}(t)\left\{\hat{A}_{K_1\times K_2}-A^*_{K_1\times K_2}\right\}v_{K_2}(\boldsymbol{x})\right)^2dF_{T,X}(t,\boldsymbol{x})= O( \|\hat{A}_{K_1\times K_2}-A_{K_1\times K_2}^*\|).$$ Therefore, we can obtain that

For term (ref): By the definition of $A^*_{K_1\times K_2}$ in (ref), we have{

} We shall show that both (ref) and (ref) are of $o_p(1)$. Noting $\rho''=-\rho'$, we can telescope (ref) as follows: {

} We shall show that (ref), (ref) and (ref) are all of $o_p(1)$. Note that second moment of (ref) is {

} where the third equality holds because $$\int_{\mathcal{T}}\int_{\mathcal{X}}\pi_0(t,\boldsymbol{x})\cdot m(t;\boldsymbol{\beta}_0)\varepsilon(t,\boldsymbol{x};\boldsymbol{\beta}_0)\bigg[\frac{\rho'\left(u_{K_1}^{\top}(t)\Lambda_{K_1\times K_2}^*v_{K_2}(\boldsymbol{x})\right)}{\pi_0(t,\boldsymbol{x})}\bigg]u_{K_1}^{\top}(t) \left\{u_{K_1}(T)v^{\top}_{K_2}(\boldsymbol{X})\right\}v_{K_2}(\boldsymbol{x})dF_{X,T}(\boldsymbol{x},t) $$ is the weighted $L^2$-projection of $m(t;\boldsymbol{\beta}_0)\varepsilon(t,\boldsymbol{x};\boldsymbol{\beta}_0)\bigg[\frac{\rho'\left(u_{K_1}^{\top}(t)\Lambda_{K_1\times K_2}^*v_{K_2}(\boldsymbol{x})\right)}{\pi_0(t,\boldsymbol{x})}\bigg]$ on the space linearly spanned by $\{u_{K_1}(t),v_{K_2}(\boldsymbol{x})\}$ with the weighted measure $\pi_0(t,\boldsymbol{x})dF_{T,X}(t,\boldsymbol{x})$. Similarly, we can also show (ref) and (ref) are of $o_p(1)$. Therefore, (ref) is of $o_p(1)$.\\

For the term (ref), since $\rho''(v)=-\rho'(v)$ and the fact $\mathbb{E}\left[\pi_0(T,\boldsymbol{X})m(T;\boldsymbol{\beta}_0)\varepsilon(T,\boldsymbol{X};\boldsymbol{\beta}_0)\right]=0$, we telescope it as follows:{

} For the term (ref), since

and by Assumptions (ref), (ref), and (ref), we can deduce that

For the term (ref), noting the fact that $\mathbb{E}\left[\pi_0(T,\boldsymbol{X})m(T;\boldsymbol{\beta}_0)\varepsilon(T,\boldsymbol{X};\boldsymbol{\beta}_0)|\boldsymbol{X}\right]=\int_{\mathcal{T}}m(t;\boldsymbol{\beta}_0)\varepsilon(t,\boldsymbol{X};\boldsymbol{\beta}_0)$ $dF_T(t)$, we can rewrite (ref) as follows:

By computing the second moment of (ref), we can obtain that{

} where $T^*\sim F_T$, $\boldsymbol{X}^*\sim F_X$, and $T^*$ is independent of $\boldsymbol{X}^*$; the first inequality holds by Jensen's inequality; the last equality follows from Lemma (ref) and the fact that

is the $L^2$-projection of $m(T^*;\boldsymbol{\beta}_0)\varepsilon(T^*,\boldsymbol{X}^*;\boldsymbol{\beta}_0)$ on the space spanned by $\{u_{K_1}(T^*),v_{K_2}(\boldsymbol{X}^*)\}$, which implies {

} Thus (ref) is of $o_p(1)$ by Chebyshev's inequality. Similar argument can be applied to show that both (ref) and (ref) are of $o_p(1)$. Therefore, we can have that

Then, we can obtain that

Summing up all orders (ref)-(ref) and using Assumption (ref), we have{

}

Variance Estimation

masry1996multivariate studies the strong consistency of kernel regression estimation. The following conditions are imposed so that Theorem 6 of masry1996multivariate applies. Let $\boldsymbol{u}=(u_1,...,u_{r+1})$ and $K(\boldsymbol{u})=\prod_{j=1}^{r+1}k(u_j)$.\\

\noindentCondition 1. The kernel $K(\boldsymbol{u})\in L_1$ satisfies $\|\boldsymbol{u}\|K(\boldsymbol{u})\in L_{1}$ and $\|\boldsymbol{u}\|^2K(\boldsymbol{u})\in L_{1}$.

\noindentCondition 2. The density function $f_{Y,X,T}(y,\boldsymbol{x},t)$ is uniformly bounded away from zero and above, and also it is uniformly continuous on $\mathbb{R}^{r+1}$.

\noindentCondition 3. (a) The kernel $K(\cdot)$ is bounded with compact support; (b) let $H_j(\boldsymbol{u}):=\boldsymbol{u}^jK(\boldsymbol{u})$, and $|H_j(\boldsymbol{u})-H_j(\boldsymbol{v})|\leq C\|\boldsymbol{u}-\boldsymbol{v}\|$ for all $j$ with $0\leq j\leq 3$.

\noindentCondition 4. The functions $f_{Y,X,T}(y,\boldsymbol{x},t)$, $\mathbb{E}\left[\pi_0(T,\boldsymbol{X})L'(Y-g(T;\boldsymbol{\beta}))|T=t,\boldsymbol{X}=\boldsymbol{x}\right]$, $\mathbb{E}[\pi_0(T,\boldsymbol{X})L'(Y-g(T;\boldsymbol{\beta}))|T=t]$ and $\mathbb{E}\left[\pi_0(T,\boldsymbol{X})L'(Y-g(T;\boldsymbol{\beta}))|\boldsymbol{X}=\boldsymbol{x}\right]$ are twice differentiable, and the derivatives are Lipschitz continuous and uniformly bounded.

\noindentCondition 5. $\mathbb{E}\left[\sup_{\boldsymbol{ \beta}\in \Theta}|L'(Y-g(T;\boldsymbol{\beta}))|^{\sigma}\right]<\infty$ for some $\sigma>2$.

\noindentCondition 6.The bandwidths $h_1\asymp \cdots \asymp h_r \asymp h_Y \asymp h_T\asymp h_N$ go to zero slowly enough such that

Some Extensions

Proof of Theorem 7.1

(Proof of Consistency). Let

then $\hat{\theta}_K(t)=\hat{\gamma}^{\top}u_{K_1}(t)$. By assumption, there exists $\gamma^*\in\mathbb{R}^{K_1}$ such that

We first claim that

and the proof will be established later. With the claim (ref), we first show that $\int_{\mathcal{T}}|\hat{\theta}_K(t)-\theta(t)|^2dF_T(t)=O_p\left(\frac{\zeta(K)^2K}{N}+\zeta(K)^2K^{-2\alpha}+K_1^{-2\tilde{\alpha}}\right)$. Note that

With the claim (ref), we next show that $\sup_{t\in\mathcal{T}}|\hat{\theta}_K(t)-\theta(t)|=O_p[\zeta_1(K_1)(\zeta(K)\sqrt{K/N}+\zeta(K)K^{-\alpha}+K_1^{-\alpha})]$. Note that

Finally, we turn back to prove the claim (ref). Note that

where

We first compute the probability order of $A_{1N}$. We use the following notation:

Then we can obtain that

where the first inequality follows from the fact that $tr(AB)\leq \lambda_{\max}(B)tr(A)$ for any symmetric matrix $B$ and positive semidefinite matrix $A$, the second inequality follows from the same fact and the fact that $U_{N\times K_1}(U^{\top}_{N\times K_1}U_{N\times K_1})^{-1}U_{N\times K_1}^{\top}$ is a projection matrix with maximum eigenvalue 1, and the fourth inequality follows from the facts that $|\lambda_{\min}(\hat{\Phi}_{K_1\times K_1})|^{-1}=O_p(1)$, $\sup_{(t,x)\in\mathcal{T}\times \mathcal{X}}|\hat{\pi}_K(t,x)-{\pi}_0(t,x)|=O_p\left(\zeta(K)K^{-\alpha}+\zeta(K)\sqrt{K/N}\right)$ and $N^{-1}\sum_{i=1}^NY_i^2=O_p(1)$. \\

Next, we compute the probability order of $A_{2N}$. We can deduce that

where the last equality follows that $|\lambda_{\min}(\hat{\Phi}_{K_1\times K_1})|^{-1}=O_p(1)$ and $N^{-2}\|U_{N\times K_1}^{\top}\mathcal{E}_N\|^2=O_p(K_1/N)$ by Markov's inequality. \\

We finally compute the probability order of $A_{3N}$. We define the notation

then it follows that with probability approaching to 1,

where the first inequality follows from the fact that $tr(AB)\leq \lambda_{\max}(B)tr(A)$ for any symmetric matrix $B$ and positive semidefinite matrix $A$, the second inequality follows from the same fact and the fact that $U_{N\times K_1}(U^{\top}_{N\times K_1}U_{N\times K_1})^{-1}U_{N\times K_1}^{\top}$ is a projection matrix with maximum eigenvalue 1, and the last equality follows from the fact that $|\lambda_{\min}(\hat{\Phi}_{K_1\times K_1})|^{-1}=O_p(1)$ and the fact that $\frac{1}{N}\sum_{i=1}^N\left\{\mathbb{E}[{\pi}_0(T_i,\boldsymbol{X}_i)Y_i|T_i]-(\gamma^*)^{\top}u_{K_1}(T_i)\right\}^2\leq \sup_{t\in\mathcal{T}}|\mathbb{E}[{\pi}_0(T,X)Y|T]-(\gamma^*)^{\top}u_{K_1}(t)|^2=O(K_1^{-2\tilde{\alpha}})$. Thus we complete the proof of (ref).\\

(Proof of Asymptotic Normality). We have the following decomposition for $\hat{\theta}(t)-\theta(t)$:

where

Then we have that

We shall show that $b_{1N}(t)$ contributes to the asymptotic variance; and $b_{2N}(t)+b_{3N}(t)$ contributes to the asymptotic bias which is asymptotically negligible. Thus to complete the proof of asymptotic normality, it is sufficient to prove the following results:

We first prove Result (i). Note that $\lambda_{\min}(\Sigma_{K_1\times K_1})\geq \underline{c}_{\sigma^2}\lambda_{\min}(\Phi_{K_1\times K_1})=\underline{c}_{\sigma^2}\lambda_{\min}(I_{K_1\times K_1})\geq \underline{c}_{\sigma^2}$, we can have

For the claim (ii). Let

Similar to the proof of Lemma (ref), we can have that

Then

For $B_{1N,1}(t)$, we can simply apply the Liapounov CLT and show that $B_{1N,1}(t)\xrightarrow{d}N(0,1)$. For $B_{1N,2}(t)$, let $\boldsymbol{T}=(T_1,...,T_N)$, we can obtain that

where we use the fact that $N^{-1} \mathbb{E}\left[U_{N\times K_1}^{\top}\mathcal{E}_N\mathcal{E}_N^{\top}U_{N\times K_1}|\boldsymbol{T}\right]=N^{-1}\sum_{i=1}^Nu_{K_1}(T_i)u_{K_1}(T_i)^{\top}\sigma^2(T_i)$ has bounded maximum eigenvalue. Therefore, $B_{1N,1}(t)=o_p(1)$ by the conditional Chebyshev's inequality. Thus (ii) holds. \\

For (iii), by Cauchy-Schwarz's inequality, we can obtain that

Similarly, we can show show that $\sqrt{N}V_{K}^{-1/2}|b_{3N}(t)|=o_p(1)$. This completes the proof of the Theorem.

Proof Theorem 7.2

Similar to the proof of Lemma (ref), let $\mu(t,x)=\mathbb{E}[Y|T=t,X=x]$, we decompose $\sqrt{N}(\hat{\psi}_K-\psi)$ as follows:

Using the similar argument for showing that (ref)-(ref) are all $o_p(1)$ in the proof of Lemma (ref), we can obtain that the terms (ref)-(ref) are all $o_p(1)$, while the term (ref) is asymptotically normal and attains the efficiency bound.

\singlespacing