Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
48,438 characters · 18 sections · 72 citation commands
Triple/Debiased Lasso for Statistical Inference of Conditional Average Treatment Effects
The estimation of causal effects from observational data, especially in the context of binary treatments, is a crucial task in various disciplines, including economics Wager2018, medicine Assmann2000, and online advertising Bottou13. Our focus is on estimating Conditional Average Treatment Effects hahn1998role,Heckman1997,Abrevaya2015, the difference between expected outcomes of binary treatments conditioned on covariates. CATEs play an important role as a metric that represents individual heterogeneous treatment effects.
In this study, given observations generated from a distribution, we consider estimating the CATE. There are two main challenges in CATE estimation. First, we can only observe the outcomes of the treatments that are actually assigned, while the outcomes of the unassigned treatments remain unknown. This issue requires us to deal with the unobserved outcomes in some manner to estimate the CATE accurately. Second, estimating the CATE necessitates regression models to approximate the conditional expected outcomes of the treatments, which demands specific estimation methods depending on the regression models employed.
In this problem, we assume linear regression models for CATE estimation with high-dimensional covariates. We regress the covariates on estimated counterfactual differences of outcomes to estimate the CATE. To handle high dimensionality, we employ Lasso regularization, which leverages the sparsity of linear regression models.
Throughout this study, we assume sparsity for linear CATE regression models without imposing sparsity on regression models for each treatment's outcome; that is, we impose sparsity only for the difference of outcomes, not for each outcome. In this setting, we propose a three-step estimation method: (i) estimating nuisance parameters with some estimation methods with a convergence rate guarantee ; (ii) estimating the difference in outcomes and regressing covariates on them with Lasso regularization; and (iii) debiasing the bias introduced by the use of Lasso regularization using the debiased Lasso. The first and second steps depend on double/debiased machine learning (DML) technique klaassen1987,ZhengWenjing2011CTME,ChernozhukovVictor2018Dmlf, while the third step is based on the debiased Lasso vandeGeer2014. Therefore, we refer to our estimation method as the Triple/Debiased Lasso (TDL) for CATE estimation. We demonstrate the obtained estimator's consistency and asymptotic normality.
The challenge in parameter estimation in high-dimensional regression scenarios, particularly when the parameter space exceeds the sample size, is significant. Our approach adopts high-dimensional but sparse linear regression models for the CATE and applies Lasso regression tibshirani96regression, zhao06a, vandeGeer2008. Following existing studies, such as kato2023cate, we do not assume sparsity for regression models between covariates and each treatment's outcome. Instead, we directly assume sparsity for the regression models of the difference between the outcomes.
Since we cannot observe the outcomes of the unassigned treatments, we approximate them with a doubly robust (DR) estimator bang2005drestimation with nuisance parameters estimated via the cross-fitting in DML. This method guarantees the first convergence of the bias caused by the DR estimation.
Then, we regress covariates on the estimated difference with Lasso regularization. Since Lasso regularization also yields bias, we debias it using the debiased (desparsified) Lasso technique vandeGeer2014. This technique leads us to construct a confidence interval for each regression coefficient.
Furthermore, in Lasso regression, we utilize the weighted least squares to reduce the asymptotic variance of the regression coefficient estimators due to heterogeneous variances.
The use of DR estimation prior to regression has existing studies, such as vanderLaan2006, Lee2017, ninomiya2021selective, Fan2022, and tan2024combining. Such approaches are summarized as the DR-learner by Kunzel2019. Among methods categorized as variants of the DR-learner, Fan2022 and ninomiya2021selective consider DR estimator with confidence intervals. While Fan2022 employs the nonparametric regression with multiplier-bootstrap, ninomiya2021selective assumes linear models and employs Lasso regression with the selective inference. Our method is also categorized into the DR-learner with linear models and DML, but we construct a confidence interval using the debiased Lasso.
Our basic estimation strategy is similar to that in Fan2022, which also employs DML to debias the bias caused by the first-step machine learning estimator. The difference between our estimator and their estimator lies in the use of weighted least squares and debiased Lasso. While Fan2022 does not take into account the existence of the heterogeneous variances, the conditional variance given covariates differ across the covariates due to the covariate-dependent propensity score; that is, the variance is heterogeneous, and under the standard arguments in linear regression, we can minimize the asymptotic variance of regression coefficient estimators by using the weighted least squares. Following this argument, we add the weighted least squares between the procedures of DML and inference, which has not been incorporated in Fan2022. Furthermore, we utilize the debiased Lasso to obtain a confidence interval, following vandeGeer2014, in contrast to the multiplier-bootstrap in Fan2022.
Organization. Our problem setting is formulated in Section (ref). For linear regression models, we introduce Lasso-based estimators with the DR estimation, DML, and debiased Lasso in Section (ref), called the TDL estimator. For the TDL estimator, we show the consistency and asymptotic normality in Section (ref). Section (ref) introduces related work.
This section formulates the problem of CATE estimation. In Section (ref), potential outcomes are introduced, following the Neyman-Rubin causal model Neyman1923,Rubin1974. In Section (ref), the observations are defined. Subsequently, in Section (ref), the CATE, the target of our estimation using the observations, is defined.
In our problem, two treatments $\{1, 0\}$ are given.\footnote{For example, we often regard treatment $1$ as a treatment of interest, and treatment $0$ as the control treatment.} Each treatment yields a corresponding outcome. Following the Neyman-Rubin causal model Neyman1923,Rubin1974, for each treatment $d\in\{1, 0\}$, we introduce potential outcome random variables $Y(d) \in\mathbb{R}$. We assume that $Y(1)$ and $Y(0)$ are bounded random variables.
We characterize individuals who receive treatments by $p$-dimensional covariates $X\in\mathcal{X}$, where $\mathcal{X}\subset\mathbb{R}^p$ is the covariate space.
In the previous section, we introduced the potential outcomes $Y(1)$ and $Y(0)$. However, we can only observe one of the outcomes corresponding to a treatment actually assigned, which is denoted by $D\in\{1, 0\}$. Let $D \in \{1, 0\}$. Then, an observable outcome $Y\in\mathbb{R}$ is defined as
Here, if $D = 1$, $Y = Y(1)$ holds; if $D = 0$, $Y = Y(0)$ holds.
Let $P_0\in\mathcal{P}$ be the “true” distribution that generates $(Y, D, X)$, where $\mathcal{P}$ is the set of all possible distributions. Let us denote the sample size by $n\in\mathbb{N}$. For each $i \in [n] \coloneqq \{1,2,\dots,n\}$, we define a triple $(Y_i, D_i, X_i)$ as an independent and identically distributed (i.i.d.) copy of $(Y, D, X)$. Then, we define our observations as the following dataset $\mathcal{D}$:
For $P\in\mathcal{P}$, let $\mathbb{E}_P[W]$, $\mathrm{Var}_P[W]$, and $\mathbb{P}_P[W]$ be the expectation operator, the variance operator, and the probability law a distribution $P$ of a random variable $W$. For each $d\in\{1, 0\}$ and any $x\in\mathcal{X}$, let us denote the conditional expected outcome of $Y(d)$ by $\mu_P(1)(x) \coloneqq \mathbb{E}_P[Y(d)\mid X = x]$ and the propensity score by $\pi_0(d\mid x) \coloneqq \mathbb{P}_{P_0}[D = d\mid X]$.
In this study, our interest lies in CATEs at $X = x \in \mathcal{X}$ under $P_0$, defined as
This quantity has been widely used in empirical studies of various fields, such as epidemiology, economics, and political science, because it captures heterogeneous treatment effects for each individual represented by a characteristic $x\in\mathcal{X}$.
For identification of $f_0(x)$, we make the following assumptions, called the unconfoundedness and the overlap of assignment support, respectively.
Under the Assumption (ref), assignment $D$ is independent of potential outcomes $(Y(1), Y(0))$ conditioned on covariates. Assuming the unconfoundedness is the standard approach in treatment effect estimation Rosenbaum1983. Assumption (ref) is made to avoid disjoint support between treatment and control groups. These assumptions guarantee the identifiability of the CATE imbens_rubin_2015.
\paragraph{Notation.} Let $[p] \coloneqq \{1,\dots,p\}$. Let us define a ($n\times p$)-matrix $ \bm{X} \coloneqq
=
$, and a $n$-dimensional vector $ \mathbb{D} \coloneqq (D_1,D_2,...,D_n)^\top $.
Let us define $n$-dimensional column vectors ${\mathbb{Y}}$ and ${\mathbb{X}}_j$ for each $j\in[p]$ as $ {\mathbb{Y}} = (Y_1\ Y_2\ \cdots\ Y_n)^\top$, $ {\mathbb{X}}_j = (X_{1,j}\ X_{2,j}\ \cdots\ X_{n,j})^\top_{i\in[n]} $. Here, we can also denote $\bm{X}$ by $ \bm{X} =
$. For a vector $v = (v_1,\dots, v_p)\in\mathbb{R}^p$, let us denote $\ell_q$ norm by $\|v\|_q \coloneqq (\sum^p_{i=1}|v_i|^q)$ for $q \geq 1$. Let $\mathcal{M}$ be the set of measurable functions $\mu:\{1, 0\}\times \mathcal{X}\to \mathbb{R}$, and $\Pi$ be the set of measurable functions $\pi:\{1, 0\}\times \mathcal{X} \to [0, 1]$ such that $\pi(1\mid x) + \pi(0\mid x) = 1$ for any $x\in\mathcal{X}$. Let $\mathrm{Var}(\cdot)$ be the variance operator.
This section introduces high-dimensional linear regression models with sparsity for estimating the CATE.
In this study, under $P\in\mathcal{P}$, we posit the following linear regression models between an individual treatment effect $Y(1) - Y(0)$ and covariates $X$:
where $\bm{\beta} \in \mathbb{R}^p$ is a $p$-dimensional parameter, while $\varepsilon$ is an independent noise variable with a zero mean and finite variance. Let us define $\varepsilon(d) \coloneqq Y(d) - \mu_P(d)(X)$ for each $d\in\{1, 0\}$. In our analysis, we make the following assumptions among $\varepsilon$, $\varepsilon^1$, and $\varepsilon^0$.
Let $\varepsilon_i$ and $\varepsilon_i(d)$ be i.i.d. copies of $\varepsilon$ and $\varepsilon(d)$, and $\bm{\varepsilon}$ and $\bm{\varepsilon}(d)$ be $(\varepsilon_1\ \cdots\ \varepsilon_n)^\top$ and $(\varepsilon(d)_1\ \cdots\ \varepsilon(d)_n)^\top$, respectively.
We also allow for our linear regression models to be high-dimensional; that is, $p > n$. In such a situation, a common approach to obtaining a consistent estimator is to assume that $\bm{\beta}$ has sparsity, that is, most of the elements of $\bm{\beta}$ are zero.
Let $\bm{\beta}_0$ be the true parameter under $P_0$. We denote the active set of regression coefficients by \[\mathcal{S}_0 \coloneqq \{j \in [p] : \beta_{0, j} = 0\},\] where $\beta_{0, j}$ ($j\in[p]$) is $j$-th element of $\bm{\beta}_0$. Let $s_0 \coloneqq |\mathcal{S}_0|$.
This section introduces our proposed estimator, referred to as the Weighted TDL (WTDL) estimator for CATE estimation with Lasso. Our estimator comprises the following components:
Our methodology incorporates DML and the debiased Lasso, leading us to term our approach TDL. The derivation and detailed explanation of the WTDL are presented in Appendix (ref).
\paragraph{Estimation with the true nuisance parameters ($\mu_0$ and $\pi_0$).} As well as the standard DR estimator bang2005drestimation, we first define a function $Q:\mathcal{Y}\times \{1, 0\}, \times \mathcal{X}\times \mathcal{M} \times \Pi \to \mathbb{R}$ as
Here, $Q(Y, D, X; \mu_0, \pi_0)$ works as an unbiased estimator of $Y(1) - Y(0)$. Here, the following linear regression model holds under $P_0$:
where $\overline{\bm{\varepsilon}}$ is an unobservable $n$-dimensional vector whose $i$-th element $\overline{\varepsilon}_i$ is equal to
Note that $\mathbb{E}_{P_0}\left[\overline{\varepsilon}_i \mid X_i\right] = 0$ since
Let us also define
where $\sigma^2_{\varepsilon}(d)(x)$ denotes the variance of $\varepsilon(d)$ conditioned on $x \in \mathcal{X}$.
\paragraph{Estimation with estimated nuisance parameters.} Since we do not know $\mu_0$ and $\pi_0$, we replace them with their estimators, which cause bias in the difference estimator $Q(Y, D, X; \mu_0, \pi_0)$. We employ the cross-fitting in DML to debias the bias caused in the estimation of $\mu_0$ and $\pi_0$. The cross-fitting is a variant of the sample-splitting technique that has been utilized in the semiparametric analysis, such as ..., and refined by ChernozhukovVictor2018Dmlf. In this technique, we first split samples into $m$ subgroups with sample sizes $n_\ell = n / m$. For simplicity, without loss of generality, we assume that $n/m$ is an integer. For each $\ell \in [m]$, we define $\mathcal{D}^\ell \coloneqq \left\{(Y_{\ell, (i)}, D_{\ell, (i)}, X_{\ell, (i)})\right\}^{n_\ell}_{i=1}$ and $\overline{\mathcal{D}}^\ell \coloneqq \mathcal{D} \backslash \mathcal{D}^\ell$, where $(Y_{\ell, (i)}, D_{\ell, (i)}, X_{\ell, (i)})$ denotes an $i$-th element in the $\ell$-th subgroup. Let $\widehat{\mu}_{\ell, n}$ and $\widehat{\pi}_{\ell, n}$ be some estimators constructed only using $\overline{\mathcal{D}}^\ell$, without using $\mathcal{D}^\ell$. The estimators are assumed to satisfy a convergence rate condition described in Assumption (ref). Then, we construct a difference estimator as
where $\ell(i)$ denotes a subgroup $\ell \in [m]$ such that sample $i$ belongs to it. Here, the following linear regression model holds under $P_0$:
where $\widehat{\bm{\varepsilon}}^{\mathrm{DML}}$ is an unobservable $n$-dimensional vector whose $i$-th element $\widehat{{\varepsilon}}^{\mathrm{DML}}_i$ is equal to
In Lemma (ref), we show that for each $i\in[n]$, $\sqrt{n}| \widehat{u}^{\mathrm{DML}}_i - \overline{u}_i \mid = o_P(1)$ holds, which plays an important role for deriving a confidence interval. We refer to this estimator as the DML-DR CATELasso (DML-CATELasso) estimator. Note that this estimator does not have the asymptotic normality due to the bias caused by the use of the Lasso regularization. Furthermore, if we are only interested in consistency, we do not have to apply the cross-fitting.
We later show that the mean of $\widehat{{\varepsilon}}^{\mathrm{DML}}_i$ conditioned on $\bm{X}_i$ converges to zero as $n\to\infty$, and the variance of $\widehat{{\varepsilon}}^{\mathrm{DML}}_i$ conditioned on $\bm{X}_i$ converges to
where $\sigma^2_{\varepsilon}(d)(x)$ denotes the variance of $\varepsilon(d)$ conditioned on $x \in \mathcal{X}$. Due to the dependency of $\sigma^2_{\bar{\varepsilon}}(x)$ on $x\in\mathcal{X}$, we can consider the weighted least squares using $\sigma^2_{\bar{\varepsilon}}(x)$ to reduce the variance of estimators $\bm{\beta}_0$. By using the weighted least sqaures and the Lasso regularization, we estimate $\bm{\beta}_0$ as
where
and $\widehat{\sigma}^2_{\bar{\varepsilon}}$ is a consistent estimator of $\sigma^2_{\bar{\varepsilon}}(x)$. We refer to this estimator as the Weighted DML-DR CATELasso (WDML-CATELasso) estimator. Let us also define
The WDML-CATELasso estimator has a bias caused by Lasso. We then debias the bias using the debiased Lasso.
The estimator $\widehat{\bm{\beta}}^{\mathrm{WDML}}_n$ satisfies the following KKT conditions:
where $\widehat{\bm{\kappa}}^{\mathrm{WDML}} \coloneqq \ensuremath{\operatorname{\mathrm{sign}}}\left(\widehat{\bm{\beta}}^{\mathrm{WDML}}_n\right) = \Big(\ensuremath{\operatorname{\mathrm{sign}}}\left(\widehat{\beta}^{\mathrm{WDML}}_{n, 1}\right),\cdots, \ensuremath{\operatorname{\mathrm{sign}}}\left(\widehat{\beta}^{\mathrm{WDML}}_{n, p}\right)\Big)^\top$, and for each $j\in[p]$,
Let $\widehat{\Sigma} = \bm{X}^\top\bm{X} / n$. Let us also define $\widehat{\Theta}$ as a reasonable approximation for the inverse of $\widehat{\Sigma}$. Then, we consider debiasing the bias caused by the Lasso regularization in the WDML-CATELasso estimator to obtain a confidence interval about $\bm{\beta}_0$. From the KKT condition in (ref), we have
where we used $\widehat{\mathbb{Q}}^{\mathrm{WDML}} = \bm{W}\bm{\beta}_0 + \widehat{\bm{\varepsilon}}^{\mathrm{WDML}}$, and
Then, it holds that
where $\widehat{\Delta}^{\mathrm{WDML}} \coloneqq \sqrt{n}\widehat{\Theta}\widehat{\Sigma} \left( \widehat{\bm{\beta}}^{\mathrm{WDML}}_n - \bm{\beta}_0\right)$, which is the bias term that is expected to approaches zero as $n \to \infty$. Then, we define the Weighted TML (WTML) estimator as
Here, it holds that
\paragraph{Nodewise Lasso regression for constructing $\widehat{\Theta}$.} The remaining problem is the construction of an approximate inverse $\widehat{\Theta}$. Since $\widehat{\Theta}$ is necessary to construct confidence intervals, its construction has been well explored by existing studies, such as vandeGeer2014, Javanmard14a, Cai2017, and Javanmard2018. Our formulation follows the arguments in Meinshausen2006 and vandeGeer2014.
Let $\bm{W}_{(-j)}$ be the design submatrix excluding the $j$-th column. Let us also define $n$-dimensional column vectors ${\mathbb{W}}_j$ for each $j\in[p]$ as ${\mathbb{X}}_j = (X_{1,j}/\widehat{\sigma}_{\bar{\varepsilon}}(\bm{X}_1)\ X_{2,j}/\widehat{\sigma}_{\bar{\varepsilon}}(\bm{X}_2)\ \cdots\ X_{n,j}/\widehat{\sigma}_{\bar{\varepsilon}}(\bm{X}_n))^\top$.
In our construction, for each $j\in[p]$, we carry out the nodewise Lasso regression as follows:
with components of $\widehat{\bm{\gamma}}_j = \{\widehat{\gamma}_{j,k}; k \in [p], k\neq j\}$. Denote by
Let $\ensuremath{\operatorname{\mathrm{sign}}}\left(\widehat{\bm{\gamma}}_j\right) = \left(\ensuremath{\operatorname{\mathrm{sign}}}\left(\widehat{\gamma}_{j, 1}\right),\cdots, \ensuremath{\operatorname{\mathrm{sign}}}\left(\widehat{\gamma}_{j, p}\right)\right)^\top$. Then, we define
where for each $j\in[p]$ and $d\in\{1, 0\}$,
By using $\widehat{C}$ and $\widehat{T}^{-2}$, we construct the approximate inverses $\widehat{\Theta}$ by \[\widehat{\Theta}_{\mathrm{nodewise}} \coloneqq \widehat{T}^{-2}\widehat{C}. \] Finally, we obtain the WTDL estimator with $\widehat{\Theta}_{\mathrm{nodewise}}$ as
We refer to this estimator as the WTDL estimator since we apply the DML and debiased Lasso together in weighted least squares. Here, it holds that
The detailed statement of this results are shown in Lemma (ref) with its proof. The term $\widehat{\Theta} \lambda \widehat{\bm{\kappa}} $ debiases the orignal Lasso estimator. The term comes from the KKT conditions. This term is similar to one in the debiased Lasso in vandeGeer2014 but different from it because we penalizes the difference $\bm{\beta}$. As shown in Section (ref), we can develop the confidence intervals for the debiased CATE Lasso estimator if the inverses $\widehat{\Theta}$ is appropriately approximated.
Lastly, we summarize the above arguments. First, in Section (ref), we assume sparsity directly to CATEs. In Section (ref), we discuss our estimation strategy. In Section (ref), we define the DML-CATELasso estimator. Then, by using the debiased Lasso as in Section (ref), we obtain the confidence interval about $\bm{\beta}_0$ by using an approximated inverse constructed using the node-wise regression. The pseudo-code is shown in Algorithm (ref). Our main estimator is the WTDL estimator. In the following sections, we investigate the theoretical properties of the WTDL estimator.
This section provides several theoretical properties of our WDML-CATELasso and WTDL estimators.
First, we provides the consistency of the WDML estimator $\widehat{\bm{\beta}}^{\mathrm{WDML}}_n$. In our analysis, we a $\beta_{0, j}$ Our analysis starts from the introduction of the compatibility condition Buhlmann2011, which plays an important role in identifiability of $\bm{\beta}_0$. Let us denote the $j$-th element of $\bm{\beta}_0$. For the index set $\mathcal{S}_0 \subset [p]$ by $\beta_{0, j}$. Let us define $\bm{\beta}_{\mathcal{S}_0}$ and $\bm{\beta}_{\mathcal{S}^c_0}$ as vectors whose $j$-th elements are
respectively. Under this definition, we have $\bm{\beta} = \bm{\beta}_{\mathcal{S}_0} + \bm{\beta}_{\mathcal{S}^c_0}$. Then, we define the compatibility condition.
We denote the diagonal elements of a ($p\times p$)-matrix $A$ by $A_j$ for $j\in[p]$. Based on the compatibility condition, we assume the following.
Let $\Sigma$ be the population of $\widehat{\Sigma}$. Then, in the following theorem, we show the consistency of the WDML-CATELasso estimator $\widehat{\bm{\beta}}^{\mathrm{WDML}}$ in a case where the covariates $\bm{W}_i$, treatment assignments $D_i$ are random and $p = p_n \to \infty$ as $n\to \infty$, and $\widetilde{W}$ is sub-Gaussian.
This result implies consistency of $\widehat{\bm{\beta}}$; that is, $\widehat{\bm{\beta}}_n\xrightarrow{\mathrm{p}}\bm{\beta}_0$. The proof is provided in Appendix (ref).
This section provides the asymptotic normality of the TDL estimator $\widehat{\bm{b}}^{\mathrm{TDL}}_n$. Before showing the result, we make the following assumption.
Then, we show the following lemmas. The first lemma is about the fast convergence of the asymptotic bias of the DML-CATELasso estimator. The proof of this lemma is shown in Appendix (ref).
The second lemma is about the form of the TDL estimator. The proof is shown in Appendix (ref).
This result implies that under the raw-sparsity condition of $\bm{X}$, the bias term in the debiased CATE Lasso estimator converges to zero with the $1/\sqrt{n}$ order. Therefore, we can develop the asymptotic distribution with ignoring the bias term.
Then, from the central limit theorem, the confidence interval is given as follows.
This theorem gives us an asymptotic confidence interval for the WTDL estimator. Thus, we show the consistency and confidence intervals for the WTDL estimator.
Assumption (ref) is difficult to guarantee when the dimension $p$ is large. For instance, it is common to deal with this by imposing sparsity or, in the case of using neural networks, by making assumptions about the structure of the neural network.
However, there are methods to ensure this condition without imposing sparsity. For example, one method is to use the benign-overfitting theory by Bartlett2020. This approach demonstrates that appropriate convergence rates can be obtained even when $p$ is infinite, by using assumptions about the eigen space of the inverse matrix of the covariate $X$'s design matrix. Although confidence intervals are not provided, kato2022benignoverfitting has discussed consistent estimation of CATE through linear regression in a setting where $p$ is infinite and sparsity is not imposed.
We denote $\widehat{\mu}(d)(x)$ and $\widehat{\pi}(d\mid x)$ as some estimators of $\mu_0(d)(x)$ and $\pi(d\mid x)$, respectively, to introduce related work. These estimators can vary depending on the methodology used. For instance, traditional approaches might employ parametric regression or kernel-based nonparametric regression to construct these estimators. In contrast, modern methodologies might utilize random forests or neural networks. The construction method impacts the confidence interval, especially due to the violation of the Donsker condition.
Pioneering studies on CATE estimation include those by Heckman1997 and Heckman2005. Works by LeeWhang2009 and Hsu2017 discuss both estimation and hypothesis testing of CATEs. Studies by Cai2017,Cai2021 focus on confidence intervals in high-dimensional contexts. Abrevaya2015 addresses the nonparametric identification of CATEs and introduces a Nadaraya-Watson-based estimator.
The advent of machine learning algorithms has led to various proposed methods for estimating CATEs. Kunzel2019 summarize some of these methods as meta-learners, as shown in the subsequent section.
Furthermore, the literature on high-dimensional linear regression deserves mention. Beyond the Lasso estimator, numerous high-dimensional linear regression techniques have been proposed, including Ridge Hoerl1970 and Elastic Net ZouHastie2005 under sparsity conditions. High-dimensional regression methods not involving regularization, known as ridgeless estimation or interpolating estimators, have also been developed Bartlett2020. Bartlett2020 introduces the benign-overfitting framework for interpolating estimators, and Tsigler2023 demonstrates benign overfitting in ridge regression. The debiased Lasso was introduced by zhangzhang2014, with extensions by Javanmard14a and vandeGeer2014. Related studies include those by Belloni2014, Belloni2014Post, and Belloni2016, with Belloni2014 considering treatment effect estimation in a similar vein to our work.
In the literature of semiparametric analysis, we can consider an estimator including nuisance parameters estimated by some machine learning algorithms. If the nuisance estimators do not satisfy the Donsker condition, the estimator of the parameter of interest can be potentially biased. To debias the estimator, ChernozhukovVictor2018Dmlf proposes DML, which is a refinement of the sample-splitting methods used in early works such as klaassen1987 and ZhengWenjing2011CTME. Fan2022 proposes Lasso with DML and multiplier-bootstrap to obtain a confidence interval. See also Chernozhukov2024book.
Additionally, numerous methods have also been proposed for CATE estimation Li2017,Kallus2017,Powers2017,Subbaswamy2018,Zhao2019,Nie2020,Hahn2020,sugasawa2023bayesian,lan2023causal,kato2023cate,Jin2023.
This study delved into th Lasso estimation of CATEs with sparsity, we introduced a three-stage estimation methodology: (i) estimating nuisance parameters, which include conditional expected outcomes and propensity scores; (ii) estimating the difference in outcomes, denoted as $\widehat{\mathbb{Q}}^{\mathrm{DML}}$, through cross-fitting and regressing covariates $\bm{W}$ on this difference using Lasso regularization; and (iii) applying the debiased Lasso technique to mitigate the bias introduced by the Lasso regularization. The estimator derived from this approach is termed the TDL estimator. We demonstrated the consistency and asymptotic normality of the TDL estimator and discuss the conditions under which asymptotic normality is achieved.
\onecolumn