Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
43,350 characters · 21 sections · 67 citation commands
Benign-Overfitting in Conditional Average Treatment Effect Prediction with Linear Regression
The problem of predicting the causal effects of treatment from observations is a central task in various fields, such as economics Wager2018, medicine Assmann2000,Foster2011, and online advertisement Bottou13. The exact treatment effect is a counterfactual value, and it is usually intractable to know it directly. Therefore, we are often interested in the average treatment effect (ATE), which is defined as the difference between the expected potential outcomes of the two treatments Neyman,Rubin1974,imbens_rubin_2015. Furthermore, in the past few years, the growth of large observational data has encouraged the development of predicting the individualized treatment effect, also called the conditional ATE (CATE) hahn1998role,Heckman1997,Abrevaya2015, to allow individuals to have different ATEs; that is, the treatment effect can be heterogeneous among individuals.
CATE prediction is increasing importance in statistics and machine learning Qian2011,Zhao2012,Imai2013,Zhou2017. A naive way to predict the CATE is to estimate the ATE on the subgroups Assmann2000,Cook2004; that is, computing the ATE on each group separated based on the covariates. When the covariates are continuous, it is common to assume some statistical model for CATE, and various methods have been proposed to learn the model Weisberg2015. In particular, in recent years, there has been a lot of interest in how to train models given high-dimensional data and models Belloni2011,Belloni2014,Candes2007,Moon2007,Sun2012,Song2015. Especially, several machine learning studies estimate and predict CATE using large-scale flexible models such as neural networks Johansson2016,Shalit2017,Yao2018,Atan2018,Farrell2020.
Despite the developments, the validity of CATE prediction using large-scale models is still an ongoing issue. When the data are not perfectly observed, as in the CATE problem, the flexible models is more likely to overfit the observations. Kunzel2019 points out the possibility of overfitting of a method for CATE prediction. For ordinary regression problems without no treatment effects, new various theories for overparameterized models have emerged such as the benign overfitting Bartlett2020; that is, a prediction error can be sufficiently small, even though the predictor overfits training data by numerous parameters greater than the sample size. However, it is not clear whether the recent overparameterization theory are applicable to the problems of CATE prediction.
In this study, we investigate CATE prediction with linear regression models whose number of parameter is larger than the number of observations by following Bartlett2020. Specifically, we consider the excess risk of two standard prediction methods: the T-learner Kunzel2019 and inverse probability weight (IPW)-learner with linear regression models. The T-learner separately constructs the interpolating estimator of each treatment effect. The IPW-learner first approximates a response variable of the CATE problem by using treatment assignment probability, called the propensity score, then develops prediction for the approximated response. For both methods, we derive the upper and lower bounds on the excess risk of CATE prediction and then investigate the conditions under which the upper bound goes to zero.
As a result, we find that the design of the treatment assignment plays an important role in benign overfitting. For the T-learner case, when the treatment assignment does not depend on the covariates, which is standard in randomized controlled trials (RCTs), the prediction risk goes to zero under the same conditions as in Bartlett2020. In contrast, when the treatment assignment depends on the covariates owing to the selection bias, the convergence of the risk is not guaranteed. This result is consistent with the previous works Kunzel2019,Nie2020, which claims the danger of overfitting of the T-learner. For the IPW-learner case, on the other hand, the prediction risk converges to zero, regardless of the assignment rule. These results give implications for CATE prediction of overparameterization, and also provide insights into the use of other methods. For example, when using the two-step algorithms, such as doubly robust method ChernozhukovVictor2018Dmlf,Kennedy2020 and R-learner Nie2020, our result implies the importance of correctly estimating IPW in the first stage nuisance parameter estimation instead of using the T-learner.
\paragraph{Related work.} There is a rich literature on the overparameterized setting. The most closest theory to our work is the benign overfitting in linear regression Bartlett2020, which reveals a sufficient condition under which the prediction risk converges to zero with the overparameterized linear model. Subsequent to Bartlett2020, the framework is extended to ridge regression Tsigler2020, multiclass classification wang2021benign, and a max-margin classifier cao2021risk. koehler2021uniform reveals a connection between benign overfitting and the notion of uniform convergence. For other studies studies on overparameterization, numerous works study the precious asymptotics of overparameterized models by using the random matrix theory. Muthukumar2019,hastie2019surprises,dobriban2018high consider linear regression or shallow neural networks, dobriban2018high,Denny2020 study a ridge regression problem, and Chatterji2021 studies a binary classification problem, BelkinHsu20182,BelkinRakhlin2019,Liang2020 study interpolating kernel methods.
CATE prediction has also been proposed using kernel-based methods Fan2008, Gaussian processes Alaa2017,Alaa2018, generative adversarial nets yoon2018ganite, boosting, tree-based methods Zeileis2008,Su2009,ImaiSt2011,Kang2012,Lipkovich2011,loh2012,Wager2018,Athey2019,Chatla2020, nearest neighbor matching, series estimation, and Bayesiaan additive regression trees Hill2011. Gunter2011,ImaiSt2011,Imai2013 formulate the CATE estimation problem as a variable selection problem. Cai2017,Cai2021 study confidence intervals for high-dimensional cases. As a unifying framework, Kunzel2019 introduces meta-learners, such as the T-learner and X-learner. Other various methods have also been proposed Li2017,Kallus2017,Powers2017,Subbaswamy2018,Zhao2019,Hahn2020,Nie2020,Kennedy2020.
\paragraph{Notation.} We define a (potentially infinite-dimensional) Hilbert space $\mathbb{H}$ with a norm $\|\cdot\|$. For two vectors $z,z' \in \mathbbm{H}$, $z^\top z'$ denotes an inner product between $z$ and $z'$, and $z z'^{\top}$ denotes a tensor product. For an operator $\Sigma: \mathbb{H} \to \mathbb{H}$, we use $\mu_1(\Sigma)\ge\mu_2(\Sigma)\ge \cdots$ to denote the eigenvalues of $\Sigma$ in descending order, and we denote the operator norm of $\Sigma$ by $\|\Sigma\|$. We use $I$ to denote the identity operator on $\mathbb{H}$ and $I_n$ to denote the $n\times n$ identity matrix. For an event $E$, $\mathbbm{1}\{E\}$ is an indicator function, which is $1$ if $E$ is true, and $0$ if $E$ is false. For a sequence $\{a_n\}_n$, $O(a_n)$ and $o(a_n)$ denote Landau's big and small o notation, and $O_\mathbb{P}(a_n)$ and $o_\mathbb{P}(a_n)$ denote its probabilistic version. We write $a_n = \Omega (b_n)$ for $\limsup_{n \to \infty} |a_n/b_n| > 0$, and $a_n = \omega (b_n)$ for $\limsup_{n \to \infty} |a_n/b_n| = \infty$ Also, $a_n = \Theta(b_n)$ means that both of $a_n = O(b_n)$ and $a_n = \Omega(b_n)$ hold.
We introduce the notion of conditoinal average treatment effect (CATE). Suppose that there are two treatments $a \in\{1, 0\}$, where treatment $a=1$ corresponds to the active treatment, and treatment $a=0$ corresponds to the control treatment. We have access to $n$ training examples $\{(x_i, d_i, y_{i})\}^n_{i=1}$, each of which consists of a covariate $x_i$ from $\mathbb{H}$, a treatment indicator $d_i\in\{1, 0\}$, and a real-valued response variable $y_i$, and the examples are independent and identical copies of a jointly random element $(x,d,y_1, y_0) \sim P$ from the population distribution $P$. $p(d = a| x )$ denotes the probability that an individual with covariate $x$ receives treatment $a\in\{1,0\}$, which is called the propensity score. We then posit the existence of potential outcome random variables $y_{1}$ and $y_{0}$ corresponding respectively to the response subject would have experienced with and without treatment. By using the potential outcomes, the response variable $y$ can be written as \[y = \mathbbm{1}[d = 1]y_{1} + \mathbbm{1}[d = 0]y_{0}.\] We define the CATE at $x$ as
The main difficulty is that we can only observe a realized one of the two potential outcomes $y_{a}$ if $d=a$ for $a \in \{0,1\}$ from the given training example; thus, the model cannot be directly learned by minimizing the corresponding empirical risk.
In this study, we consider a linear model for the potential outcome variables $y_a, a \in \{0,1\}$, then study the prediction of $\tau^*(x)$. For each $a \in \{0,1\}$, suppose that there exists a true parameter vector $\theta_a^* \in \mathbbm{H}$ such that the variables $x$ and $y_a$ follows the following linear model
where $\varepsilon_a$ is an independent noise variable whose mean is zero and variance is finite. Note that $\varepsilon_0$ and $\varepsilon_1$ are independent to each other. Note that $\theta_a^*$ is the optimal parameter for predicting $y_a$ without the selection, that is, $\mathbb{E}(y_a-x^\top\theta^*_a)^2=\min_{\theta \in \mathbb{H}} \mathbb{E}(y_a-x^\top\theta)^2$ holds.
For each $a\in\{0,1\}$, we give the following notations. We define a {population covariance operator} $\Sigma=\mathbb{E}[x x^\top]$, and a {covariance operator with the selection assignment} $\Sigma_a=\mathbb{E}[\mathbbm{1}[d = a]x x^\top] = \mathbb{E}[p(d=a| x)x x^\top]$. We consider an empirical potential outcome $y_{a,i}$ corresponding $y_i$ for $i=1,...,n$, and also define an empirical vector $\boldsymbol{y}_a\in\mathbb{R}^n$ whose $i$-th entry is $\mathbbm{1}[d_i = a]y_i$, and a vector $\boldsymbol{\varepsilon}_a\in\mathbb{R}^n$ whose $i$-th entry $\varepsilon_{a,i}=\mathbbm{1}[d_i = a]\varepsilon_{a,i}$, where $\varepsilon_{a,i}= (y_{a,i} - (\theta_a^*)^\top x_i)$. Further, following Bartlett2020, we use the infinite matrix notation: $X_a$ denotes a linear map from $\mathbb{H}$ to $\mathbb{R}^n$ corresponding to $(\mathbbm{1}[d_1 = a]x_1^\top,\ldots,\mathbbm{1}[d_n = a]x_n^\top)^\top$, so that $X_a\theta\in\mathbb{R}^n$ has its $i$-th component as $\mathbbm{1}[d_i = a]x_{i}^\top\theta$ for $\theta \in \mathbb{H}$. We use the similar notation for the linear map $X^\top_a$ from $\mathbb{R}^n$ to $\mathbb{H}$. Let us also define the linear map $X=X_1 + X_0$ from $\mathbb{H}$ to $\mathbb{R}^n$; that is, it corresponds to $x_1,\ldots,x_n\in\mathbb{H}$. We also define $X^\top = X_1^\top + X_0^\top$.
Given the observations $\{(x_i, d_i, y_{i})\}^n_{i=1}$, we consider an {\em estimator} which returns parameter ${\theta}\in\mathbb{H}$. Then, we predict the CATE given covariate $x\in\mathbb{H}$ as $x^\top\theta$. We measure the predictive performance of this estimator by using the excess risk.
This risk is used to predict the difference between the two groups to differentiate them from the treatment. In the ordinary regression case, such as the excess risk defined in Bartlett2020, the differences, $y_{1} - y_{0}$ and $\theta^*_1 - \theta^*_0$, are not included.
To study the prediction risk, we need to make regular assumptions.
Assumption (ref) is a common setting in linear regression. For example, Assumption (ref) is satisfied if $x$ and $y_a$ are jointly Gaussian with zero mean and $\ensuremath{\operatorname{\mathrm{rank}}}(\Sigma)>n$ Bartlett2020.
Assumption (ref) expresses a natural setting wherein the assignment is independent of the output conditioned on the covariates. This is the standard approach in treatment effect prediction Rosenbaum1983.
Assumption (ref) allows us to avoid overlap in treatment assignments. The situation where no covariates are selected at all can also be avoided. By this assumption, we can guarantee the identifiability of the assignment and the parameters imbens_rubin_2015.
We can construct specific method to predict the CATE $\tau^*(x)$ with linear models in several ways. In this study, we consider the following two prediction methods: the T-learner and IPW-learner.
In the first stage, we estimate $\theta_a^*$ for each $a \in \{0,1\}$ separately. We consider an interpolating estimator, which can fit the data perfectly when the dimension of $x$ is larger than the sample size $n$; that is an interpolating estimator $\hat{\theta}_a$ satisfies $X_a\hat{\theta}_a=\boldsymbol{y}_a$ when $\mathbb{H} = \mathbb{R}^p $, where $p > n$. Specifically, we define the interpolating estimator for each $a \in \{0,1\}$ as
where $\left(X^\top_a X_a\right)^\dagger$ denotes the pseudoinverse of the bounded linear operator $X^\top_a X_a$ (for infinite-dimensional $\mathbb{H}$, the existence of the pseudoinverse is guaranteed because $X^\top_a X_a$ is bounded and has a closed range; see Desoer1963). When $\mathbb{H}$ has dimension $p$ with $p<n$ and $X_a$ has rank $p$, there is a unique solution to the normal equations. On the contrary, the condition (ref) in Assumption (ref) implies that we can find many solutions $\theta\in\mathbb{H}$ to the normal equations that achieve $X_a\theta=y_a$. Hence, in this case, the minimum norm solution is given by $\hat\theta_a = X^\top_a\left(X_aX^\top_a\right)^{-1}\boldsymbol{y}_a$.
In the second stage, we define a difference of the above estimators as $\hat{\theta}^{\mathrm{T\mathchar`-learner}} := \hat{\theta}_1 - \hat{\theta}_0$. Then, we predict the CATE $\tau^*(x)$ by
This approach of taking the difference between separate estimators from different treatment groups is a commonly considered method in causal inference without overparameterization. This approach is named "T"-learner after the Two separate models used.
We utilize an approach to correct shift of distributions of the observed and population covariates. Suppose that the propensity score $p(d=1| x)$ is known, which follows the setting of Horvitz1952,Austin2011, we define a corrected response variable as
This variable $\hat{y}_i$ has the following property:
Then, we consider a regression problem with the corrected variable $\hat{y}_i$. We define a vector $\hat{\bm{y}} =(\hat{y}_1,...,\hat{y}_n)$. Then, we consider regressing $\hat{\bm{y}}$ on $X = X_1 + X_0$. We define an interpolating estimator using $(x_i, \hat{y}_i)$ as
Then, we predict $\tau^*(x)$ by
To develop an upper bound of the risk of the T-learner, we define a notion of the effective rank to describe the benign overfitting, by following Bartlett2020.
The effective rank is a measure of the complexity of covariances, using a tail of eigenvalues of the covariance matrix. It is used in the analysis of random matrices, such as in Koltchinskii2017, to represent the benign overfitting of ordinary linear regression.
Further, with a given $n \in \mathbb{N}$ and a constant $b > 0$, we define the following number of eigenvalues of the covariance operator $\Sigma$ which has a sufficiently large volume: \[ k^* = k_n^* = \min\left\{k\ge 0: r_k(\Sigma)\ge bn\right\}, \] where the minimum of the empty set is defined as $\infty$. This notation is a continuation of the one used in Bartlett2020. The constant $b$ will be specified in theorems.
Also, with a given $n \in \mathbb{N}, \delta \in (0,1)$ and a covariance operator $\Sigma$, we define the following terms
These terms are used to describe the excess risk. In general, while $\mathcal{B}_{n,\delta}(\Sigma)$ corresponds to a bias, $\mathcal{V}_{n}(\Sigma)$ relates to the variance of the prediction. How the aforementioned describe the error is explained by the following theorem.
We derive our main result on the excess risk of the predictor. At the first step, we derive an upper bound of the risk, which is decomposed into bias and variance related terms. For $a \in \{0,1\}$, we define $P_a = (X_a X^\top_a )^{-1} X_a$, and a projection operator to a complement space of $X_a$ as $\Pi_a^\bot := I - X^\top_a(X_aX^\top_a)^{-1}X_a$.
The terms $2\theta^{*\top}_a B_a \theta^*_a $ and $\bm{\varepsilon}^\top_a C_a \bm{\varepsilon}_a$ mean bias and variance of the error by $\hat{\theta}_a$ for each $a \in \{0,1\}$. In contrast, the rest of the terms $\theta^{*\top}_a D_{a,1-a} \bm \varepsilon_{1-a}$, $\theta^{*\top}_1B_{1,0}\theta^*_0$, and $\bm \varepsilon^\top_1F_{1,0}\bm \varepsilon_0$ denotes the cross effects between $\hat{\theta}_1$ and $\hat{\theta}_0$. These items are specific to our setting.
By using the decomposition and the effective rank and notations, we present the following upper bound on the excess risk in the following theorem:
We obtain the following three implications from this result. (i) Regarding the bias terms, all the covariances $\Sigma, \Sigma_1$, and $\Sigma_2$ affect the upper bound. This means that if, for example, the treatment assignment significantly increases the effective rank of the covariance $\Sigma_1$ for the particular treatment group, it prevents this upper bound from being reduced to zero. (ii) The variance terms depend only on the covariance $\Sigma$ of the population distribution and are unaffected by the covariance of each treatment group. This result contrasts with that of bias. (iii) Importantly, deviations of the covariance $\left\|\Sigma - \zeta^*_a\Sigma_a\right\|$ appear in the bound. This means that if the covariance $\Sigma_a$ of each treatment group $a \in \{0,1\}$ has an eigenspace that is largely different from the covariance $\Sigma$ of the population distribution, a non-negligible error will occur. This item has important implications for predicting treatment effects.
For the ordinary regression without treatment effects, Bartlett2020 shows that an interpolating estimator has the following upper bound on the excess risk:
where $\theta^* \in \mathbb{H}$ is a true parameter for the linear regression model. Our derived upper bound differs from that of Bartlett2020 in several aspects. First, there is a deviation of the covariance, $\left\|\Sigma - \zeta^*_a\Sigma_a\right\|$. This term is due to the change in the distribution by the treatment assignment. Second, because of the cross terms from the different control groups; that is, $\hat{\theta}_1$, $\hat{\theta}_0$, $\epsilon_1$, and $\epsilon_0$, the terms $c\left\|\theta^*_1\right\|\left\|\theta^*_0\right\|\mathcal{B}_{n,\delta}(\Sigma)$ and $c(\left\|\theta^{*}_1\right\| + \left\|\theta^{*}_0\right\|)\log(1/\delta)\sqrt{\mathcal{V}_n(\Sigma)}$ appear. Especially, the first difference is critical because it restricts the problem instances where benign overfitting occurs.
This difference indicates that, unlike the ordinary linear regression, the CATE prediction requires additional conditions in the overparameterization scheme. In the next section, we discuss the conditions for benign overfitting.
We discuss the conditions for the convergence of the derived upper bound to zero, which is referred to as benign overfitting. For $\Sigma$, we consider the following two cases. In the first case, $\Sigma$ is a fixed operator between infinite-dimensional Hilbert spaces. In the second case, $\Sigma$ can change with $n$. For the latter case, we denote $\Sigma$ by $\Sigma_n$ to represent the dependency (resp. $\Sigma_a$ by $\Sigma_{a,n}$ for $a\in\{0,1\}$). Without loss of generality, in the following discussion, we may assume tbat $\|\Sigma\| = 1$. In the latter case, for an ordinary linear regression case without treatment effects, Bartlett2020 reveals that benign overfitting occurs when a sequence of covariance operators $\Sigma_n$ satisfies
Bartlett2020 further analyzes the conditions for the upper bound to go to zero in their Theorem 2. If (ref) holds, both $\mathcal{B}_{n,\delta}(\Sigma)$ and $\mathcal{V}_n(\Sigma)$ in (ref) obviously converge to zero as $\delta \to 1$.
To describe the situation where the benign overfitting occurs, we summarize desirable properties of covariance operators $\Sigma$ as follow. Here, we consider both the case (a) where $\Sigma$ is independent of $n$, and the case (b) where $\Sigma = \Sigma_n$ depends on $n$ and the dimension of $\Sigma_n$ increases with $n$. These properties are developed by Bartlett2020.
This condition concerns the covariance $\Sigma$ of the population distribution, which also evaluates the covariance $\Sigma_a$ of each treatment group $a \in \{0,1\}$.
In CATE prediction, in addition to the abovementioned condition, we need conditions:
This condition is satisfied when the eigenspace of the covariance $\Sigma_a$ of each assigned group coincides with the eigenspace of the covariance $\Sigma$ of the population distribution. In other words, intuitively, the assignment rule must not change the data structure of the observation.
Then, combining Theorem 2 of Bartlett2020 and our new conditions (ref), we summarize the conditions for the benign overfitting in the following theorem.
Except for the conditions (ref), the statements are the identical to Theorem 2 in Bartlett2020. Therefore, in CATE prediction, it is important to determine whether (ref) is satisfied or not. (ref) is more problematic because the term does not usually go to zero even if we put some conditions on the eigenvalues. To consider which cases satisfy (ref), we divide the problem instances into two situations: cases under RCTs and selection bias.
\paragraph{Randomized Control Trial (RCT)}: When we consider RCTs; tht is, when a treatment assignment does not depend on covariates, we can simplify the covariance $\Sigma_a$ for each treatment group simpler. If the variable $d$ for the treatment assignment does not depend on $x$, then for each $a\in\{1,0\}$, we have
Then, for $a \in \{0,1\}$, we have $\min_{\zeta\in\mathbb{R}^+} \|\Sigma - \zeta\Sigma_a\| = \|\Sigma - \frac{1}{p(d=a)}\Sigma_a\| = 0$ holds. This is a situation that standard RCTs satisfy, whereby we randomly assign treatments independently from the covariates. In addition, we also have $r_0(\Sigma_a) = p(d=a) r_0 (\Sigma)$. Hence, the following statement holds without a proof.
Therefore, when predicting the CATE on data collected by RCTs, the excess prediction risk converges to zero under the same conditions for linear regression discussed in Bartlett2020.
\paragraph{Selection Bias}: If there is a selection bias in the sense that the treatment assignment depends on the covariate $x$, the conditions under which benign overfitting occurs are severe. Under selection bias, the covariance $\Sigma_a$ for each treatment group $a \in \{0,1\}$ is given as $\Sigma_a = \mathbb{E}_x[\mathbb{E}[p(d=a|x)x x^\top| x]]$. This disallows us from decomposing $\Sigma_a$ into $p(d=a| x)$ and $\Sigma$. Therefore, we cannot make the deviation $\min_{\zeta\in\mathbb{R}^+} \left\|\Sigma - \zeta\Sigma_a\right\|$ zero.
First, to evaluate the IPW-learner, we define a new excess risk as \[ \tilde{R}\big(\theta\big) : = \mathbb{E}_{x,y}\Big[\big(\hat{y} - x^\top \theta \big)^2 - \big(\hat{y} - x^\top \theta^* \big)^2\Big]. \] Importantly, this new risk is same to the risk given in Definition (ref).
Next, when using the IPW estimator, we confirm that Assumptions 1, 3, and 4 in Definition 1 of Bartlett2020 can be replaced as follows.
Since the statements in Lemma (ref) correspond to Assumptions 1, 3, and 4 in Definition 1 in Bartlett2020, we can directly apply Theorem 1 of Bartlett2020 to obtain the following result by combining them with Assumptions (ref).
Thus, the upper bound has the same form as that of Bartlett2020, although some constant terms are affected by the construction of the IPW estimator. As presented in Bartlett2020, if (ref) is satisfied, the upper bound goes to zero. This means that this upper bound goes to zero under the same condition as that in Bartlett2020.
In this result, unlike the case where we use the T-learner, the upper bound goes to zero for both cases with RCTs and selection bias. This is because the deviation term $\|\Sigma - \zeta^*\Sigma_a\|$ in Theorem (ref) does not appear when using the IPW-learner.
When the model is not overparametrized, both the T-learner and IPW-learner have a risk converges to zero as $n \to \infty$ Abrevaya2015. This fact is contrasive to our results with overparameterization, especially for the case of the T-leaner. This result implies the potential danger of using the T-learner in large-scale models.
The doubly robust estimator Porter2011,Funk2011,Foster2019,Kennedy2020 is another common choice for the CATE prediction. An advantage of the double robust estimator is that an asymptotic variance of the estimator is semiparametric efficient; that is, the asymptotic variance achieves its lower bound hahn1998role,ChernozhukovVictor2018Dmlf. However, some problems appear in the overparameterization setting, when this method estimates an conditional outcome $\mathbb{E}[y_{a,i}| x_i]$ as preparation. Specifically, according to our results, the T-learner is not guaranteed to be valid to estimate the outcome under overparameterization. Even if the IPW-learner is used, we need the correct propensity score. Unless these issues are resolved, it is difficult to utilize the doubly robust estimator with overparameterization.
Our result implies the importance of bias correction methods. When data is collected via RCTs, we recommend the usual method, such as performing regressions for each assigned group like the T-leaner. In contrast, if there is selection bias, we recommend to achieve a correct assigned probability and utilize the IPW-learner.
We investigate the necessary conditions for benign overfitting in CATE prediction using a linear regression model. For predicting the CATE, we consider two methods: the T-learner and IPW learner. When the treatment assignment $d$ does not depend on the covariates, both the T-learner and IPW-learner with interpolating prediction rule show benign-overfitting. However, when the treatment assignment $d_i$ depends on the covariates, the excess risk of the T-learner does not converge to zero, while that of the IPW-learner converges. Thus, this paper shows the situation in which CATE prediction with an overparameterized model works.