Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
53,567 characters · 14 sections · 53 citation commands
Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals with Application to Proximal Causal Inference
Keywords: Doubly Robust Functionals, Minimax Learning, Average Causal Effect, Proximal Causal Inference, Kernel Methods\\
Accepted to the 25th International Conference on Artificial Intelligence and Statistics\\ (AISTATS) 2022.
Suppose independent and identically distributed data from a distribution $P$ over variables $V$ are given and we are interested in estimating a finite-dimensional functional of the distribution $\psi_0\coloneqq\psi(P)$. A common way to do so is to use moment functions to construct estimating equations for the parameter of interest. Such moment functions usually depend on other (possibly infinite-dimensional) functions indexing the distribution called nuisance functions. That is, despite the fact that these functions are not of primary interest, they may be needed to obtain an estimator of the target parameter. The concern is that bias in estimating the required nuisance functions can induce excessive bias for the parameter of interest, thus compromising one's ability for accurate inference about $\psi_0$.
For many applications, one can construct an estimating equation in which the moment function depends on two variation independent nuisance functions. An estimator based on such a moment function is called doubly robust if it is consistent even if one of the nuisance functions is not estimated consistently, provided that the other nuisance function is. That is, double robustness gives the user two chances to estimate the parameter of interest correctly. In addition, in cases where one of the nuisance functions has a slow convergence rate, a parametric ($\sqrt{n}$) convergence rate for the functional of interest can still be obtained if the other nuisance function can be estimated at a fast enough rate tsiatis2007semiparametric. The average causal effect is perhaps the most well-studied functional for which under certain conditions a doubly robust moment function exists. For this functional, under the assumption of no unobserved confounders, the nuisance functions are the outcome regression function and the propensity score hernan2020causal.
robins2001comment and chernozhukov2016locally gave conditions for the existence and constructing doubly robust moment conditions in semiparametric models. A common approach for obtaining such a moment function is based on using the influence function (IF) of the parameter of interest. robins2008higher introduced a large class of doubly robust IFs in which nuisance functions are always in the form of a regression function. However, for some functionals of interest, specially in the field of causal inference, the nuisance functions are solutions to complicated integral equations. A prominent example of this case appears in the recently proposed proximal causal inference framework tchetgen2020introduction, miao2018identifying,cui2020semiparametric. This framework enables estimating the average causal effect when unobserved confounders are present in the system, yet requires the existence of two conditionally independent sets of proxies of the latent confounders that are sufficiently rich to fulfill certain completeness conditions, also referred to as proximal relevance assumption (e.g., using measurements of biomarkers as proxies of patients underlying biological confounding variables). The proximal causal inference framework unifies and connects several existing causal identification and inference frameworks which leverage various types of proxies, such as the instrumental variables, negative controls, synthetic controls, and difference-in-differences. As an example of a nuisance function which is a solution to an integral equation, as we will explain in detail in Section (ref), under the assumptions of the proximal causal inference framework, a nuisance function $h$ can be used for identifying the average causal effect of a treatment variable on an outcome variable, where $h$ is a solution to the integral equation in display (ref).
Motivated by proximal causal inference framework, in this paper, we first extend the IF class of robins2008higher to include doubly robust IFs in which the nuisance functions are solutions to integral equations (Section (ref)). In this case, one cannot simply fit a flexible model for the nuisance functions by solving a regression problem. We show that in this case, the doubly robust moment functions, besides their aforementioned desired properties, can also be used for constructing estimating equations for the nuisance functions (Section (ref)). This enables us to solve the integral equations without resorting to parametric models. The main idea in our nuisance function estimation approach is to estimate each function such that it keeps the expected value of the moment function fixed for the perturbations in the other nuisance function. To implement this idea, we first define a perturbation function which captures the change in the mean of the moment function for a deviation in the nuisance functions. Then we solve a regularized minimax optimization problem which estimates one nuisance function as the function minimizing the perturbation for the worst-case deviation in the other nuisance function. The proposed approach yields unbiased estimating equations for each nuisance function that are free of any nuisance function. Moreover, the use of perturbation function elaborates the connection between nuisance function estimation and reduces the sensitivity/bias in the estimation of the functional of interest.
Framing the problem of estimating the nuisance parameters as an optimization problem enables us to use high-performance non-parametric machine learning tools to design our learners. Here we use reproducing kernel Hilbert spaces (RKHS) (Section (ref)) and present the closed-form solution for the minimax optimization problem for estimating the nuisance functions. We characterize the convergence rate of the nuisance functions based on the recently proposed method of dikkala2020minimax that leverages the localized Rademacher complexity of the class of nuisance functions.
Equipped with high-performance estimators for the nuisance functions, we use the cross-fitting approach bickel1988estimating,van2000asymptotic,robins2008higher,zheng2010asymptotic,chernozhukov2018double to design an estimator for the parameter of interest (Section (ref)). We characterize the measure of ill-posedness of the integral equations that we need to solve, and investigate the complexities caused by ill-posedness of the integral equations and their effect on convergence rate of the parameter of interest. We present the requirements on the convergence rate of the nuisance functions and ill-posedness of the system which guarantees the estimator to be asymptotically linear and root $n$ consistent, i.e., attains parametric convergence rate (Section (ref)). Hence, one can use the influence function of the proposed estimator to obtain confidence intervals for the parameter of interest. In Section (ref), we demonstrate how proximal causal inference framework fits in our estimation setup and evaluate our proposed method by estimating average causal effect in the proximal causal inference framework on synthetic data as well as real-data in Section (ref).
There are few other works on using a doubly robust moment function for estimating the nuisance functions. One approach is to estimate the nuisance functions by minimizing the variance of the doubly robust estimator cao2009improving,tsiatis2011improved,van2010collaborative. Another perspective is focusing on bias reduction rather than variance reduction and our proposed method also falls into this category van2014targeted, vermeulen2015bias, avagyan2017honest,cui2019selective. Especially, vermeulen2015bias proposed the bias reduced doubly robust estimation approach, which locally minimizes the squared first-order asymptotic bias of the doubly robust estimator in the direction of the nuisance parameters under misspecification of both working models. vermeulen2015bias only consider parametric working models for the nuisance functions; a restriction that we avoid in the present work similar to robins2008higher and cui2019selective. In fact, our method can be viewed as a proposal for generalization of that work to the non-parametric setup.
There is a growing attention in the literature to the use of machine learning approaches for estimating causal quantities athey2016approximate,farbmacher2020causal,kallus2018policy,nie2017quasi,shalit2017estimating,wager2018estimation,chernozhukov2019semi,oprescu2019orthogonal,kallus2019localized,dikkala2020minimax,bennett2019deep,hartford2017deep, with a particular recent interest in minimax machine learning methods bennett2019deep,dikkala2020minimax,muandet2019dual,liao2020provably,chernozhukov2020adversarial,kallus2021causal. However, it is important to note that in several of these works such as dikkala2020minimax and bennett2019deep, the target is a dose-response curve. Therefore, the target of estimation is unique. In our work, we use the machine learning tools for estimating functions which are nuisance functions for our parameter of interest, and in general, they do not need to be uniquely identified for the parameter of interest to be uniquely identified.
Finally, regarding the proximal causal inference framework, existing results mostly rely on parametric assumptions for the working models for the nuisance functions, with the exception of singh2020kernelnc, mastouri2021proximal, and the independent, concurrent work kallus2021causal. However, the functional classes, assumptions, and convergence analysis for the parameter of interest in those works are different from ours. Specifically, singh2020kernelnc does not use the influence function for estimating the causal effect. mastouri2021proximal only considers the so-called proximal g-formula approach for estimation, which is the result mentioned in Theorem (ref) of our paper. Because the influence function is not used in that work, the bias will be first order and therefore, larger than the bias in our work which is not only second order but also product bias. That is, the bias in our method is guaranteed to be of smaller order. In fact, while our estimator is root-$n$ consistent (under the assumptions of Theorem (ref)), the estimator in that work will generally fail to be root-$n$ consistent due to slow convergence rate of their nonparametric estimator of the bridge function. kallus2021causal consider the use of a minimax learning method in proximal causal inference framework. However, the functional class, assumptions, and convergence analysis for the parameter of interest in that work is different from ours. That work is primarily focused on average causal effect in proximal causal inference, yet we work with a broad functional class and we focus on the double robustness property to derive the estimators and convergence analysis. The functional class considered in kallus2021causal is a special case of our class of functionals. This can be seen by including the function $\pi$ in that paper in functions $g_1$ and $g_2$ in our functional. Also, we make explicit use of double-robustness or product-bias property in establishing that our estimator is root-$n$ consistent, regular and asymptotically linear. Please see the Supplementary Materials for a detailed description of the related work.
Let $V$ denote the variables from which independent and identically distributed data is collected, and let $\psi_0$ be the finite-dimensional parameter of interest. We consider the class of regular parameters $\psi_0$ for which the influence function (IF) is of the form\footnote{See van2000asymptotic for the definition of regular parameters and their influence functions.}
where $h_0(\cdot)$ and $q_0(\cdot)$ are nuisance functions, $V_q$ and $V_h$ are (not necessarily disjoint) subsets of $V$, and $g_1,g_2,g_3$, and $g_4$ are known measurable functions. The IF can be used as a moment function for estimating $\psi_0$, and with slight abuse of notation, we define the moment function
Parameters with IF of the form (ref) appear in many settings, especially in causal inference and missing data problems. It includes functionals such as average causal effect, expected product of conditional expectations, expected conditional covariance, semiparametric regression, marginal structural mean models, marginal mean under missing at random assumption and certain cases of data missing not-at-random problems, etc. This also applies to the nonparametric instrumental variable problems. For instance, the moment function in chernozhukov2016locallyv2 is in the proposed class with the choice of $\gamma$ as $h$ and $\lambda$ as $q$. This class is an extension of the class introduced in robins2008higher in that we allow the nuisance functions to have different arguments. That is, in the class of robins2008higher, $V_q=V_h$. However, as mentioned in Section (ref), requiring that $V_q=V_h$ will exclude functionals for which the nuisance functions are not solutions to regression problems. Specially, it will exclude functionals in the proximal causal inference framework such as the proximal extensions of the aforementioned examples. Also it will exclude functionals arising in shadow variable approaches to missing data miao2015identification. In li2021identification, noting that $\mathbb{E}[h(X,Z)\mid R=1,X,Y]=\gamma(X,Y)$, the influence function given in Equation (16) is in the proposed class with the choice of $\delta$ as $h$ and $\gamma$ as $q$. Our framework provides a unified nonparametric adversarial estimation framework for all these problems. An important example we will use in this paper is the proximal average causal effect, with the IF given in (ref).
We require the following assumption on the IF, which is satisfied for a large class of doubly robust IFs, including the one for average causal effect, discussed in Section (ref).
As an example, in the proximal causal inference framework that we focus on in Section (ref), one can always choose parameters such that Assumption (ref) is satisfied. In the integral equation for the proximal setup for estimating the average causal effect, two out of three parameters of the function $h$, the conditional distribution of $W$ given $Z,A,X$, and the conditional distribution of $Y$ given $Z,A,X$ can always be chosen to be variational independent. One can choose to pick the first two, which is the requirement of Assumption (ref).
The form of the IF (ref) provides us with the double robustness property, which states that an estimator based on the moment function $IF(V;\psi,q,h)$ is consistent even if we misspecify one of the nuisance functions (but not both). This property is especially crucial in setups with high-dimensional or non-parametric nuisance functions.
We note that the nuisance functions in IF (ref) are not functions of the parameter of interest. Therefore, we can first focus on estimating the nuisance functions. In this subsection, we establish that the doubly robust moment function, besides its desired properties for estimating the parameter of interest, can also be leveraged for constructing estimating equations for the nuisance functions.
We define the perturbation at function pair $(q,h)$ towards the pair $(\dot{q},\dot{h})$ as
Double robustness property in equation (ref) indicates that at the pair $(q_0,h)$ for true function $q_0$ and any function $h$, the expected value of the perturbation function towards any other function $h+\dot{h}$ should be zero. This motivates the following approach for estimating the nuisance functions: We choose $(\hat{q},\hat{h})$ as the true nuisance function pair if it is the solution to the following estimating equations.
Using these equations, and leveraging the fact that the IF is linear in each nuisance function, we propose the following optimization-based estimation approach for estimating the nuisance functions.
Optimization (ref) demonstrates the connection between estimating the nuisance functions and the parameter of interest. Specifically, it shows that finding the nuisance functions indeed requires minimizing the perturbation function, and hence the sensitivity of the parameter of interest to the nuisance functions.
We note that due to Assumption (ref), the optimum value of (ref) can be obtained by solving two separate optimization problems.
In the case that $V_q=V_h$, conditional moment equations (ref) suggest that the nuisance functions can be estimated by solving standard regression problems. However when $V_q\neq V_h$, those moment equations will be inverse problems known as Fredholm integral equation of the first kind.
Unfortunately the optimization problems in Proposition (ref) are not stable. For instance, for the first optimization, if for a choice of $q$, $\mathbb{E}[q(V_q)g_1(V)+g_3(V)\mid V_h]$ is not zero, there exists a choice of $\dot{h}$ that can make $\mathbb{E}\big[\dot{h}(V_h)[q(V_q)g_1(V)+g_3(V)]\big]$ arbitrary large, regardless of how close $q$ is to the ground-truth. That is, the objective function can be made arbitrarily large for sufficiently rich function space for $\dot{h}$ unless $q$ is evaluated at the truth, and this can happen even in the neighborhood of the truth. Since in reality we work with finite data, for any choice of $q$ we get a large value for the objective function depending on the function space and data, and we cannot converge to the correct parameter; leading to miss-specification. Therefore, we add a squared regularization term for robustness against misspecification and stability of the optimization, as well as improving the convergence rates. Hence, we will instead focus on the following regularized optimization problems.
In the following result, we show that the proposed penalized optimizations solve the integral equations (ref). That is, the stability term does not introduce bias.
Proposition (ref) and Theorem (ref) show the connection between perturbing doubly robust functionals and conditional moment function; an observation which to the best of our knowledge is new to this work.
We use cross-fitting estimation approach of chernozhukov2018double for separating the estimation of the nuisance functions from the parameter of interest. This approach provides us with the benefit that weaker smoothness requirements are needed for the nuisance functions. In the cross-fitting approach, we partition the samples into $L$ equal size parts $\{I_1,...,I_L\}$. Consider the finite data version of the estimators from the optimization problems (ref) and (ref), obtained by replacing the population expectation operator $\mathbb{E}[\cdot]$ with the sample average operator $\hat{\mathbb{E}}[\cdot]$. For $\ell\in\{1,...,L\}$, we estimate the nuisance functions $(\hat{q}_\ell,\hat{h}_\ell)$ on data from all parts but $I_\ell$. For all $\ell$, let $\hat{\psi}_\ell$ be the estimation of $\psi_0$ obtained by solving $\frac{1}{|I_\ell|}\sum_{i\in I_\ell}IF(V_i;\hat{\psi}_\ell,\hat{q}_\ell,\hat{h}_\ell)=0$. Our final estimator of $\psi_0$ is obtained by
In this subsection, we study the asymptotic properties of the cross-fitting estimator in display (ref). We require the following regularity conditions for the results.
In addition to the regularity conditions, we require conditions on the convergence of the nuisance functions. In our results, we show that the projected space is the right space to impose the convergence constraints on. Below, we use the conventional notations $\mathbb{P}_n[\cdot]$ and $\text{P}[\cdot]$ to denote empirical and population expectations with respect to the variable $V$. Note that for a function $\hat{f}$, unlike the operator $\mathbb{E}[\hat{f}(V)]$, in $\text{P}[\hat{f}(V)]$, the operator does not take the expectation with respect to the possible randomness in the function $\hat{f}$. For any function $f$, we use the norm notation $\|f\|_2\coloneqq \sqrt{\text{P}[f^2]}$. We need the following definition for our convergence assumption.
$\tau_q(\delta)$ and $\tau_h(\delta)$ are defined as measures of ill-posedness of the conditional expectation operator. This measure was originally proposed in chen2012estimation and is commonly used in the econometrics literature. The intuition behind this measure is as follows: Projecting functions on a space makes them closer (and hence hard to distinguish) in terms of the $L_2$-norm. The measure of ill-posedness relates a certain $L_2$ distance post-projection, to the corresponding largest $L_2$ distance of the functions before projection (i.e. the extent to which projection shrinks the distance). In other words, how much resolution is lost due to the projection. Therefore, a large ill-posedness measure $\tau_h(\delta)$ implies that $V_q$ does not capture much of the information in $V_h$.
Part $(i)$ of Assumption (ref) requires the consistency of the estimators of the nuisance functions. In Section (ref), we provide an RKHS-based estimation framework, under which, for certain choices of the kernel (such as the Gaussian or Sobolev kernels), this assumption is satisfied. Part $(ii)$ of Assumption (ref) requires a certain rate of convergence for the estimated nuisance functions. However, instead of requiring a convergence rate for each nuisance function, we require a rate for their product. Hence, in case that one of them is not converging fast enough the other nuisance function can compensate. This is one of the main attractions of double robust estimators. In Section (ref), we will study the local measure of ill-posedness and convergence rates related to part $(ii)$ for the case that a reproducing kernel Hilbert space is used as the hypothesis class.
We have the following result regarding the convergence of the cross-fitting estimator.
As a corollary of Theorem (ref), one can use the influence function of the proposed estimator to obtain confidence intervals for the parameter of interest.
In this section, we apply the proposed minimax estimation approach to estimating average causal effect from observational data. Let $A$ be a binary treatment variable and $Y$ be the outcome variable. For $a\in\{0,1\}$, let $Y^{(a)}$ be the counterfactual outcome variable representing the outcome if (contrary to the fact) the treatment is set to value $a$. The Average causal effect (ACE) captures the difference in the expected value of the counterfactual outcome variables, that is $\textit{ACE}=\mathbb{E}[Y^{(1)}-Y^{(0)}]$. Therefore, it suffices to focus on the problem of estimating the counterfactual mean of form $\psi_0=\mathbb{E}[Y^{(a)}]$, for $a\in\{0,1\}$. The most popular assumption for studying average causal effect is the so-called conditional exchangeability assumption hernan2020causal. Intuitively, this assumption requires that we have collected enough covariates of the units in the study that conditional on those covariates, the observational data is effectively as good as a conditionally randomized experiment. That is, conditional on the covariates, the units in the treated and untreated groups are exchangeable.
Despite its popularity, the exchangeability assumption is often violated even in laboratory settings as there may be confounders of the treatment and the outcome which are latent. Hence, causal inference approaches are needed that are capable of handling unobserved confounders. Proximal causal inference is one such approach in which, although latent confounders are allowed, presence of certain proxy variables for the confounders is required. Formally, the proximal causal inference framework allows for unobserved confounder variable (possibly vector-valued) $U$, observed confounder $X$, and requires access to proxy variables $Z$ and $W$ which satisfy the following conditions.
Note that Assumption (ref) implies $Y\perp\mkern-9.5mu\perp Z|A,U,X$ and $W\perp\mkern-9.5mu\perp (A,Z)|U,X$, which can be taken as primitive identification conditions if one does not wish to entertain an intervention on $Z$. Figure (ref) depicts an example graphical representation that satisfies these assumptions (the gray variable $U$ is unobserved).
In order to obtain identifiability, miao2018identifying considered the following assumptions.
miao2018identifying established the following nonparametric identification result.
cui2020semiparametric proposed an alternative proximal identification result based on the following counterpart of Assumption (ref).
cui2020semiparametric established the following nonparametric identification result.
Let $V=(X,Z,W,A,Y)$. cui2020semiparametric derived the locally semiparametric efficient influence function for the parameter of interest $\psi_0$ in a nonparametric model in which both confounding bridge functions are unrestricted, at the intersection submodel where both are uniquely identified as
This influence function satisfies our generic doubly robust functional form in expression (ref) with the choice of $g_1(V)=-I(A=a)$, $g_2(V)=I(A=a)Y$, $g_3(V)=1$, and $g_4(V)=0$. Therefore, we can apply Theorem (ref) to obtain the following result.
Note that surprisingly, the propensity score does not appear in the moment equations in Proposition (ref); a fact previously noted in cui2020semiparametric, although motivated from a different perspective.
In this section, we describe solving the minimax problems (ref) and (ref) when an RKHS is used as the hypothesis class. We demonstrate that in this case, fast convergence rates can be obtained and we present a closed-form solution for the optimization problem which renders an easy implementation of the method possible. We further provide results regarding estimating the ill-posedness measure for RKHSes.
Consider the finite data version of the estimators by replacing the population expectation operator $\mathbb{E}[\cdot]$ with sample average operator $\hat{\mathbb{E}}[\cdot]$. We propose the following Tikhonov regularization-based optimizations:
where we assume $\mathcal{H}$ and $\mathcal{Q}$ are RKHSes with kernels $K_{\mathcal{H}}$ and $K_{\mathcal{Q}}$, respectively, equipped with the RKHS norms $\|\cdot\|_{\mathcal{H}}$ and $\|\cdot\|_{\mathcal{Q}}$. Since the form of the two optimization problems are the same, in the following, we only discuss optimization problem (ref). We drop the superscripts $h$ from $\lambda^{h}_{\mathcal{Q}}$ and $\lambda^{h}_{\mathcal{H}}$ to make the notations less cluttered, and denote sample size by $n$.
We start by showing that the optimization problem in Equation (ref) has a closed-form solution. Define $K_{\mathcal{Q},n}=(K_\mathcal{Q}(V_{q_i},V_{q_j}))_{i,j=1}^{n}$ and $K_{\mathcal{H},n}=(K_\mathcal{H}(V_{h_i},V_{h_j}))_{i,j=1}^{n}$ as the empirical kernel matrices corresponding to spaces $\mathcal{Q}$ and $\mathcal{H}$, respectively.
In Section (ref), we observed that the solution to the population level optimization problem (ref) satisfies the conditional moment equation $\mathbb{E}[\hat{h}(V_h)g_1(V)+g_2(V)|V_q]=0$, which is equivalent to $\mathbb{E}\big[g_1(V)\{\hat{h}(V_h)-{h}_0(V_h)\}\big|V_q\big]=0$. Hence, to quantify the performance of an estimator $\hat{h}$, we define the projected risk of the estimator as
We will use the recently proposed approach of dikkala2020minimax for bounding the projected risk of the regularized minimax estimator in (ref). The bound that we provide is based on the critical radii of the involved function classes, which are defined by upper bounding the localized Rademacher complexity of the class. See the Supplementary Materials for the definition of localized Rademacher complexity, critical radius, as well as a computation method when an RKHS is used as the function class.
For any function class $\mathcal{F}$, let $\mathcal{F}_B\coloneqq\{f\in\mathcal{F}:\|f\|^2_{\mathcal{F}}\le B\}$. We require the following conditions on the function classes $\mathcal{H}$ and $\mathcal{Q}$ for the convergence rate results.
In Section (ref), for any given value $\delta>0$, we defined the local measure of ill-posedness. Due to the Tikhonov regularization used in our minimax estimator, assume that there exists a constant $C$, such that the output of the estimator satisfies $\hat{h}^*-h_0\in\mathcal{H}_C$. Therefore, we define $\mathcal{H}_C^{|\delta}\coloneqq\big\{ h\in\mathcal{H}_C:\big\| \textup{P}[h(V_h)|V_q] \big\|_2\le\delta \big\}$, and focus on the measure of ill-posedness $\tau_h(\delta)\coloneqq\sup_{h\in\mathcal{H}_C^{|\delta}}\|h\|_2$.
If $\mathcal{H}$ is an RKHS, $\tau_h(\delta)$ can be bounded using the eigenvalues of the RKHS and a measure of dependency between $V_q$ and $V_h$. This also implies that convergence of the projected risk to zero leads to vanishing of the RMSE, i.e., as required in Assumption (ref), the estimator is consistent. Let $\{\mu_j\}_{j=1}^\infty$ and $\{\phi_j\}_{j=1}^\infty$ be the eigenvalues and eigenfunctions of the RKHS $\mathcal{H}$. For any $m\in\mathbb{N}_+$, let $V_m$ be the $m\times m$ matrix with entry $(i,j)$ defines as $[V_m]_{i,j}=\mathbb{E}\big[ \mathbb{E}[\phi_i(V_h)|V_q]\mathbb{E}[\phi_j(V_h)|V_q] \big]$. Note that the quantity on the right hand side of the equality measures how much the eigenfucntions are smoothened by the conditional expectation operator. For instance, if $V_h=V_q$, as is in the case of ACE under no unmeasured confounders, then $V_m$ will be the identity matrix of size $m$. Similarly, for the proximal setup, if the proxy variables are very informative about the unobserved confounder, $V_h$ and $V_q$ will be highly correlated, and the matrix $V_m$ will be close to identity matrix.
As an example of the application of Lemma (ref), suppose we use an RKHS, such as a Sobolev RKHS, with polynomial eigendecay, i.e., $\mu_m\sim m^{-a}$, and the smoothing effect of the conditional expectation operator is also polynomial, i.e., $\lambda_m\sim m^{-b}$, which implies that $\tau_h(\delta)=O(\delta^{\frac{a}{a+b}})$. By Theorem (ref), we need $\sqrt{n}r_h(n)r_q(n)^{\frac{a}{a+b}}=o(1)$. Therefore, if $r_h(n)=n^{-r_h}$ and $r_q(n)=n^{-r_q}$, we require $\frac{1}{2}<r_h+\frac{ar_q}{a+b}$ for $\sqrt{n}$-convergence of the functional of interest.
{\bf Synthetic-Data Experiments.} We investigate operational characteristics of our framework by applying doubly robust kernel estimators developed in Section (ref) to estimate $h_0$ and $q_0$ in Section (ref) in a range of simulation settings.\footnote{The implementation is publicly available at \href{https://github.com/andrewyyp/Kernel-Doubly-Robust}{https://github.com/andrewyyp/Kernel-Doubly-Robust}.} We generate the data such that the assumptions in Section (ref) hold. We consider sample size $n \in \{800, 1600, 3200\}$ and repeat each simulation for 100 times. The details of the data generating mechanisms as well as additional simulation results considering different data generating mechanisms are provided in the Supplementary Materials. We estimate the average causal effect using 5 folds cross fitting, with 5 folds cross validation to tune the hyperparameters and kernel bandwidths when learning $h_0$ and $q_0$. We then compare the performance of estimation of ACE obtained from the following three approaches: $(i)$ Proximal outcome regression (POR) estimator, which is based on the result in Theorem (ref), $(ii)$ Proximal inverse probability weighting (PIPW) estimator, which is based on the result in Theorem (ref), and $(iii)$ Proximal doubly robust (PDR) estimator, which is based on the IF in display (ref). The results of the comparisons are shown in Figure (ref). All three estimators attain smaller bias as sample size becomes larger. As expected, the PDR estimator outperforms POR and PIPW estimators in all cases.
{\bf Real-Data Analysis.} We reanalyze the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT) with the aim of evaluating the causal effect of right heart catheterization (RHC) during the initial care of critically ill patients in the intensive care unit (ICU) on survival time up to 30 days connors1996effectiveness. The same dataset has been analyzed using proximal framework in tchetgen2020introduction, cui2020semiparametric with parametric estimation on nuisance parameters. Data are available on 5735 individuals, 2184 treated and 3551 controls. In total, 3817 patients survived and 1918 died within 30 days. The outcome Y is the number of days between admission and death or censoring at day 30. We include 11 baseline covariates to adjust for potential confounding. We kept choices of $Z$ and $W$ the same as in tchetgen2020introduction, cui2020semiparametric. Details on baseline covariates, hyperparameters and bandwidths are given in the Supplementary Materials. The results are summarized in Table (ref).
Concordance between the three proximal estimators offers confidence in modeling assumptions, indicating that RHC may have a more harmful effect on 30 day survival among critically ill patients admitted into an intensive care unit than previously reported. In closing, our proximal estimates are well aligned with results obtained in tchetgen2020introduction, cui2020semiparametric.
We have demonstrated that the double-robustness property of a moment function can be used for constructing estimating equations for the nuisance components of the moment function. We framed the idea in terms of a minimax optimization approach, in which we choose a nuisance function such that it minimizes the perturbation of the expected value of the moment function for the worst-case deviation in the other nuisance function. Framing the problem in terms of an optimization, enabled us to use tools from machine learning for implementing the idea and techniques from statistical learning theory to analyze the convergence rates. Specifically, we used kernel methods for designing the learners and Rademacher complexity analysis for obtaining the convergence rates. As is the case with most of nonparametric learners, the approach introduces challenges such as choosing the hyper parameters and having larger time complexity. However, it gives us robustness with respect to model misspecification compared to parametric learners. We provided conditions on the product error of the nuisance functions, as well as the ill-posedness of a conditional expectation operator to obtain the property of asymptotic linearity for the estimator of the parameter of interest. We investigated the application of the developed methodology in estimating the average causal effect in the proximal causal inference framework. The experiment results confirmed the superiority of the double robust learner compared to methods that only use one part of the distribution.