Prediction-Powered Causal Inference by Automatic Debiased Machine Learning and Semi-Supervised Riesz Regression

Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.

100,989 characters · 31 sections · 56 citation commands

Rendered from LaTeX for readability, not typeset faithfully. Citation keys are highlighted; maths is left as source; figures, tables and equation environments are summarised rather than reproduced; unrecognised commands are greyed out so nothing is silently dropped. Email addresses are removed.

abstractThis study investigates semiparametric efficient estimation of causal and structural parameters in a semi-supervised setting. In our setting, unlabeled auxiliary regressors are available in addition to labeled observations consisting of outcomes and regressors. Our goal is to construct estimators of causal and structural parameters whose asymptotic variances are smaller than those of estimators constructed using only labeled data. We refer to this framework as prediction-powered causal inference (PPCI). We first derive the efficient influence function and the efficiency bound, which imply that the use of auxiliary regressors can attain a smaller asymptotic variance than the efficiency bound attainable from labeled observations alone. Then, by combining the efficient influence function with the debiased machine learning (DML) framework, we propose methods that we call DML-PPCI. If we construct an estimating-equation estimator, we refer to the method as EE-DML-PPCI; if we construct a targeted-learning estimator, we refer to the method as TMLE-DML-PPCI. The asymptotic variances of both estimators match our derived efficiency bound. In the construction of the estimators, estimation of the efficient influence function plays an important role. In our study, the efficient influence function is also a Neyman orthogonal score, which depends on the Riesz representer and the regression function. For Riesz representer estimation, we develop semi-supervised generalized Riesz regression with convergence rate guarantees.

{\flushleft{{\bf Keywords:} causal inference; semi-supervised learning; prediction-powered inference; semiparametric efficiency; double machine learning; Riesz regression; density-ratio estimation; covariate balancing}}

Introduction

This study investigates efficient causal parameter estimation in a semi-supervised setting. The estimation targets are causal parameters defined via functionals of regression functions, including the average treatment effect (ATE), the average marginal effect (AME), and the average policy effect (APE) as special cases. We aim to construct estimators of these causal parameters with smaller variances by using an auxiliary unlabeled dataset of regressors, in addition to a conventional labeled dataset containing outcomes and regressors. This setup is closely related to the literature on prediction-powered inference Ilker2024predictionpowered, but our goal is to estimate causal parameters. Hence, we refer to this framework as prediction-powered causal inference (PPCI). We show that, by using such an auxiliary unlabeled dataset, we can construct estimators whose asymptotic variances are smaller than those of estimators constructed using only the labeled dataset.

Efficient estimation of causal parameters has been a core interest in causal inference. To discuss efficiency, we consider the asymptotic efficiency bound, also called the semiparametric efficiency bound or the H\'{a}jek--Le Cam bound LeCam1986asymptoticmethods,VanderVaart1998asymptoticstatistics, which characterizes the theoretically best asymptotic variance among regular estimators. We refer to causal parameter estimators whose asymptotic variances attain the efficiency bound as asymptotically efficient estimators. Asymptotically efficient estimators yield accurate estimation of causal parameters in terms of asymptotic mean squared error and tight confidence intervals. Therefore, constructing asymptotically efficient estimators is a standard goal in causal parameter estimation.

In this study, we consider the semi-supervised setup and develop efficiency bounds and asymptotically efficient estimators for this setup. The efficiency bound has been intensively studied in the standard setup with only labeled data, while it has not been fully studied in settings where unlabeled data can be used for general causal and structural parameters represented as functionals of regression functions. In this study, we show that if such unlabeled data are used appropriately, then we can construct estimators whose asymptotic variances are smaller than those of estimators constructed without using unlabeled data.

For example, consider estimating the effect of a new medicine in clinical trials. If population characteristics of patients without outcomes are available in addition to clinical trial data, then these unlabeled patient characteristics provide information about the regressor distribution over which the causal parameter is averaged. This information reduces the regressor-averaging component of the efficiency bound, even though it does not reduce the conditional outcome-noise component. This result shows that efficiency gains can arise from data that contain no outcome information, provided that such data inform the regressor distribution over which the target functional is averaged. Such approaches can be understood as semi-supervised causal inference or prediction-powered inference for causal inference.

In constructing asymptotically efficient estimators, we propose debiased machine learning (DML)-PPCI. DML is a framework for constructing asymptotically efficient estimators through Neyman orthogonal scores. Several approaches can be used in DML and in more general constructions of asymptotically efficient estimators. In this study, among them, we focus on estimating equations and targeted maximum likelihood (TMLE) VanderVaart2002semiparametricstatistics,vanderLaan2011targetedlearning,vanderLaan2006targetedmaximum. We refer to DML-PPCI based on the estimating-equation approach as EE-DML-PPCI, and we refer to DML-PPCI based on TMLE as TMLE-DML-PPCI. For these estimators, we show asymptotic efficiency by proving that their asymptotic variances match our derived efficiency bounds.

Contribution

We summarize our contributions. First, we formulate PPCI under the two-sample and one-sample scenarios within the automatic DML (ADML) framework. Second, we derive the efficient influence functions and asymptotic efficiency bounds in our semi-supervised setup. Third, we construct asymptotically efficient estimators whose asymptotic variances match the derived efficiency bound. Fourth, we develop a semi-supervised version of generalized Riesz regression for estimating the Riesz representer appearing in the Neyman orthogonal score.

\paragraph{Problem formulation.} In the problem formulation, we define our parameters of interest as functionals of regression functions, as in ADML Chernozhukov2022automaticdebiased. This framework covers causal and structural parameters that can be written as functionals of regression functions, including ATE, AME, and APE. In addition, we define the data-generating process (DGP) using the two-sample and one-sample scenarios Niu2016theoreticalcomparisons,Kato2025puate. In the two-sample scenario, we assume that there exist two independent datasets, while in the one-sample scenario, we assume that there exists one dataset and that, from this dataset, we can observe labeled data. The two-sample scenario is closely related to the stratified sampling scheme Wooldridge2001asymptoticproperties,Lancaster1996casecontrolstudies, while the one-sample scenario is closely related to the missing-value literature Rubin1974estimatingcausal,Kennedy2020efficientnonparametric.

\paragraph{Efficiency bound.} For this setup, we derive the efficiency bounds, which can be computed from the efficient influence function. The efficiency bounds depend on the DGP. Therefore, for the two-sample and one-sample scenarios, we derive the corresponding efficiency bounds separately Uehara2020offpolicy. We show that using an auxiliary unlabeled dataset can reduce the asymptotic variance compared with the case where only the labeled dataset is used. The efficiency bound implies the possibility of an efficiency gain, and feasibility is confirmed by constructing estimators whose asymptotic variances match the efficiency bounds.

\paragraph{Asymptotically efficient estimators.} We refer to our estimation method as DML-PPCI. We develop an estimating-equation version, called EE-DML-PPCI, and a targeted maximum likelihood version, called TMLE-DML-PPCI. We show that these estimators have asymptotic variances matching the efficiency bounds derived in this study. Therefore, these estimators are asymptotically efficient Schuler2024introductionmodern.

\paragraph{Semi-supervised generalized Riesz regression.} In DML-PPCI, we need to estimate the efficient influence function. The efficient influence function in our setup depends on two nuisance parameters, the regression function and the Riesz representer. To estimate the Riesz representer, we employ Riesz regression proposed by Chernozhukov2022automaticdebiased or its generalization, generalized Riesz regression, proposed by Kato2026aunified. Note that, under some restrictions on Riesz representer models, Riesz regression is mathematically equivalent to covariate balancing BrunsSmith2025augmentedbalancing,Zhao2019covariatebalancing. We extend generalized Riesz regression to the semi-supervised setup so that an unlabeled dataset is also used to estimate the Riesz representer. This extension can be interpreted as providing an implementable nuisance-parameter estimation method for prediction-powered causal inference. For semi-supervised generalized Riesz regression, we provide convergence rates, which cover finite pseudo-dimension classes and deep ReLU sieves, including the H\"{o}lder-smooth case, the unbounded-support case, and the approximate low-dimensional manifold case. We then translate these Riesz representer rates into sufficient conditions for the product-rate requirement in DML-PPCI.

Related Work

This study is closely related to asymptotic efficiency theory and semi-supervised learning.

Asymptotic efficiency theory aims to construct estimators whose asymptotic variances match the asymptotic efficiency bound, especially in the sense of the semiparametric efficiency bound or the H\'{a}jek--Le Cam asymptotic efficiency bound. Various approaches and techniques have been proposed, such as estimating equations, TMLE, and sample splitting Klaassen1987consistentestimation. The DML framework organizes these approaches by introducing Neyman orthogonal scores and cross-fitting Chernozhukov2018doubledebiased. The ADML framework is a generalization of DML that deals with many causal and structural parameters written as functionals of a regression function and the corresponding Riesz representer. Under the ADML framework, even if we do not know the closed form of the efficient influence function or the Neyman orthogonal score, we can estimate it using Riesz regression Chernozhukov2022automaticdebiased, which is related to semiparametric and sieve Riesz modeling Chen2015sievewald,Chen2015sievesemiparametric and density-ratio estimation Sugiyama2011densityratio.

Semi-supervised learning studies how labeled and unlabeled data can be combined to improve learning Zhu2005semisupervised,Chapelle2006semisupervisedlearning. Covariate-shift methods are a variant of this setup and use information on a target covariate distribution to improve prediction or estimation under distributional changes Shimodaira2000improvingpredictive. Angelopoulos2023predictionpowered proposes prediction-powered inference as a framework for using machine-learning predictions together with gold-standard data to conduct valid inference. This idea has already been extended to causal inference by studies such as Cadei2026predictionpowered and Ilker2024predictionpowered. However, this study differs from these studies in its focus. We derive the semiparametric efficiency bound for causal and structural regression functionals when the auxiliary observations are unlabeled regressors and then construct ADML-type estimators that attain this bound.

Our study is also related to the literature on semi-supervised regression. Azriel2022semisupervised studies best linear approximation under misspecification and shows that unlabeled regressors can improve the asymptotic variance of least-squares-type estimators when nonlinear features of the conditional mean interact with the marginal regressor distribution. Wasserman2007statisticalanalysis analyzes semi-supervised regression through minimax theory and shows that unlabeled data do not automatically improve rates, especially for graph-Laplacian regularization. They emphasize that rate improvements require assumptions that connect the regression function and the regressor distribution. We use this insight to state primitive Riesz representer rates only under explicit smoothness, tail, or low-dimensional-structure conditions.

For estimating the Riesz representer, we employ Riesz regression. Note that there is a duality between Riesz regression and covariate balancing BrunsSmith2022outcomeassumptions,Zhao2019covariatebalancing,Hainmueller2012entropybalancing,Imai2013covariatebalancing,Zubizarreta2015stableweights,Sugiyama2008directimportance, and based on this relationship, Kato2026aunified develops generalized Riesz regression that formulates Riesz representer estimation as Riesz representer fitting under the Bregman divergence, which incorporates various existing methods such as Riesz regression, calibrated estimation Tan2019regularizedcalbrated, tailored loss minimization Zhao2019covariatebalancing, density-ratio estimation, and covariate balancing. In this study, we extend generalized Riesz regression to the semi-supervised setting so that we also utilize unlabeled datasets.

The convergence analysis for semi-supervised generalized Riesz regression is closely related to density-ratio estimation under the Bregman divergence Zheng2022anerror,Kato2021nonnegativebregman,Kato2025rieszregression. Zheng2022anerror establishes non-asymptotic error bounds for density-ratio estimation with deep ReLU feedforward neural networks, including minimax-optimal rates up to logarithmic factors under H\"{o}lder smoothness, extensions to unbounded support, and rates under approximate low-dimensional manifold structure. We use their result to derive error rates for our proposed semi-supervised generalized Riesz regression.

The stratified sampling scheme plays an important role in our theoretical analysis Wooldridge2001asymptoticproperties. When the labeled and unlabeled datasets are independent samples from different strata with fixed sampling proportions, the usual one-sample efficiency bound is not directly applicable. Closely related work includes off-policy evaluation and learning for external validity under covariate shift Uehara2020offpolicy and DML for covariate shift Chernozhukov2025automaticdebiased,Kato2024doubledebiasedcovariateshift.

This study builds on our previous studies Kato2026aunified,Uehara2020offpolicy,Kato2024activeadaptive,Kato2025semisupervised,Kato2024doubledebiasedcovariateshift. In particular, the idea of asymptotic efficiency under stratified sampling is inspired by Uehara2020offpolicy. An early version of PPCI is presented in Kato2024activeadaptive for adaptive experiments. Compared with those studies, we consider a general class of regression-functional causal and structural parameters, whereas those studies focus on specific applications, such as policy evaluation or adaptive experimental design under covariate shift.

Setup

Let $Y \in {\mathbb{R}}$ be an outcome and let $X,{\widetilde{X}}\in{\mathcal{X}}$ be regressors, where ${\mathcal{X}}$ is the regressor space. We observe the outcome $Y$ for $X$, but we do not observe the corresponding outcome for $\widetilde{X}$. Therefore, we refer to $W=(X,Y)$ as labeled data and $\widetilde{X}$ as unlabeled data. In applications, unlabeled data are often test data or evaluation data for which the chosen treatment or policy will be implemented.

Parameter of Interest

Our goal is to estimate a parameter of the form

align[align omitted — 184 chars of source]

where $\gamma_0(x)\coloneqq{\mathbb{E}}_{P_0}\left[Y\mid X=x\right]$ and $m(X,\gamma)$ is a known functional map.

The distribution $V_{0X}$ is the marginal regressor distribution used to evaluate the target parameter. For simplicity, we assume that $V_{0X}$ has the density

align[align omitted — 147 chars of source]

We assume that the constant $\kappa$ is known. It is distinct from the labeled sampling proportion $\rho = n/(n + m)$ introduced in Section (ref).

We define the parameter of interest as a functional of the regression function, which includes various causal parameters as special cases. We give examples below.

example[Examples of parameters of interest] By specifying the parameter functional $m$, we obtain the following estimands. \begin{itemize} • ATE. Let $X=(D,Z)$, where $D\in\{0,1\}$ is a treatment and $Z$ is a vector of covariates. Let $\gamma_0(d,z)={\mathbb{E}}\left[Y\mid D=d,Z=z\right]$. Define \begin{align*} m^{\mathrm{ATE}}(X,\gamma)=\gamma(1,Z)-\gamma(0,Z). \end{align*} Then, the ATE is defined as \[\theta^{\mathrm{ATE}}_0={\mathbb{E}}_{V_{0Z}}\left[\gamma_0(1,Z)-\gamma_0(0,Z)\right].\] • AME. Let $X=(D,Z)$, where $D$ is a continuously distributed treatment or policy variable and $Z$ is a vector of covariates. Suppose that $\gamma(d,z)$ is differentiable in $d$. Define \begin{align*} m^{\mathrm{AME}}(X,\gamma)=\partial_d\gamma(D,Z). \end{align*} Then, the AME is defined as \[\theta^{\mathrm{AME}}_0={\mathbb{E}}_{V_{0X}}\left[\partial_d\gamma_0(D,Z)\right].\] This parameter is the average derivative of the regression function under the evaluation regressor distribution $V_{0X}$. • APE. Let $X=(D,Z)$, where $D$ is a treatment or policy variable and $Z$ is a vector of covariates. Let $\pi_1(d\mid z)$ and $\pi_0(d\mid z)$ be two policy rules, written as conditional probability mass functions or conditional densities with respect to a measure $\nu$. Define \begin{align*} m^{\mathrm{APE}}(X,\gamma) = \int \left(\pi_1(d\mid Z)-\pi_0(d\mid Z)\right)\gamma(d,Z){\mathrm{d}}\nu(d). \end{align*} Then, the APE is defined as \[\theta^{\mathrm{APE}}_0 = {\mathbb{E}}_{V_{0Z}}\left[ \int \left(\pi_1(d\mid Z)-\pi_0(d\mid Z)\right)\gamma_0(d,Z){\mathrm{d}}\nu(d) \right].\] If the policies are deterministic and $D$ is discrete, this reduces to $m^{\mathrm{APE}}(X,\gamma) = \gamma(\pi_1(Z),Z)-\gamma(\pi_0(Z),Z)$, and hence the APE is defined as \[\theta^{\mathrm{APE}}_0 = {\mathbb{E}}_{V_{0Z}}\left[ \gamma_0(\pi_1(Z),Z)-\gamma_0(\pi_0(Z),Z)\right].\] • Covariate shift adaptation. Let $m(X,\gamma)=\gamma(X)$. Then, the target parameter is the mean of the outcome under $V_{0X}$, namely ${\mathbb{E}}_{V_{0X}}\left[Y\right] = {\mathbb{E}}_{V_{0X}}\left[\gamma_0(X)\right]$. \end{itemize}

remark[Relation to the ADML notation] Although ADML is often written using full observed-data notation, this study focuses on functionals of the form ${\mathbb{E}}\left[m(X,\gamma_0)\right]$. This restriction covers the common ADML causal parameters used in our analysis and is important in the present semi-supervised setting because outcomes are not observed for the unlabeled sample. Throughout this study, the plug-in component is evaluated as $m(X,\gamma)$ on labeled regressors and as $m({\widetilde{X}},\gamma)$ on unlabeled regressors. Functionals that directly require outcomes inside the plug-in component require additional nuisance functions and a different efficiency analysis.

Observations

In this study, we consider two different sampling schemes (DGPs): the one-sample scenario and the two-sample scenario.

\paragraph{One-sample scenario.} In this scenario, there is a potential complete dataset

align*[align* omitted — 66 chars of source]

Labeling then occurs. For each $k=1,2,\dots,N$, let $S_k\in\{1,0\}$ be a labeling indicator: if $S_k=1$, then $Y^*_k$ is observed; otherwise, $Y^*_k$ is missing. Therefore, we observe the dataset

align*[align* omitted — 66 chars of source]

where

align*[align* omitted — 50 chars of source]

and $\mathrm{NA}$ denotes a missing value. In this case, the labeled and unlabeled datasets are

align*[align* omitted — 193 chars of source]

whose sample sizes are $n=\sum^N_{k=1}S_k$ and $m=\sum^N_{k=1}(1-S_k)$, respectively.

\paragraph{Two-sample scenario.} We observe the following two independent stratified datasets, which we call the labeled dataset and the unlabeled dataset:

align*[align* omitted — 153 chars of source]

\paragraph{Comparison.} The one-sample scenario treats labeling as a selection process. In contrast, the two-sample scenario considers a setting in which the usual observations $(X_i,Y_i)$ are available and auxiliary observations ${\widetilde{X}}_j$ are independently observed. This difference leads to slightly different approaches to efficiency analysis and different asymptotic regimes. Therefore, we consider the two cases separately.

Goal

Our goal is to estimate the parameter of interest $\theta_0$ efficiently from the observations. Here, efficiency means that the asymptotic variance of an estimator matches the asymptotic lower bound for regular estimators. This lower bound is called the H\'{a}jek--Le Cam asymptotic efficiency bound, and its semiparametric extension is often called the semiparametric efficiency bound.

It is known that if an estimator is regular and asymptotically linear for the efficient influence function, then the estimator is efficient. In many cases, such an estimator is constructed from the efficient influence function by solving an estimating equation or by using TMLE. Therefore, in our methodological and theoretical analysis, we start by deriving the efficiency bound, then propose estimators of $\theta_0$, and then show their asymptotic efficiency.

In the subsequent sections, since the two-sample scenario is closer to the standard setting in semi-supervised learning, we first introduce the efficiency bound and efficient estimators under the two-sample scenario. Then, we introduce the corresponding efficiency bound and efficient estimators under the one-sample scenario.

\paragraph{Notation.} Let $\sigma^2_0(x)\coloneqq\mathrm{Var}(Y\mid X=x)$, where the variance is taken over the conditional distribution $P_0$ given $X=x$. Let $\|f\|_{P,2}\coloneqq \left({\mathbb{E}}_{P_0}\left[f(X)^2\right]\right)^{1/2}$ and $\|f\|_{Q,2}\coloneqq \left({\mathbb{E}}_{Q_{0X}}\left[f\left({\widetilde{X}}\right)^2\right]\right)ight)^2}}^{1/2}$. For applications with discrete treatments, let $e_0(d\mid z)\coloneqq P_0(D=d\mid Z=z)$. Let $r_{0X}(x)\coloneqq v_{0X}(x)/p_{0X}(x)$ be the density ratio between the evaluation and labeled regressor distributions for $X$. When $X=(D,Z)$, let $r_{0Z}(z)\coloneqq v_{0Z}(z)/p_{0Z}(z)$.

Efficiency Bound and Efficient Influence Function in the Two-Sample Scenario

This section derives the semiparametric efficiency bound under the two-sample scenario, where two independent datasets are observed.

Detailed DGP

We define the DGP of the two-sample scenario more precisely. We assume that $W\coloneqq(X,Y)$ follows a distribution $P_0$, and let $P_{0X}$ be the marginal distribution of $X$. We also assume that ${\widetilde{X}}$ follows a distribution $Q_{0X}$. For simplicity, we assume that $P_{0X}$ and $Q_{0X}$ have probability density functions $p_{0X}$ and $q_{0X}$ with respect to a common dominating measure. This assumption can be relaxed by defining the corresponding Radon--Nikodym derivatives with respect to appropriate dominating measures.

\paragraph{DGP.} We observe the following two independent stratified datasets, which we call the labeled dataset and the unlabeled dataset:

align*[align* omitted — 268 chars of source]

where $W_i=(X_i,Y_i)$ is an independent copy of $W=(X,Y)$, and ${\widetilde{X}}_j$ is an independent copy of ${\widetilde{X}}$.

Let $N\coloneqq n+m$. We denote the sampling proportion of the labeled stratum by $\rho\coloneqq n/N\in(0,1)$. In the asymptotic analysis, $\rho$ is fixed by design. Thus, we consider a deterministic sequence of admissible total sample sizes for which $n=\rho N$ and $m=(1-\rho)N$ are integers, and we take the limit $N\to\infty$ along this sequence.

\paragraph{Parameter of interest.} Under this setup, the target parameter can be decomposed as

align*[align* omitted — 134 chars of source]

where

align*[align* omitted — 188 chars of source]

Efficient Influence Function

This section derives the efficient influence function, from which we obtain both the efficiency bound and the efficient estimators. Since one of the key components in the efficient influence function is the Riesz representer, we first introduce the Riesz representer and then derive the efficient influence function.

\paragraph{Riesz representer.} In our parameter of interest, the efficient influence function depends on the Riesz representer, which is obtained by applying the Riesz representation theorem to the parameter functional. Let $\Gamma\subseteq L^2(P_{0X})$ be the regression space under consideration. Assume that the linear functional

align*[align* omitted — 111 chars of source]

is continuous on $\Gamma$ under the $L^2(P_{0X})$ norm. Then, by the Riesz representation theorem, there exists a unique $\alpha_{0,\kappa}\in\overline{\Gamma}\subseteq L^2(P_{0X})$ such that

align[align omitted — 193 chars of source]

This representer generally depends on the evaluation density $v_{0X}$. We can estimate the Riesz representer without obtaining a closed-form expression. For reference, we present representative Riesz representers below for several cases raised in Example (ref).

example[Examples of Riesz representers] Recall that $r_{0Z}(z)\coloneqq v_{0Z}(z)/p_{0Z}(z)$ is the density ratio between the evaluation and labeled covariate distributions for $Z$. \begin{itemize} • ATE. The Riesz representer is given as \begin{align*} \alpha^{\mathrm{ATE}}_{0,\kappa}(D,Z) = r_{0Z}(Z) \left(\frac{\mathbbm{1}[D=1]}{e_0(1\mid Z)}- \frac{\mathbbm{1}[D=0]}{e_0(0\mid Z)}\right), \end{align*} where $e_0(D\mid Z)=P_0(D\mid Z)$ is the propensity score. • AME. Suppose that $X=(D,Z)$, that $D$ is continuously distributed, and that $v_{0X}(d,z)$ is differentiable in $d$. Suppose also that the boundary term from integration by parts is zero. Then, the Riesz representer is given as \begin{align*} \alpha^{\mathrm{AME}}_{0,\kappa}(D,Z) = - \frac{\partial_d v_{0X}(D,Z)}{p_{0X}(D,Z)}. \end{align*} In the special case $V_{0X}=P_{0X}$, this reduces to \begin{align*} \alpha^{\mathrm{AME}}_{0,\kappa}(D,Z) = -\partial_d\log p_{0X}(D,Z), \end{align*} which is the usual Riesz representer for the average derivative Chernozhukov2022automaticdebiased. • APE. If $\pi_1(d\mid z)$ and $\pi_0(d\mid z)$ are two policy rules that are dominated by $e_0(d\mid z)$, then the Riesz representer is given as \begin{align*} \alpha^{\mathrm{APE}}_{0,\kappa}(D,Z) = r_{0Z}(Z) \frac{\pi_1(D\mid Z)-\pi_0(D\mid Z)}{e_0(D\mid Z)}. \end{align*} If the policies are deterministic and $D$ is discrete, this becomes \begin{align*} \alpha^{\mathrm{APE}}_{0,\kappa}(D,Z) = r_{0Z}(Z) \left( \frac{\mathbbm{1}[D=\pi_1(Z)]}{e_0(\pi_1(Z)\mid Z)} - \frac{\mathbbm{1}[D=\pi_0(Z)]}{e_0(\pi_0(Z)\mid Z)} \right). \end{align*} • Covariate shift adaptation. The Riesz representer is given as \begin{align*} \alpha^{\mathrm{CS}}_{0,\kappa}(X) = \frac{v_{0X}(X)}{p_{0X}(X)}. \end{align*} \end{itemize}

\paragraph{Efficient influence function.} Using the Riesz representer, we derive the efficient influence function.

assumption[Regularity for the semi-supervised ADML efficiency bound] The following conditions hold. \begin{enumerate} • The evaluation regressor distribution is dominated by the labeled regressor distribution: $V_{0X}\ll P_{0X}$. In particular, if $\kappa<1$, then $Q_{0X}\ll P_{0X}$. Moreover, $v_{0X}(X)/p_{0X}(X)$ is bounded $P_{0X}$-almost surely. • The regression satisfies $\gamma_0\in\Gamma$, and ${\mathbb{E}}_{P_0}\left[(Y-\gamma_0(X))^2\right]<\infty$. • The map $\gamma\mapsto m(\cdot,\gamma)$ is linear, and $m(\cdot,\gamma_0)\in L^2(P_{0X})\cap L^2(Q_{0X})$. • The Riesz representer $\alpha_{0,\kappa}$ in (ref) exists and satisfies \begin{align*} {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)^2\sigma_0^2(X)\right]<\infty . \end{align*} \end{enumerate}

The following theorem holds. The proof is given in the Appendix.

theorem[Efficient influence functions under the two-sample scenario] Suppose that Assumption (ref) holds. Under the nonparametric model for the labeled distribution $P_0$ and the unlabeled regressor distribution $Q_{0X}$, the efficient influence functions for the labeled and unlabeled strata are \begin{align} \psi^{\mathrm{TS}}_0(W) &\coloneqq \psi^{\mathrm{TS}}\left(W;\gamma_0,\alpha_{0,\kappa},\theta_{P,0}\right) \coloneqq \alpha_{0,\kappa}(X)\left(Y-\gamma_0(X)\right) + \kappa\left(m(X,\gamma_0)-\theta_{P,0}\right), \\ \widetilde{\psi}^{\mathrm{TS}}_0\left({\widetilde{X}}\right) &\coloneqq \widetilde{\psi}^{\mathrm{TS}}\left({\widetilde{X}};\gamma_0,\theta_{Q,0}\right) \coloneqq (1-\kappa)\left(m({\widetilde{X}},\gamma_0)-\theta_{Q,0}\right). \end{align}

Because the data are sampled from two strata, the efficient influence function is a pair of stratum-specific influence functions. The first component corresponds to the labeled stratum $W=(X,Y)\sim P_0$, and the second component corresponds to the unlabeled stratum ${\widetilde{X}}\sim Q_{0X}$.

Both influence functions are centered within their own strata:

align*[align* omitted — 188 chars of source]

Efficiency Bound

The stratum-specific efficient influence functions imply the following efficiency bound.

theorem[Efficiency bound under the two-sample scenario] Suppose that Assumption (ref) holds. Let $N=n+m$, and suppose that the stratified design fixes $n=\rho N$ and $m=(1-\rho)N$ along an admissible deterministic sequence for a constant $\rho\in(0,1)$. Then, the asymptotic variance of any regular estimator of $\theta_0$, under the $\sqrt N$ normalization, is bounded below by \begin{align} V^{\mathrm{TS}}_0(\kappa,\rho) &\coloneqq \frac{1}{\rho} {\mathbb{E}}_{P_0}\left[\psi^{\mathrm{TS}}_0(W)^2\right] + \frac{1}{1-\rho} {\mathbb{E}}_{Q_{0X}}\left[\widetilde{\psi}^{\mathrm{TS}}_0\left({\widetilde{X}}\right)^2\right]. \end{align} Equivalently, we have \begin{align} V^{\mathrm{TS}}_0(\kappa,\rho) &= \frac{1}{\rho} {\mathbb{E}}_{P_0}\left[\alpha_{0,\kappa}(X)^2\sigma_0^2(X)\right] + \frac{\kappa^2}{\rho} \operatorname{Var}_{P_{0X}}\big(m(X,\gamma_0)\big) + \frac{(1-\kappa)^2}{1-\rho} \operatorname{Var}_{Q_{0X}}\big(m({\widetilde{X}},\gamma_0)\big). \end{align}

The first term in (ref) comes from the outcome residual and is governed by the labeled sample because only labeled observations contain $Y$. The second and third terms are regressor averaging components. The second term is associated with the labeled regressor distribution, and the third term is associated with the auxiliary regressor distribution. Thus, unlabeled regressors improve efficiency by reducing the noise in the regressor averaging part of the target, but they do not reduce the outcome residual component.

Asymptotically Efficient Estimators

Based on the derived efficiency bound and efficient influence functions, we construct estimators whose asymptotic variances match the efficiency bound. The estimators are based on the stratum-specific efficient score

align*[align* omitted — 310 chars of source]

We propose two estimator constructions: an estimating-equation estimator and a TMLE estimator. We refer to the overall method as DML-PPCI and specify these two estimators as EE-DML-PPCI and TMLE-DML-PPCI. Both estimators use the same nuisance estimators: the regression function $\gamma_0$ and the Riesz representer $\alpha_{0,\kappa}$. If these nuisance estimators do not satisfy the Donsker condition, then we also apply cross-fitting to control the empirical process terms.

\paragraph{EE-DML-PPCI.} Given estimators $\widehat{\gamma}$ and $\widehat{\alpha}$ of the nuisance parameters $\gamma_0$ and $\alpha_{0,\kappa}$, we construct an estimator as \[ \widehat{\theta} \coloneqq \frac{1}{n}\sum^n_{i=1}\left( \widehat{\alpha}(X_i)\left(Y_i-\widehat{\gamma}(X_i)\right) +\kappa m\big(X_i,\widehat{\gamma}\big) \right)gamma}} } +(1-\kappa)\frac{1}{m}\sum^m_{j=1}m\big({\widetilde{X}}_j,\widehat{\gamma}\big). \] This estimator decomposes as $\widehat{\theta}=\widehat{\theta}_{\mathrm{L}}+\widehat{\theta}_{\mathrm{U}}$, where the labeled and unlabeled parts are

align*[align* omitted — 368 chars of source]

The labeled part $\widehat{\theta}_{\mathrm{L}}$ estimates $\kappa\theta_{P,0}$ through the labeled estimating equation, and the unlabeled part $\widehat{\theta}_{\mathrm{U}}$ estimates $(1-\kappa)\theta_{Q,0}$ from the unlabeled regressors. We also refer to this estimator as the augmented Riesz-weighted (ARW) estimator, which is a generalization of the augmented inverse probability weighting (AIPW) estimator.

remark[Error product property] Under Neyman orthogonality, the influence function often has the property that its bias can be written as the product of the errors of two nuisance parameters. Our derived influence function also has this property: \begin{align} &{\mathbb{E}}_{P_0}\left[\alpha(X)\left(Y-\gamma(X)\right)\right] +{\mathcal{L}}_{0,\kappa}(\gamma)-\theta_0 = {\mathbb{E}}_{P_0}\left[\left(\alpha_{0,\kappa}(X)-\alpha(X)\right)\left(\gamma(X)-\gamma_0(X)\right)\right]. \end{align} Thus, the leading population bias is a second-order product of the regression error and the Riesz representer error, and it is negligible when the product of the two nuisance estimation errors converges sufficiently fast.

\paragraph{TMLE-DML-PPCI.} Next, we propose the TMLE version of DML-PPCI, which is a regression-adjustment estimator with a bias-corrected regression-function estimator. Let $\widehat\gamma$ and $\widehat{\alpha}$ be initial estimators of $\gamma_0$ and $\alpha_{0,\kappa}$. TMLE first updates the initial regression-function estimator $\widehat\gamma$ in the direction of the estimated Riesz representer and then plugs the updated regression-function estimator $\widehat\gamma^{(1)}$ into the semi-supervised target functional.

In our semi-supervised setup, the targeting step is calibrated by the same evaluation distribution that appears in the target functional. For example, under a Gaussian likelihood, we update the initial estimator as

align*[align* omitted — 99 chars of source]

where the fluctuation is given by

align[align omitted — 337 chars of source]

This update follows the same logic as density-ratio-corrected likelihood maximization under covariate shift Shimodaira2000improvingpredictive. The one-dimensional fluctuation is fitted for the regressor law under which the target functional is evaluated. The numerator is computed from the labeled residuals because outcomes are observed only in the labeled sample, and the density-ratio adjustment is already contained in the estimated Riesz representer $\widehat\alpha$. The denominator estimates the target law average of $\widehat\alpha^2$.

Then, we obtain the TMLE-DML-PPCI estimator as the following plug-in estimator:

align[align omitted — 286 chars of source]

This construction is a semi-supervised analogue of Auto-TML in Chernozhukov2022automaticdebiased.

remarkMore generally, we can consider the following link-scale fluctuation based on the likelihood: \begin{align*} \operatorname{link}\left(\widehat\gamma^{(1)}(x)\right) = \operatorname{link}\left(\widehat\gamma(x)\right) +\widehat\varepsilon\widehat\alpha(x), \end{align*} where $\operatorname{link}$ is a link function.

\paragraph{Algorithm.} We summarize the procedures for EE-DML-PPCI and TMLE-DML-PPCI in Algorithm (ref). In the pseudocode below, we use cross-fitting to apply the proposed framework when Donsker-type conditions are not imposed on the nuisance estimators.

algorithm[algorithm omitted — 3,175 chars of source]

In the pseudocode, we split the datasets into $K$ subsets, where $K\ge 2$ is a fixed integer. Partition the labeled indices $[n]$ into $K$ folds ${\mathcal{I}}_1,\ldots,{\mathcal{I}}_K$ and the unlabeled indices $[m]$ into $K$ folds ${\mathcal{J}}_1,\ldots,{\mathcal{J}}_K$. We take the folds to be balanced in the sense that $|{\mathcal{I}}_k|/n=|{\mathcal{J}}_k|/m=1/K+o(1)$. Write

align*[align* omitted — 148 chars of source]

For each fold $k$, estimate the nuisance functions using only observations in ${\mathcal{I}}_{-k}$ and ${\mathcal{J}}_{-k}$. Denote the resulting estimators by

align*[align* omitted — 68 chars of source]

The fitted nuisance functions are then evaluated only on the held-out fold. The residual $Y_i-\widehat\gamma_k(X_i)$ is evaluated only for $i\in {\mathcal{I}}_k$, whereas the plug-in terms $m(X_i,\widehat\gamma_k)$ and $m({\widetilde{X}}_j,\widehat\gamma_k)$ are evaluated on the held-out labeled and unlabeled folds, respectively.

The cross-fitted EE-DML-PPCI estimator is

align[align omitted — 419 chars of source]

The cross-fitted TMLE-DML-PPCI estimator uses the fold-specific fluctuation

align[align omitted — 444 chars of source]

Using the fluctuation, we set $\widehat\gamma_k^{(1)}(x)=\widehat\gamma_k(x)+\widehat\varepsilon_k\widehat\alpha_k(x)$, and obtain

align[align omitted — 335 chars of source]

If the nuisance classes satisfy the required Donsker-type empirical-process conditions, the estimator can be constructed without sample splitting by estimating the nuisance functions on the full sample and evaluating the same score on the full sample.

Nuisance Parameter Estimation

\paragraph{Riesz representer estimation.} We estimate the Riesz representer using semi-supervised generalized Riesz regression, which we propose in this study. See Section (ref) for details. Here, we briefly describe the fold-specific version used for DML-PPCI with cross-fitting. For a candidate regression function $\gamma\in\Gamma$, define the training-sample linear functional

align[align omitted — 313 chars of source]

Given a convex Bregman generator $g$, the fold-specific semi-supervised generalized Riesz regression estimator is

align[align omitted — 243 chars of source]

where ${\mathcal{H}}_n$ is a user-chosen representer class, $Reg_{\alpha}$ is a regularization functional, and

align[align omitted — 360 chars of source]

The first term in (ref) uses labeled regressors because the Riesz inner product is defined under $P_{0X}$. The second term uses both labeled and unlabeled regressors because it estimates the target functional under $V_{0X}=\kappa P_{0X}+(1-\kappa)Q_{0X}$.

\paragraph{Regression function estimation.} The regression function is identified from labeled observations. For each fold $k$, estimate $\gamma_0(x)={\mathbb{E}}\left[Y\mid X=x\right]$ by any estimation method trained on ${\mathcal{I}}_{-k}$. For example, with squared loss,

align[align omitted — 280 chars of source]

where $Reg_\gamma$ is a regularization functional. For binary or bounded outcomes, the squared loss can be replaced by an appropriate negative log-likelihood. The theoretical results for the estimators of $\theta_0$ below do not require a particular regression method. They only require mean-square convergence rates for $\widehat\gamma_k$ and $\widehat\alpha_k$. Unlabeled data can also be incorporated through semi-supervised learning or covariate shift adaptation Shimodaira2000improvingpredictive. We leave the choice of semi-supervised regression method as application-specific, because the appropriate construction depends on the data-generating setting and the target parameter.

Theoretical Properties

For simplicity, we denote both $\widehat\theta^{\text{TS}}_{\mathrm{EE}}$ and $\widehat\theta^{\text{TS}}_{\mathrm{TMLE}}$ by $\widehat{\theta}$. This section establishes the asymptotic properties of estimators $\widehat{\theta}$ of $\theta_0$ constructed by the proposed DML-PPCI, covering EE-DML-PPCI and TMLE-DML-PPCI.

Consistency

We first show the consistency of $\widehat{\theta}$.

assumption[Convergence condition for the regression function estimator] It holds that \begin{align*}\|\widehat\gamma-\gamma_0\|_{P,2} =o_p(1)\quad (N\to\infty).\end{align*}

Note that, since the density ratio $q_{0X}/p_{0X}$ is bounded under Assumption (ref), this assumption also implies $\|\widehat\gamma-\gamma_0\|_{Q,2} =o_p(1)$.

assumption[Convergence condition for the Riesz representer estimator] It holds that \begin{align*}\|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2} =o_p(1)\quad (N\to\infty).\end{align*}

The following theorem holds.

theorem[Consistency of DML-PPCI] Suppose that either Assumption (ref) or Assumption (ref) holds. Then, we have \begin{align*} \widehat\theta \xrightarrow{{\mathrm{p}}} \theta_0 \quad (N\to\infty). \end{align*}

We do not require that both Assumptions (ref) and (ref) hold. Such a property is called double robustness Bang2005doublyrobust. To show asymptotic normality, both Assumptions (ref) and (ref) need to hold.

Asymptotic Properties

In addition to Assumptions (ref) and (ref), we impose the following assumptions.

assumption[Convergence rate condition] It holds that \begin{align*}\|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2}\|\widehat\gamma -\gamma_0\|_{P,2}=o_p(N^{-1/2})\quad (N\to\infty).\end{align*}

assumption[Donsker condition or cross-fitting] The nuisance estimators $\widehat\alpha$ and $\widehat\gamma$ either satisfy the required Donsker-type stochastic equicontinuity conditions or are constructed via cross-fitting. Moreover, the corresponding score components have uniformly bounded $(2+\delta)$-moments for some $\delta>0$.

assumption[Targeting calibration condition] The denominator $D_N$ in (ref) is bounded away from zero, and \begin{align*} L_N-D_N=o_p(1)\quad (N\to\infty), \end{align*} where \begin{align*} D_N \coloneqq \kappa\frac{1}{n}\sum^n_{i=1}\widehat\alpha(X_i)^2 +(1-\kappa)\frac{1}{m}\sum^m_{j=1}\widehat\alpha\big({\widetilde{X}}_j\big)^2, \qquad L_N \coloneqq \frac{\kappa}{n}\sum^n_{i=1} m(X_i,\widehat\alpha) +\frac{1-\kappa}{m}\sum^m_{j=1} m({\widetilde{X}}_j,\widehat\alpha). \end{align*} This condition equates the empirical second moment of $\widehat\alpha$ under the target regressor law with the plug-in derivative of the target functional along $\widehat\alpha$, and it is the condition under which the importance-weighted targeting update reproduces the orthogonal score.

The following theorem holds.

theorem[Asymptotic linearity and efficiency of DML-PPCI] Suppose that Assumptions (ref), (ref), (ref), and (ref) hold. For TMLE-DML-PPCI, suppose in addition that Assumption (ref) holds. Then, we have \begin{align} \widehat\theta-\theta_0 = \frac{1}{n}\sum^n_{i=1}\psi^{\mathrm{TS}}_0(W_i) + \frac{1}{m}\sum^m_{j=1}\widetilde{\psi}^{\mathrm{TS}}_0({\widetilde{X}}_j) +o_p(N^{-1/2})\quad (N\to\infty). \end{align} Consequently, we have \begin{align} \sqrt N\left(\widehat\theta -\theta_0\right) \xrightarrow{d} N\left(0,V^{\mathrm{TS}}_0(\kappa,\rho)\right)\quad (N\to\infty), \end{align} where $V^{\mathrm{TS}}_0(\kappa,\rho)$ is the efficiency bound in Theorem (ref). Thus, the estimators are regular and semiparametrically efficient.

Efficiency Gain

We now compare the semi-supervised bound with the labeled-only bound in the same-population case. Suppose that

align*[align* omitted — 32 chars of source]

Then, $V_{0X}=P_{0X}$ for every $\kappa\in[0,1]$, so changing $\kappa$ does not change the estimand.

Let $\alpha_{0,P}$ denote the Riesz representer for the ordinary labeled-population functional

align*[align* omitted — 75 chars of source]

In this same-population case, $\alpha_{0,\kappa}=\alpha_{0,P}$ and $\theta_{P,0}=\theta_{Q,0}=\theta_0$. Define

align*[align* omitted — 195 chars of source]

Then, we have

align[align omitted — 217 chars of source]

The value of $\kappa$ that minimizes (ref) is

align*[align* omitted — 32 chars of source]

At $\kappa=\rho$, the semi-supervised efficiency bound becomes

align[align omitted — 112 chars of source]

By contrast, an estimator that uses only the $n$ labeled observations has the ordinary efficiency bound $A_0+B_0$ under $\sqrt n$ normalization in the standard setup of causal inference or DML.

Under the common $\sqrt N$ normalization and the design identity $n/N=\rho$, this becomes

align[align omitted — 111 chars of source]

Therefore, we have

align[align omitted — 207 chars of source]

This identity gives the efficiency gain from using the proposed method. Specifically, using unlabeled data reduces the variance stemming from the regressor-averaging component $\operatorname{Var}_{P_{0X}}\left(m(X, \gamma_0)\right)$, while the residual outcome-noise component $A_0$ remains inflated by $1/\rho$ because outcomes are observed only in the labeled stratum.

If $P_{0X}\ne Q_{0X}$, the parameter $\kappa$ is part of the estimand because changing $\kappa$ changes the target regressor distribution. In that case, $\kappa$ should not be chosen by minimizing (ref) unless the scientific target is invariant to $\kappa$.

Semi-Supervised Generalized Riesz Regression

In this section, we develop a semi-supervised version of generalized Riesz regression, building on Kato2025directbias and Kato2026aunified. Riesz regression was proposed by Chernozhukov2022automaticdebiased for estimating the Riesz representer $\alpha_0$. Kato2026aunified generalizes Riesz regression using Bregman divergence minimization, building on related results in Zhao2019covariatebalancing and BrunsSmith2025augmentedbalancing.

Bregman--Riesz Objective

Let ${\mathcal{A}}\subset{\mathbb{R}}$ be an interval and let $g\colon{\mathcal{A}}\to{\mathbb{R}}$ be a differentiable strictly convex function. We write $\partial g$ for its derivative. For a candidate representer $\alpha$, we write $u_\alpha\coloneqq\partial g\circ\alpha$ and $b_\alpha\coloneqq u_\alpha\alpha-g(\alpha)$.

\paragraph{Population objective.} For $a_0,a\in{\mathcal{A}}$, define the Bregman divergence by

align*[align* omitted — 102 chars of source]

By ignoring terms irrelevant to the optimization over $\alpha$, the population semi-supervised Bregman--Riesz objective is

align[align omitted — 351 chars of source]

\paragraph{Empirical objective.} The empirical objective is

align[align omitted — 349 chars of source]

Here, $\widehat{{\mathcal{L}}}_\kappa$ is the empirical evaluation functional

align*[align* omitted — 187 chars of source]

the full-sample analogue of (ref); in this notation $\widehat{{\mathcal{R}}}_{g,\kappa}(\alpha)=\frac{1}{n}\sum^n_{i=1} b_\alpha(X_i)-\widehat{{\mathcal{L}}}_\kappa(u_\alpha)$.

Given a hypothesis class ${\mathcal{H}}_n$, we estimate $\alpha_{0,\kappa}$ by

align[align omitted — 219 chars of source]

where $Reg_{\alpha}$ is some regularization functional.

remark[Bregman--Riesz Identity] The objective in (ref) is a Bregman projection of the true Riesz representer. Indeed, from the Riesz representer theorem, we have ${\mathcal{L}}_{0,\kappa}(u_\alpha) = {\mathbb{E}}_{P_{0X}}\left[\alpha_{0,\kappa}(X)u_\alpha(X)\right]$. Therefore, it holds that \begin{align} {\mathcal{R}}_{g,\kappa}(\alpha)-{\mathcal{R}}_{g,\kappa}(\alpha_{0,\kappa}) = {\mathbb{E}}_{P_{0X}}\left[ BD_g^{\dagger}\big(\alpha_{0,\kappa}(X)\mid \alpha(X)\big) \right]. \end{align} Thus, if $\alpha_{0,\kappa}\in{\mathcal{H}}_n$, the population minimizer of ${\mathcal{R}}_{g,\kappa}$ over ${\mathcal{H}}_n$ is $\alpha_{0,\kappa}$. If $\alpha_{0,\kappa}\notin{\mathcal{H}}_n$, the population minimizer is the Bregman projection of $\alpha_{0,\kappa}$ onto ${\mathcal{H}}_n$.

remark[Dual coordinates] It is often convenient to work in the dual coordinate. Let \begin{align*} g^*(u)\coloneqq \sup_{a\in{\mathcal{A}}}\{au-g(a)\} \end{align*} be the convex conjugate. If $u=\partial g(a)$, then $g^*(u)=au-g(a)$ and $\partial g^*(u)=(\partial g)^{-1}(u)$. Let $f$ be a dual-coordinate function and define \begin{align*} \alpha_f(x) \coloneqq (\partial g)^{-1}\left(f(x)\right). \end{align*} Then (ref) can be written as \begin{align} \widehat{{\mathcal{Q}}}_{g,\kappa}(f) &\coloneqq \frac{1}{n}\sum^n_{i=1} g^*\left(f(X_i)\right) - \widehat{{\mathcal{L}}}_{\kappa}(f), \end{align} up to the reparametrization $\alpha=\alpha_f$. Hence, for a dual class ${\mathcal{F}}_n$, we have \begin{align} \widehat f \in \arg\min_{f\in{\mathcal{F}}_n} \left\{ \widehat{{\mathcal{Q}}}_{g,\kappa}(f)+\lambda_n\Omega(f) \right\}, \end{align} where $\widehat\alpha=\alpha_{\widehat f}$. If ${\mathcal{F}}_n$ is linear in finite-dimensional parameters and $g^*$ is convex, the dual problem is a convex empirical risk minimization problem even when the primal link $\alpha_f=(\partial g)^{-1}\circ f$ is nonlinear.

remark[Reduction to density-ratio estimation] If $m(X,\gamma)=\gamma(X)$, then $\alpha_{0,\kappa}(x)=v_{0X}(x)/p_{0X}(x)$. For $\kappa=0$, this is the density ratio $q_{0X}(x)/p_{0X}(x)$, and (ref) becomes the standard objective of density-ratio estimation under the Bregman divergence. Hence, the present construction strictly generalizes direct density-ratio estimation under covariate shift. Note that density-ratio estimation does not necessarily require the existence of probability densities with respect to the Lebesgue measure. The density notation is a simplification, and the method can also be applied to other cases, such as discrete variables with probability mass functions.

Automatic Regressor Balancing

The Bregman--Riesz objective automatically produces balancing equations when the dual coordinate is linear in a set of regressors Kato2026aunified. For a candidate representer $\alpha$ and a candidate regression function $\gamma$, define the semi-supervised imbalance gap

align[align omitted — 278 chars of source]

The population analogue is

align*[align* omitted — 160 chars of source]

The Riesz representer is characterized by $\Delta_{0,\kappa}(\alpha_{0,\kappa},\gamma)=0$ for all valid $\gamma$.

Let $\phi(x)=(\phi_1(x),\ldots,\phi_p(x))^\top$ be a dictionary and consider the dual-linear model

align[align omitted — 155 chars of source]

For $q\in[1,\infty)$, set $\Omega_q(\beta)=\|\beta\|_q^q/q$. The empirical estimator is

align[align omitted — 373 chars of source]

proposition[Automatic semi-supervised regressor balancing] Assume that (ref) has a minimizer and that the usual first-order optimality conditions hold. Then, for each $j=1,\ldots,p$, there exists $s_j\in\partial\left(|\beta_j|^q/q\right)|_{\beta_j=\widehat\beta_j}$ such that \begin{align} \widehat{\Delta}_\kappa(\widehat\alpha,\phi_j)+\lambda_n s_j=0. \end{align} In particular, if $\lambda_n=0$, then we have \begin{align*} \widehat{\Delta}_\kappa(\widehat\alpha,\phi_j)=0, \qquad j=1,\ldots,p. \end{align*} If $q=1$, then we have \begin{align*} \max_{1\le j\le p} |\widehat{\Delta}_\kappa(\widehat\alpha,\phi_j)| \le \lambda_n. \end{align*} If $q>1$, then we have \begin{align*} |\widehat{\Delta}_\kappa(\widehat\alpha,\phi_j)| = \lambda_n |\widehat\beta_j|^{q-1}, \qquad j=1,\ldots,p. \end{align*}

Proposition (ref) shows that balancing is not an additional constraint imposed after representer estimation. It is the KKT condition of the generalized Riesz regression problem. The functions to be balanced are functions of the full regressor $X$. In treatment-effect applications with $X=(D,Z)$, this means that the dictionary should generally contain treatment-specific functions, such as $D\phi(Z)$ and $(1-D)\phi(Z)$, rather than only common functions of $Z$.

This balancing property directly controls the deterministic component of the estimated Neyman score. Let $\varepsilon_i=Y_i-\gamma_0(X_i)$ and define the empirical semi-supervised Neyman error by

align[align omitted — 321 chars of source]

By linearity of $\widehat{{\mathcal{L}}}_\kappa$, we have

align[align omitted — 276 chars of source]

Under cross-fitting, the first term is conditionally mean zero given the training sample and the held-out regressors. The second term is the deterministic deviation caused by imperfect regressor balance. If

align*[align* omitted — 67 chars of source]

then Proposition (ref) yields the bound

align[align omitted — 243 chars of source]

for the $\ell_1$-penalized case. Thus, the represented part of the score-relevant regression error is automatically balanced, and only regularization slack and approximation error remain.

Choice of the Convex Function

Different choices of the convex function yield different divergences. The following loss functions are examples.

\paragraph{Squared-loss-type Riesz regression.} If we set the convex function to

align*[align* omitted — 52 chars of source]

then we obtain the squared-loss-type objective in Riesz regression, which is identical to the original Riesz regression Chernozhukov2021automaticdebiased,Chernozhukov2022automaticdebiased and least-squares importance fitting (LSIF) in density-ratio estimation Kanamori2009aleastsquares. Note that in this case, the derivative of the convex function is given as $\partial g^{\mathrm{SQ}}(a)=a-C$. With the affine link $\alpha_\beta(x)=C+\phi(x)^\top\beta$, the dual coordinate is linear, and the objective becomes the semi-supervised analogue of least-squares Riesz regression.

\paragraph{UKL (unnormalized KL)-type Riesz regression.} When $\alpha_{0,\kappa}$ is nonnegative, or when a signed representer is decomposed into known-sign branches, one can use the positive-branch unnormalized KL (UKL) objective

align*[align* omitted — 100 chars of source]

The link $\alpha_\beta(x)=\exp\left(\phi(x)^\top\beta\right)$ is compatible with automatic regressor balancing. In addition, this choice yields an entropy-balancing or KLIEP-type objective as its dual.

\paragraph{Other loss functions.} We can also use other convex functions in the Bregman divergence with the automatic regressor balancing property. Binary KL-type (BKL) losses connect to logistic likelihoods, Basu-power losses interpolate between squared-loss and KL-type behavior, and PU (positive and unlabeled)-type losses are useful when the representer is bounded in an interval.

Candidate Hypothesis Classes

Several hypothesis classes can be used for $\alpha_{0,\kappa}$.

\paragraph{Series and sieve classes.} A basic choice is the finite-dimensional dual-linear class

align*[align* omitted — 124 chars of source]

where $\phi_n$ contains splines, polynomials, interactions, or other researcher-chosen features. In causal applications with $X=(D,Z)$, treatment-specific dictionaries are recommended because the score-relevant regression error is generally a function of the full regressor.

\paragraph{RKHS and random-feature classes.} Kernel methods can be used by taking ${\mathcal{F}}_n$ to be an RKHS ball or a finite-dimensional approximation obtained by random Fourier features or Nystr{\"o}m features. This gives functional balance over a rich class without manually specifying every interaction.

\paragraph{Tree, forest, matching, and neural classes.} Tree leaf indicators and random-forest leaf encodings produce sparse dictionaries that adapt to heterogeneous regions of the regressor space. Nearest-neighbor catchment bases give matching-type representer estimators under squared loss Kato2025nearestneighbor. Neural networks can be used either as frozen embeddings followed by a convex dual-linear Riesz fit or as a fully end-to-end nonconvex class.

remark[Non-negative correction for flexible positive-branch models] As pointed out in the density-ratio literature Kato2021nonnegativebregman,Rhodes2020telescopingdensityratio, if we use flexible models for $\alpha_{0,\kappa}$, the optimization of the Bregman objective can suffer from overfitting. If $\alpha_{0,\kappa}$ is positive and bounded by $R$, a non-negative correction can be applied. We can also apply the telescoping technique proposed by Rhodes2020telescopingdensityratio, or its extension based on the infinitesimal classification approach proposed by Choi2022densityratio. For example, Kato2026scorematchingriesz applies the infinitesimal classification approach to Riesz representer estimation without an unlabeled dataset.

Error Analysis

We now provide error bounds for the semi-supervised generalized Riesz regression estimator. The analysis follows the same structure as error analyses for density-ratio estimation and Riesz regression. First, the population objective is identified with a Bregman divergence. Second, the empirical process is decomposed into labeled and unlabeled parts. Third, strong convexity converts excess Bregman risk into $L^2(P_{0X})$ error. The first result is a general oracle inequality. Later results specialize this inequality to finite pseudo-dimension classes and deep ReLU sieves.

For $\alpha\in{\mathcal{H}}_n$, define $c_\alpha(x)\coloneqq m(x,u_\alpha)$ and $a_\alpha(x)\coloneqq b_\alpha(x)-\kappa c_\alpha(x)$. Also define ${\mathcal{A}}_n\coloneqq \{a_\alpha\colon\alpha\in{\mathcal{H}}_n\}$ and ${\mathcal{C}}_n\coloneqq \{c_\alpha\colon\alpha\in{\mathcal{H}}_n\}$. For a probability law $R$ and a function class ${\mathcal{F}}$, let

align*[align* omitted — 170 chars of source]

be the Rademacher complexity, where $S_i\sim R$ and $\xi_i$ are independent Rademacher variables. Define

align[align omitted — 314 chars of source]

where $B_A$ and $B_C$ are envelopes for ${\mathcal{A}}_n$ and ${\mathcal{C}}_n$.

assumption[Conditions for Bregman--Riesz error analysis] The following conditions hold. \begin{enumerate} • The generator $g$ is twice continuously differentiable on a compact interval ${\mathcal{A}}$, and there exist constants $0<\underline g\le \overline g<\infty$ such that \begin{align*} \underline g\le \ddot g(a)\le \overline g \qquad for all a\in{\mathcal{A}}. \end{align*} • $\alpha_{0,\kappa}(X)\in{\mathcal{A}}$ almost surely and every $\alpha\in{\mathcal{H}}_n$ takes values in ${\mathcal{A}}$. • The classes ${\mathcal{A}}_n$ and ${\mathcal{C}}_n$ have finite envelopes $B_A$ and $B_C$. • The empirical minimizer $\widehat\alpha$ in (ref) exists. The regularizer satisfies $Reg_\alpha(\alpha)\ge0$. \end{enumerate}

theorem[Oracle inequality for semi-supervised generalized Riesz regression] Suppose that Assumption (ref) holds. Then, with probability at least $1-\delta$, it holds that \begin{align} \frac{\underline g}{2} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 &\le \inf_{\alpha\in{\mathcal{H}}_n} \left\{ {\mathbb{E}}_{P_{0X}}\left[ BD_g^{\dagger}\left(\alpha_{0,\kappa}(X)\mid\alpha(X)\right) \right] +\lambda_n Reg_\alpha(\alpha) \right\} +2\mathfrak E_{n,m}(\delta). \end{align} In particular, if $\alpha_{0,\kappa}\in{\mathcal{H}}_n$, then \begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 &\le \frac{2}{\underline g} \left\{ \lambda_n Reg_\alpha(\alpha_{0,\kappa})+2\mathfrak E_{n,m}(\delta) \right\}. \end{align}

Theorem (ref) is a general distribution-free oracle inequality. It is useful for stability diagnostics and for abstract DML conditions. Under a localized empirical-process condition, the usual fast-rate version is obtained. Let $V_n$ denote a dimension or pseudo-dimension upper bound for the dual class ${\mathcal{F}}_n=\{u_\alpha\colon\alpha\in{\mathcal{H}}_n\}$, and let

align*[align* omitted — 138 chars of source]

corollary[Fast rate under finite pseudo-dimension] In addition to Assumption (ref), suppose that the localized Bernstein condition for the Bregman loss class holds and that the classes generated by ${\mathcal{F}}_n$ have pseudo-dimension bounded by $V_n$. Then, there exists a constant $C$ depending only on the envelope and strong-convexity constants such that, with probability at least $1-\delta$, it holds that \begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 &\le C\inf_{\alpha\in{\mathcal{H}}_n} \|\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 +C\lambda_n Reg_\alpha(\alpha_n^*) +C r_{n,m}^2(\delta), \end{align} where $\alpha_n^*$ is any near-best element in ${\mathcal{H}}_n$. Consequently, \begin{align*} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})} = O_p\left( \inf_{\alpha\in{\mathcal{H}}_n}\|\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})} + \sqrt{V_n\log(n+m)}\left(\frac{1}{n}+\frac{1}{m}\right)^{1/2} + \sqrt{\lambda_n Reg_\alpha(\alpha_n^*)} \right)*)} }. \end{align*}

We next provide primitive rates for deep ReLU sieves. These results make the abstract rate condition in Assumption (ref) verifiable from smoothness assumptions on the dual Riesz oracle. They also show precisely where density-ratio analysis enters the semi-supervised generalized Riesz regression problem.

Let

align*[align* omitted — 108 chars of source]

Let ${\mathcal{F}}_n$ be a class of clipped ReLU feedforward neural networks, and define $\widehat\alpha=(\partial g)^{-1}\circ\widehat f$, where $\widehat f$ minimizes the dual objective in (ref). For $s>0$, let ${\mathcal{H}}^s([0,1]^d,M_f)$ denote the H\"{o}lder ball with smoothness $s$ and radius $M_f$.

assumption[Deep ReLU rate conditions] The following conditions hold. \begin{enumerate} • Assumption (ref) holds, and the dual range of every $f\in{\mathcal{F}}_n$ is contained in a compact interval on which $(\partial g)^{-1}$ is Lipschitz. • The map $f\mapsto m(\cdot,f)$ preserves the empirical-process complexity used in Corollary (ref). In particular, the classes generated by $g^*(f)$, $m(\cdot,f)$, and their centered Bregman-loss differences have pseudo-dimension bounded by a constant multiple of the pseudo-dimension of ${\mathcal{F}}_n$. • The localized Bernstein condition in Corollary (ref) holds for the clipped ReLU loss classes. • The regularization term satisfies $\lambda_nReg_\alpha(\alpha_n^*)=o(1)$, where $\alpha_n^*$ is a near-best element induced by the ReLU approximation to $f_0$. \end{enumerate}

theorem[Deep ReLU rate under H\"{o}lder smoothness] Suppose that Assumption (ref) holds. Suppose also that $X\in[0,1]^d$, ${\widetilde{X}}\in[0,1]^d$, and $f_0\in{\mathcal{H}}^s([0,1]^d,M_f)$. Choose the clipped ReLU architecture so that the H\"{o}lder approximation error and the pseudo-dimension term are balanced. Then, for a finite constant $c$, it holds that \begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 = O_p\left( N_{\min}^{-2s/(d+2s)}\log^c N_{\min} +\lambda_nReg_\alpha(\alpha_n^*) \right). \end{align} If $n=\rho N$ and $m=(1-\rho)N$ for fixed $\rho\in(0,1)$, then we have \begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})} = O_p\left( N^{-s/(d+2s)}\log^{c/2}N +\sqrt{\lambda_nReg_\alpha(\alpha_n^*)} \right). \end{align}

The rate in Theorem (ref) is the semi-supervised analogue of density-ratio estimators based on neural networks. The only structural difference is that the objective contains two empirical processes: the labeled one for the $P_{0X}$ inner product and the unlabeled one for the $Q_{0X}$ part of the target functional. Since $N_{\min}$ controls both empirical processes, the rate is governed by the smaller sample size.

proposition[Density-ratio special case] Suppose that $m(X,\gamma)=\gamma(X)$ and $\kappa=0$. In this case, $\alpha_{0,0}(x)=q_{0X}(x)/p_{0X}(x)$. Let $D_0(x)=\log\left(q_{0X}(x)/p_{0X}(x)\right)$, and suppose that $D_0\in{\mathcal{H}}^s([0,1]^d,M_f)$ and $D_0$ is bounded. Let $\widehat D$ be the Bregman deep ReLU density-ratio estimator and set $\widehat\alpha(x)=\exp\left(\widehat D(x)\right)$. Then, it holds that \begin{align} \|\widehat\alpha-\alpha_{0,0}\|_{L^2(P_{0X})}^2 = O_p\left( N_{\min}^{-2s/(d+2s)}\log^3 N_{\min} \right). \end{align} More generally, for a mean-type target with $\kappa\in(0,1)$, the estimator \begin{align*} \widehat\alpha_{\kappa}(x)=\kappa+(1-\kappa)\exp\left(\widehat D(x)\right) \end{align*} satisfies the same rate up to the multiplicative factor $(1-\kappa)^2$ in the squared error.

Proposition (ref) gives the exact point at which the semi-supervised generalized Riesz regression objective reduces to density-ratio estimation under the Bregman divergence. It justifies the use of theoretical results developed in density-ratio estimation.

theorem[Unbounded-support extension] Suppose that Assumption (ref) holds on ${\mathbb{R}}^d$, and suppose that $f_0\in{\mathcal{H}}^s({\mathbb{R}}^d,M_f)$. Assume that there exist constants $C_1,C_2>0$ such that \begin{align} \max\left\{ P_{0X}(\|X\|_\infty\ge C_1\log N_{\min}), Q_{0X}(\|{\widetilde{X}}\|_\infty\ge C_1\log N_{\min}) \right\} \le C_2N_{\min}^{-2s/(d+2s)}. \end{align} Choose the clipped ReLU architecture as in Theorem (ref) after truncating and rescaling the covariate support. Then, for a finite constant $c'$, it holds that \begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 = O_p\left( N_{\min}^{-2s/(d+2s)}\log^{c'} N_{\min} +\lambda_nReg_\alpha(\alpha_n^*) \right). \end{align} Compared with Theorem (ref), the logarithmic exponent may increase because the approximation is performed on a cube whose side length grows at the order $\log N_{\min}$.

theorem[Approximate low-dimensional manifold extension] Suppose that Assumption (ref) holds. Suppose also that $P_{0X}$ and $Q_{0X}$ are concentrated on a $\varrho$-neighborhood of a compact $d_M$-dimensional Riemannian submanifold of $[0,1]^d$, whose condition number, volume, and geodesic covering regularity satisfy the conditions used in the approximate-manifold analysis of deep ReLU density-ratio estimation. Let $V$, $R$, and $\tau$ denote the corresponding volume, geodesic covering regularity, and condition-number parameters. Let \begin{align*} d_\delta=O\left(d_M\log(dVR\tau^{-1}/\delta)/\delta^2\right) \end{align*} be the Euclidean embedding dimension associated with relative distortion $\delta\in(0,1)$. If $f_0\in{\mathcal{H}}^s([0,1]^d,M_f)$ and the neighborhood radius $\varrho$ is no larger than the ReLU approximation scale, then, for a finite constant $c''$, it holds that \begin{align} \|\widehat\alpha-\alpha_{0,\kappa}\|_{L^2(P_{0X})}^2 = O_p\left( N_{\min}^{-2s/(d_\delta+2s)}\log^{c”}N_{\min} +\lambda_nReg_\alpha(\alpha_n^*) \right). \end{align} Thus, the exponent depends on the embedded dimension $d_\delta$ rather than the ambient dimension $d$.

Theorems (ref), (ref), and (ref) should be interpreted together with the minimax message of Wasserman2007statisticalanalysis. Unlabeled regressors do not automatically improve a nonparametric rate merely because they are available. In the rate analysis above, they have two precise roles. First, they change the target functional and the efficiency bound through $V_{0X}$. Second, they enter the Riesz objective through $\widehat{{\mathcal{L}}}_\kappa$. The rates above require smoothness of the dual Riesz oracle, tail control, or low-dimensional structure. Without such structure, Assumption (ref) is the appropriate abstract condition.

corollary[Primitive sufficient condition for the DML-PPCI product rate] Suppose that $n=\rho N$ and $m=(1-\rho)N$ for fixed $\rho\in(0,1)$. Suppose that $\widehat\alpha$ satisfies \begin{align*} \|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2} = O_p\left(N^{-s_\alpha/(d_\alpha+2s_\alpha)}\log^{c_\alpha}N\right), \end{align*} and $\widehat\gamma$ satisfies \begin{align*} \|\widehat\gamma-\gamma_0\|_{P,2} = O_p\left(N^{-s_\gamma/(d_\gamma+2s_\gamma)}\log^{c_\gamma}N\right). \end{align*} If we have \begin{align} \frac{s_\alpha}{d_\alpha+2s_\alpha} + \frac{s_\gamma}{d_\gamma+2s_\gamma} > \frac{1}{2}, \end{align} then the product-rate condition in Assumption (ref) holds as follows: \begin{align*} \|\widehat\alpha-\alpha_{0,\kappa}\|_{P,2} \|\widehat\gamma-\gamma_0\|_{P,2} = o_p(N^{-1/2}). \end{align*} For the bounded-support ReLU rate, $d_\alpha=d$. For the approximate-manifold rate, $d_\alpha=d_\delta$.

Automatic Neyman Orthogonalization

The balancing property also clarifies the relation between the Riesz-weighted estimator and the ARW estimator. Suppose that exact balance holds for the true regression function,

align*[align* omitted — 69 chars of source]

Then, we have

align[align omitted — 403 chars of source]

Thus, the Riesz-weighted estimator behaves as the infeasible semi-supervised ARW estimator that uses the true regression function.

Efficiency Bound and Efficient Influence Function in the One-Sample Scenario

Next, we consider the one-sample scenario, where we observe one sample and labels are observed with certain probabilities.

Setup

In this scenario, a complete population observation is $(X,Y^*)$, but the outcome is observed only when $S=1$. The observed data are

align*[align* omitted — 68 chars of source]

Let $P_{0X}$ be the marginal distribution of $X$, let $\pi_0(X)\coloneqq\Prb(S=1\mid X)$, and let $\gamma_0(X)\coloneqq{\mathbb{E}}\left[Y^*\mid X\right]$. We assume the missing-at-random condition

align*[align* omitted — 34 chars of source]

and the overlap condition that $\pi_0(X)$ is bounded away from zero. In the one-sample scenario, the full set of regressors is observed from $P_{0X}$. Therefore, the target parameter is

align[align omitted — 131 chars of source]

Efficient Influence Function

Let $\alpha^{\mathrm{OS}}_0$ be the Riesz representer of the one-sample functional, so that

align[align omitted — 212 chars of source]

We derive the efficient influence function. First, we impose the following assumption.

assumption[Regularity for the one-sample efficiency bound] The following conditions hold. \begin{enumerate} • The observed data $O_i=(X_i,S_i,Y_i)$, $i=1,\ldots,N$, are independent copies of $O=(X,S,Y)$, where $Y$ is observed only when $S=1$. Whenever an expression contains $S\left(Y-\gamma(X)\right)$, it denotes the observable product $S\left(Y^*-\gamma(X)\right)$. • The missing-at-random condition $Y^*\perp S\mid X$ holds. • There exists a constant $c_\pi>0$ such that $\pi_0(X)\ge c_\pi$ almost surely. • The regression satisfies $\gamma_0\in\Gamma$, ${\mathbb{E}}\left[(Y^*-\gamma_0(X))^2\right]<\infty$, the map $\gamma\mapsto m(\cdot,\gamma)$ is linear, and $m(\cdot,\gamma_0)\in L^2(P_{0X})$. • The Riesz representer $\alpha^{\mathrm{OS}}_0$ in (ref) exists and satisfies ${\mathbb{E}}\left[\left(\alpha^{\mathrm{OS}}_0(X)\right)^2\sigma_0^2(X)/\pi_0(X)\right]<\infty$. \end{enumerate}

Then, we obtain the following theorem.

theorem[Efficient influence function in the one-sample scenario] Suppose that Assumption (ref) holds. Under the nonparametric observed-data model for $O=(X,S,Y)$, the efficient influence function for $\theta^{\mathrm{OS}}_0$ is \begin{align} \psi^{\mathrm{OS}}_0(O) \coloneqq \frac{S}{\pi_0(X)}\alpha^{\mathrm{OS}}_0(X)\left(Y-\gamma_0(X)\right) + m(X,\gamma_0)-\theta^{\mathrm{OS}}_0. \end{align} Here, the residual term is evaluated only when $S=1$. Equivalently, $S\left(Y-\gamma_0(X)\right)=S\left(Y^*-\gamma_0(X)\right)$.

Efficiency Bound

The efficiency bound is given as follows.

theorem[Efficiency bound in the one-sample scenario] Suppose that Assumption (ref) holds. Then, under $\sqrt N$ normalization, the asymptotic variance of any regular estimator of $\theta^{\mathrm{OS}}_0$ is lower bounded by \begin{align} V^{\mathrm{OS}}_0 &\coloneqq {\mathbb{E}}\left[\psi^{\mathrm{OS}}_0(O)^2\right] = {\mathbb{E}}_{P_{0X}}\left[ \frac{\left(\alpha^{\mathrm{OS}}_0(X)\right)^2\sigma_0^2(X)}{\pi_0(X)} \right] + \operatorname{Var}_{P_{0X}}\left(m(X,\gamma_0)\right), \end{align} where $\sigma_0^2(X)=\mathrm{Var}(Y^*\mid X)$.

The first term in (ref) depends on $1/\pi_0(X)$ because outcomes are observed only for labeled units. The second term does not depend on $1/\pi_0(X)$ because $X$ is observed for every unit.

Efficient Estimators

Based on the efficient influence function in Theorem (ref), we construct one-sample analogues of EE-DML-PPCI and TMLE-DML-PPCI. Let $\widehat\gamma$, $\widehat\alpha$, and $\widehat\pi$ be estimators of $\gamma_0$, $\alpha^{\mathrm{OS}}_0$, and $\pi_0$, respectively. As in the two-sample scenario, if the Donsker condition does not hold for the estimators, we employ cross-fitting to control the empirical process term in the asymptotic analysis.

\paragraph{EE-DML-PPCI.} The one-sample EE-DML-PPCI estimator is

align[align omitted — 335 chars of source]

Here, the product $S_i\left(Y_i-\widehat\gamma(X_i)\right)$ is interpreted as $S_i\left(Y_i^*-\widehat\gamma(X_i)\right)$, which is observed.

\paragraph{TMLE-DML-PPCI.} The one-sample TMLE-DML-PPCI estimator first updates $\widehat\gamma$ in the direction of $\widehat\alpha$. For example, if we consider TMLE under the Gaussian likelihood, we update the initial estimator as

align[align omitted — 168 chars of source]

where $\widehat\varepsilon^{\mathrm{OS}}$ is the perturbation defined as

align[align omitted — 465 chars of source]

Then, the one-sample TMLE-DML-PPCI estimator is given as

align[align omitted — 180 chars of source]

The numerator in (ref) uses labeled residuals corrected by $S_i/\widehat\pi(X_i)$, because outcomes are observed only when $S_i=1$. The denominator estimates the target-law average of $\widehat\alpha^2$ in the one-sample scenario, where the regressor distribution is observed through $\left\{X_i\right\}^N_{i=1}$.

Asymptotic Properties

We first establish consistency and then show asymptotic linearity and efficiency. The assumptions are stated for generic nuisance estimators $\widehat\gamma$, $\widehat\alpha$, and $\widehat\pi$, in the same way as in the two-sample scenario.

assumption[Convergence condition for the regression function estimator] It holds that \begin{align} \|\widehat\gamma-\gamma_0\|_{P,2} = o_p(1) \qquad (N\to\infty). \end{align} Moreover, \begin{align} \|m(\cdot,\widehat\gamma)-m(\cdot,\gamma_0)\|_{P,2} = o_p(1) \qquad (N\to\infty). \end{align}

assumption[Convergence condition for the Riesz representer and labeling probability estimators] It holds that \begin{align} \|\widehat\alpha-\alpha^{\mathrm{OS}}_0\|_{P,2} = o_p(1), \qquad \|\widehat\pi-\pi_0\|_{P,2} = o_p(1) \qquad (N\to\infty). \end{align} Moreover, $\widehat\pi(X)$ and $\pi_0(X)$ are bounded away from zero with probability approaching one, and $\|\widehat\alpha\|_{P,2}=O_p(1)$.

The following theorem holds.

theorem[Consistency of one-sample DML-PPCI] Suppose that Assumption (ref) holds. If either Assumption (ref) holds or Assumption (ref) holds, then \begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} \xrightarrow{{\mathrm{p}}} \theta^{\mathrm{OS}}_0 \qquad (N\to\infty). \end{align}

To show asymptotic normality, both consistency conditions need to hold. In addition, we impose the following rate and empirical-process conditions.

assumption[Convergence rate condition] It holds that \begin{align} \left( \|\widehat\alpha-\alpha^{\mathrm{OS}}_0\|_{P,2} + \|\widehat\pi-\pi_0\|_{P,2} \right) \|\widehat\gamma-\gamma_0\|_{P,2} = o_p(N^{-1/2}) \qquad (N\to\infty). \end{align}

assumption[Donsker condition or cross-fitting] Define \begin{align*} \widehat\varphi^{\mathrm{OS}}(O) &\coloneqq \frac{S}{\widehat\pi(X)} \widehat\alpha(X) \left(Y-\widehat\gamma(X)\right) + m\left(X,\widehat\gamma\right), \\ \varphi^{\mathrm{OS}}_0(O) &\coloneqq \frac{S}{\pi_0(X)} \alpha^{\mathrm{OS}}_0(X) \left(Y-\gamma_0(X)\right) + m\left(X,\gamma_0\right). \end{align*} The nuisance estimators $\widehat\gamma$, $\widehat\alpha$, and $\widehat\pi$ either satisfy the required Donsker-type stochastic equicontinuity conditions or are constructed via cross-fitting so that \begin{align} & \frac{1}{N}\sum^N_{i=1} \left( \widehat\varphi^{\mathrm{OS}}(O_i) - \varphi^{\mathrm{OS}}_0(O_i) \right) - {\mathbb{E}}_{P_0}\left[ \widehat\varphi^{\mathrm{OS}}(O) - \varphi^{\mathrm{OS}}_0(O) \right] = o_p(N^{-1/2}). \end{align} Moreover, the corresponding score components have uniformly bounded $(2+\delta)$-moments for some $\delta>0$.

assumption[Targeting calibration condition] The denominator $\widehat D^{\mathrm{OS}}_N$ in the perturbation is bounded away from zero, and \begin{align} \widehat L^{\mathrm{OS}}_N - \widehat D^{\mathrm{OS}}_N = o_p(1) \qquad (N\to\infty), \end{align} where \begin{align} \widehat L^{\mathrm{OS}}_N \coloneqq \frac{1}{N}\sum^N_{i=1} m\left(X_i,\widehat\alpha\right). \end{align} Moreover, $\widehat A^{\mathrm{OS}}_N=O_p(N^{-1/2})$.

The following theorem holds.

theorem[Asymptotic linearity and efficiency of one-sample DML-PPCI] Suppose that Assumptions (ref), (ref), (ref), (ref), and (ref) hold. Then, we have \begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} - \theta^{\mathrm{OS}}_0 = \frac{1}{N}\sum^N_{i=1} \psi^{\mathrm{OS}}_0(O_i) + o_p(N^{-1/2}) \qquad (N\to\infty). \end{align} Consequently, we have \begin{align} \sqrt N \left( \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} - \theta^{\mathrm{OS}}_0 \right) \xrightarrow{d} N\left(0,V^{\mathrm{OS}}_0\right) \qquad (N\to\infty). \end{align} For TMLE-DML-PPCI, suppose in addition that Assumption (ref) holds. Then, we have \begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \widehat\theta^{\mathrm{OS}}_{\mathrm{EE}} = o_p(N^{-1/2}) \qquad (N\to\infty), \end{align} and hence \begin{align} \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \theta^{\mathrm{OS}}_0 = \frac{1}{N}\sum^N_{i=1} \psi^{\mathrm{OS}}_0(O_i) + o_p(N^{-1/2}) \qquad (N\to\infty). \end{align} Consequently, we have \begin{align} \sqrt N \left( \widehat\theta^{\mathrm{OS}}_{\mathrm{TMLE}} - \theta^{\mathrm{OS}}_0 \right) \xrightarrow{d} N\left(0,V^{\mathrm{OS}}_0\right) \qquad (N\to\infty). \end{align} Thus, EE-DML-PPCI is regular and semiparametrically efficient under Assumptions (ref), (ref), (ref), (ref), and (ref). TMLE-DML-PPCI is regular and semiparametrically efficient under the same assumptions and the additional targeting calibration condition.

Conclusion

This study develops PPCI for causal and structural regression functionals with labeled observations and unlabeled auxiliary regressors. We derive the efficient influence functions and efficiency bounds under two different DGPs, or sampling schemes, and develop an estimation method called DML-PPCI. In particular, we present EE-DML-PPCI and TMLE-DML-PPCI as special implementations of DML-PPCI. We also develop semi-supervised generalized Riesz regression for estimating the Riesz representer in the Neyman orthogonal score. Using these methods, we show that we can construct estimators whose asymptotic variances are smaller than those of estimators constructed using only the labeled dataset. These results clarify when unlabeled regressors improve efficiency.

\onecolumn