Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
73,997 characters · 14 sections · 45 citation commands
Kernel methods for long term dose response curves
\def\spacingset#1{ {#1}} \spacingset{1}
{\it Keywords:} continuous treatment, reproducing kernel Hilbert space, surrogate variable
\spacingset{1.9}
Experimental data have randomized actions, yet they are typically limited in duration due to the expense of conducting an experiment. From an artificial intelligence perspective, the long term consequence of an action may be of interest, yet only short term rewards may be collected in exploration. From a program evaluation perspective, the long term effect of an action is what often matters, yet only short term outcomes may be collected in experimental surveys.
The goal of long term causal inference is to properly combine short term experimental data, which contain randomized actions, with long term observational data, which lack randomized actions prentice1989surrogate,freedman1992statistical,begg2000use,frangakis2002principal. Our research question is how to extrapolate long term effects of continuous actions, allowing nonlinearity and heterogeneity in the link between the short term and the long term.
We study nonparametric causal functions of a continuous action $D$, which may refer to intervention intensities, medical dosages, or program lengths. For example, Project STAR randomly assigned kindergarten students to various class sizes $D$, ranging from 12 to 28 students. The short term test scores $S$ of these students were collected, however their long term earnings $Y$ were not. Separately, researchers may have access to test scores $S$ and earnings $Y$ for students in another school district. Several social scientists krueger2001effect,chetty2011does have asked: how much would the average student benefit, in the long term, from enrolling in a certain kindergarten class size? This quantity is an example of a long term dose response $\E\{Y^{(d)}\}$, where the class size is the dose, and the expected long term benefit is the response.
The difficulty in estimating long term dose response curves is the complex nonlinearity and heterogeneity in the link between short term response curve $\E\{S^{(d)}\}$ and long term response curve $\E\{Y^{(d)}\}$. For example, we would like to allow for the link between counterfactual test scores and counterfactual earnings to be nonlinear, and for students with different baseline characteristics to have different links. The identification of long term dose response curves is well known rosenman2018propensity,athey2020estimating,athey2020combining,rosenman2020combining, kallus2020role, however it appears that no nonlinear, nonparametric estimators have been previously proposed for the response curve, highlighting the difficulty of extrapolating the long term effects of continuous actions.
Our primary contribution is to propose and analyze what appears to be the first nonlinear, nonparametric estimator for the long term dose response curve. We adapt kernel ridge regression, a classic statistical algorithm wahba1990spline, to this learning problem. By embedding the distribution of the short term experimental data using kernels, we derive interpretable weights for extrapolating long term effects from short term effects. The final estimator has a simple closed form solution, while preserving nonlinearity and heterogeneity in the link between the short term and long term. As an extension, we study long term counterfactual distributions in Appendix (ref).
We use the statistical framework of the reproducing kernel Hilbert space (RKHS) steinwart2008support,berlinet2011reproducing to overcome the technical issues that arise when estimating long term dose response curves. The identifying formula for the long term dose response curve is generally unbounded when actions are continuous van1991differentiable,newey1994asymptotic, but we prove that it is bounded in the RKHS. Therefore, our proposed kernel embedding of short term experimental data exists. To adjust for how experiments may be limited in not only duration but also scope, we further define and characterize a “fused” kernel embedding.
Using this embedding, we prove the estimator is uniformly consistent across actions, with finite sample rates that combine well known minimax rates for kernel ridge regression. The rates do not directly depend on the ambient dimension of the data, but rather the effective dimension and smoothness of the data with respect to an approximating basis caponnetto2007optimal,fischer2017sobolev. The approximating basis is the spectrum of a kernel chosen by the researcher. See Section (ref) for interpretation and comparison to Sobolev rates.
We illustrate the practicality of our approach by estimating the long term dose response of Project STAR, modelling class size as a continuous action. By allowing for continuous actions and heterogeneous links, our long term dose response estimate suggests that the effects of class size are nonlinear. Using short term experimental data and long term observational data, our method measures similar long term effects as an oracle method that has access to long term experimental data.
Several works provide strong semiparametric guarantees for the long term effects of binary actions kallus2020role,chen2021semiparametric,meza2021nested. By contrast, we prove what appear to be the first nonparametric guarantees for the nonlinear, long term dose response curves of continuous actions. Whereas binary actions imply boundedness of the identifying formula, continuous actions imply unboundedness in general; see Section (ref). Continuous actions introduce technical issues that we overcome with RKHS techniques.
We complement a literature that incorporates RKHS techniques into the estimation of other causal functions. For experimental data with joint observations of the randomized actions and final rewards, previous works analyze heterogeneous effects nie2021quasi, dose response curves singh2020kernel, and time-varying dose response curves singh2021sequential. Another strand of this literature incorporates instrumental variables to deal with unobserved confounding carrasco2007linear,singh2019kernel. We study a different problem that combines short term experimental data with long term observational data. Previous methods using kernels for continuous actions do not handle the complex linkage between short term and long term effects across data sources, and therefore cannot analyze long term causal inference. Our contribution is a method to do so.
Prior to this work, battocchi2021estimating allow for continuous actions but impose restrictive linearity and homogeneity conditions, in both of the responses and in their link. By contrast, we study a nonparametric dose response estimator with nonlinearity and heterogeneity in the responses and their link. Subsequently to this paper's circulation singh2022generalized, zeng2024continuous provide complementary results for the long term dose response curve. For a given action value, the authors prove pointwise asymptotic consistency of a pseudo-outcome estimator, and pointwise asymptotic normality centered at the biased probability limit of a local linear estimator. By contrast, we prove uniform consistency across action values, with nonasymptotic rates, for a kernel ridge regression estimator. Uniform inference for kernel ridge regression over general domains is an open question in statistics, so uniform inference for our estimators is an important question for future work. Finally, we note that this paper subsumes a related working paper singh2021generalized.
Section (ref) formalizes long term causal inference as a core challenge for artificial intelligence and program evaluation. Section (ref) summarizes RKHS techniques and assumptions, which serve as our statistical framework. Section (ref) uses RKHS techniques to derive a closed form estimator, with interpretable weights for extrapolating long term effects. Section (ref) uses RKHS assumptions to derive uniform rates of convergence that reflect the effective dimension of the data. Section (ref) demonstrates that our long term dose response estimators recover the long term effects of continuous actions reasonably well, using real data.
A long term dose response curve is a causal function that summarizes the expected, counterfactual, long term reward $Y^{(d)}$, given a hypothetical intervention on a continuous action that sets $D=d$. For example, Figure (ref) shows that Project STAR randomly assigned students to many different class sizes, which we model as a continuous action. The crux of long term causal inference is that we do not have joint observations of the randomized action and long term reward. Nonetheless, we wish to estimate their causal relationship from short term experimental data $(G=\EXP)$ and long term observational data $(G=\OBS)$. We study several variations of long term dose response curves. Unless noted, expectations are with respect to a superpopulation distribution $\P$ with density $\d\P$.
The dose response curve $\theta_0(d)$ is for a representative context, drawn from the superpopulation of contexts which generate the short term experimental data $(G=\EXP)$ and the long term observational data $(G=\OBS)$. It may depart from the experimental dose response $\theta_0^{\EXP}(d)$ and the observational dose response $\theta_0^{\OBS}(d)$ due to a selection process: $G$, viewed as a random variable, may not be drawn completely at random. By distinguishing among these functions, we emphasize that the experimental data may be limited in not only duration but also scope; the long term effect of an action in a representative context may be of interest, yet only short term rewards for select contexts may be collected in exploration.
The distribution shifted dose response curve $\theta_0^{\DS}(d,\tilde{\P})$ concerns transfer learning: though our experimental and observational data are drawn from a superpopulation distribution of contexts $\P$, what would be the dose response for an alternative distribution of contexts $\tilde{\P}$ pearl2014external? We will consider a rich form of distribution shift formalized below.
Definition (ref) generalizes from long term dose response curves to long term counterfactual distributions by replacing the $\E$ symbols with $\P$ symbols. Long term counterfactual distributions capture aspects of the long term reward distribution beyond its mean, e.g. its variance and skewness. Appendix (ref) extends our results accordingly. This work appears to provide the first nonlinear, nonparametric estimators for long term dose response curves and counterfactual distributions.
Several previous works identify long term dose response curves, under a variety of identifying assumptions. For clarity of exposition, we focus on the surrogate model for analysis, then demonstrate how our results apply to related models in Remark (ref). We implement kernel estimators for multiple long term causal inference models in Section (ref).
The surrogate model posits the following data generating process. A single observation begins with a context $X$ drawn from the distribution $\P$. Conditional on the context, the selection $G\in\{\EXP,\OBS\}$ is as good as random: $\P(G=\EXP|X)$ is a function of the context $X$, but the actual realization of $G$ is a biased coin toss.
For experimental data, i.e. if $G=\EXP$, then the action $D$ is also random conditional upon $X$: the density $\d\P(D|G=\EXP,X)$ is a function of the context $X$, but the actual realization of $D$ is a random draw from the density. Finally, we observe the short term reward $S$ drawn from the density $\d\P(S|D=d,G=\EXP,X)$. From experimental data alone, the counterfactual distribution of context-specific short term effects $\P\{S^{(d)}|G=\EXP,X\}$ is identified as $\P\{S|D=d,G=\EXP,X\}$, by standard arguments, under a further overlap condition for actions across experimental contexts.
To identify long term effects, long term data are necessary. However, these long term data are observational: if $G=\OBS$, then the action $D$ is not randomized. In the simplest version of the model, the action is not observed at all when $G=\OBS$. What we do observe is the short term reward $S$ and long term reward $Y$, so $\P(Y|S,G=\OBS,X)$ is identified.
To combine short and long term data, prentice1989surrogate,athey2020estimating, and others, articulate nonparametric surrogacy and comparability assumptions. Surrogacy imposes that the action only affects the long term reward via the short term reward in the experiment: $\P(Y|S,D,G=\EXP,X)=\P(Y|S,G=\EXP,X)$. Comparability imposes that the long term reward follows a common conditional distribution across the experimental and observational data: $\P(Y|S,G=\OBS,X)=\P(Y|S,G=\EXP,X)$. With overlap, these conditions identify the counterfactual distribution of context-specific long term effects $\P\{Y^{(d)}|X\}$ as $\int \P(Y|s,G=\OBS,X)\d\P(s|d,G=\EXP,X)$.
From this identifying formula, we determine what kinds of distribution shift can be tolerated. It helps if the alternative distribution of contexts $\tilde{\P}$ preserves $\P(Y|S,G=\OBS,X)$ and $\P(S|D,G=\EXP,X)$. For distribution shift, we impose $\tilde{\P}(Y,S,G=\OBS,X)=\P(Y|S,G=\OBS,X)\tilde{\P}(S,G=\OBS,X)$ and $\tilde{\P}(S,D,G=\EXP,X)=\P(S|D,G=\EXP,X)\tilde{\P}(D,G=\EXP,X)$, again with overlap. Importantly, we tolerate shifts in the observational distribution of short term rewards i.e. $\tilde{\P}(S|G=\OBS,X)\neq \P(S|G=\OBS,X)$, and shifts in the experimental distribution of randomized actions i.e. $\tilde{\P}(D|G=\EXP,X)\neq \P(D|G=\EXP,X)$. This extension, to long term transfer learning, appears to be a modest innovation.
Lemma (ref) expresses each causal function as a double integral of the regression function $\gamma_0$ learned from observational data, integrated according to a conditional distribution learned from experimental data, and then further integrated according to a possibly conditional distribution of contexts. Since $x$ appears in $\gamma_0(s,G=\OBS,x)$, $\P(s|d,G=\EXP,x)$, and $\P(x)$, the components in this integral are coupled, so the long term dose response seems challenging to estimate.
We use techniques from the RKHS framework to overcome the practical challenge of estimating the double integral in Lemma (ref), and the technical challenge of estimating an unbounded formula in Remark (ref). These algorithmic techniques lead to a simple estimator with a closed form solution.
A scalar-valued RKHS $H$ is a Hilbert space of functions $f:\cA\rightarrow\R$, where $\cA$ is some Polish space. Functions in this space are characterized by the feature map, which takes a point $a\in\cA$ and maps it to features $\phi(a)$ in the RKHS. The closure of $\Span\{\phi(a)\}_{a\in\cA}$ is $H$; the features are the dictionary of basis functions for functions in $H$. In particular, to evaluate $f\in H$ at $a$, we take the RKHS inner product $f(a)=\langle f,\phi(a)\rangle_H$. The kernel $k:\cA\times \cA \rightarrow \R$ is the inner product of features: $k(a,a')=\langle \phi(a),\phi(a')\rangle_H $. It is continuous, symmetric, and positive definite.
The RKHS is widely used for nonparametric regression of some output $Y\in\R$ on some input $a\in\cA$. A kernel ridge regression estimator of the regression function $f_0(A)=\E(Y|A)$ is
where $\lambda>0$ is a ridge penalty hyperparameter. The minimizer has a well known closed form solution kimeldorf1971some, as a weighting $\alpha\in \R^n$ of the output vector $Y\in \R^n$:
Here, $K_{AA}\in \R^{n\times n}$ is the kernel matrix with $(i,j)$th entry $k(A_i,A_j)$, and $K_{Aa}\in\R^n$ is the kernel vector with $i$th entry $k(A_i,a)$. For tuning $\lambda$, previous work has shown that generalized cross validation and leave-one-out cross validation have closed form solutions, the former is asymptotically optimal, and they are similar in practice craven1978smoothing,li1986asymptotic. We will derive interpretable weights for long term rewards, calculated from the short term experiment, which deliver long term dose response curves.
To take the expectation of a function $f\in H$, a useful technique is the kernel mean embedding: $$ \E\{f(A)|B=b\}=\int \langle f,\phi(A) \rangle_H\d\P(a|b)=\left\langle f,\int \phi(a)\d\P(a|b) \right\rangle = \langle f, \mu_a(b) \rangle_H, $$ where $\mu_a(b)=\int \phi(a)\d\P(a|b)$ embeds the conditional distribution $\P(a|b)$ using the features $\phi(a)$ smola2007hilbert. Exchanging the expectation and inner product uses weak regularity conditions (Assumption (ref) below). In summary, the expectation of a function in the RKHS equals the product of the function and the mean embedding. We will embed the short term experimental data distribution appropriately, for the extrapolation of long term effects.
The mean embedding $\mu_a(b)=\E \{\phi(A)|B=b\}$ is a generalized regression function, so we will estimate it in a manner that generalizes (ref) and (ref). In particular, we will estimate its equivalent conditional expectation operator $E_0:f(\cdot)\mapsto \E\{f(A)|B=(\cdot)\}$, which we assume maps from an RKHS $H_{\cA}$ with features $\phi(a)$ to another RKHS $H_{\cB}$ with features $\phi(b)$, for simplicity.\footnote{Formally, $H_{\cA}$ has the kernel $k_{\cA}(a,a')$ and features $\phi_{\cA}(a)$, while $H_{\cB}$ has the possibly different kernel $k_{\cB}(b,b')$ and features $\phi_{\cB}(b)$. We suppress the subscripts to lighten notation.} This ensures that the expectation of a function of $A$, conditional upon $B$, is a smooth function of $B$. It leads to the simple equivalence $\mu_a(b)=E_0^*\phi(b)$, where $E_0^*$ is the adjoint of $E_0$. We estimate $\hat{E}$ as a generalized kernel ridge regression in the space of Hilbert-Schmidt operators $\cL_2(H_{\cA},H_{\cB})$ micchelli2005learning, and hence $\hat{\mu}_a(b)=\hat{E}^*\phi(b)$. This generalized regression is how we will learn the appropriate representation of the short term experiment for long term causal inference.
For functions defined over multiple variables, we use product kernels. Consider the function $f:\cA\times\cB\rightarrow\R$ defined over $A\in\cA$ and $B\in\cB$. We define an RKHS $H$ for this function by combining RKHSs $H_{\cA}$ and $H_{\cB}$ containing functions $f_{\cA}:\cA\rightarrow \R$ and $f_{\cB}:\cB\rightarrow \R$, respectively. Its kernel is the product of the individual kernels: $k(a,b;a',b')=k_{\cA}(a,a')k_{\cA}(b,b')$. If $k_{\cA}$ and $k_{\cB}$ are Gaussian kernels, then so is their product, making it a natural choice. Because we use product kernels, the estimators will have the symbol $\odot$ for the elementwise product of kernel matrices. Formally, $H=H_{\cA}\otimes H_{\cB}$ is a tensor product RKHS with features $\phi(a,b)=\phi(a)\otimes \phi(b)$, where the symbols $\otimes$ means outer product: $(x\otimes y)z=x\langle y,z\rangle$. For any $f\in H$, $f(a,b)=\langle f,\phi(a)\otimes \phi(b) \rangle_H$ and $\|\phi(a,b)\|_H=\|\phi(a)\|_{H_{\cA}}\|\phi(b)\|_{H_{\cB}}$. berlinet2011reproducing give background.
Our main assumption is that the RKHS used in estimation has a low effective dimension: while the features may be infinite dimensional, relatively few feature dimensions explain most of the variation in the data. We also assume that the estimands are smooth with respect to these features. In what follows, we formalize these familiar assumptions from RKHS analysis caponnetto2007optimal,fischer2017sobolev in terms of the spectrum of the kernel.
To begin, we express the features as a possibly infinite expansion of the kernel. Let $\L^2$ denote the space of square integrable functions from $\cA$ to $\R$ with respect to measure $\P$. For a kernel $k$, define the convolution operator $L:\L^2\rightarrow\L^2$ as $L:f\mapsto \int k(a,\cdot)f(a)\d\P(a)$, which may be expressed in terms of its spectrum: $Lf=\sum_{j=1}^{\infty}\eta_j\langle f,\varphi_j\rangle_H\varphi_j$. Here, $(\eta_j)$ are weakly decreasing eigenvalues, and $(\varphi_j)$ are corresponding eigenfunctions, of the data passed through the kernel. For simplicity, suppose $(\varphi_j)$ form an orthonormal basis of $\L^2$. With Gaussian data and the Gaussian kernel, $(\varphi_j)$ are weighted Hermite polynomials williams2006gaussian. In general, $$ \L^2=\left(f=\sum_{j=1}^{\infty}f_j\varphi_j:\sum_{j=1}^{\infty}f_j^2<\infty\right),\quad H=\left(f=\sum_{j=1}^{\infty}f_j\varphi_j:\sum_{j=1}^{\infty}\eta_j^{-1}f_j^2<\infty\right). $$ The RKHS $H$ is the subset of $\L^2$ for which higher order terms in the series $(\varphi_j)$ have a smaller contribution, as penalized by the eigenvalues $(\eta_j)$. By Mercer's theorem, the feature map is the infinite dictionary of basis functions given by the spectrum of the kernel: $\phi(a)=\{\eta_j^{1/2}\varphi_j(a)\}_{j=1}^{\infty}$.
Our main approximation assumption is that the features have a low effective dimension:
The eigenvalues decay at least polynomially. Any bounded kernel satisfies (ref) fischer2017sobolev. A higher value of $b$ corresponds to a lower effective dimension, better control of the variance of our estimator, and hence a faster rate. The limit $b\rightarrow \infty$ gives an RKHS with finite dimension caponnetto2007optimal. The empirical eigenvalues are simple to compute, so it is simple to validate this assumption with a diagnostic plot. Figure (ref) verifies polynomial decay of the empirical eigenvalues in the real world application of Section (ref); the Project STAR data have a low effective dimension as required by Assumptions (ref) and (ref).\footnote{Specifically, we divide each empirical eigenvalue by the trace of the corresponding matrix, to convey the fraction of variation explained.}
We also assume that the estimands are smooth respect to these features, satisfying what is known as a source condition smale2005shannon,carrasco2007linear:
Note that $H^0=\L^2$ and $H^1=H$, while $H^c$ is contained in $H$ for $c>1$. A larger value of $c$ corresponds to a smoother target $f_0$, better control of the bias of our estimator, and hence a faster rate. Figure (ref) visualizes $c=1$ versus $c=2$ in a concrete example.
These approximation assumptions generalize well known Sobolev conditions fischer2017sobolev. Consider the Sobolev space $\H_2^s$ of functions $f:\cA\rightarrow \R$ with $\cA\subset\R^p$, where $s$ is the number of square integrable derivatives, with respect to the Lebesgue measure. This space is an RKHS when $s>p/2$, with the Mat\'ern kernel berlinet2011reproducing. Take $H=\H_2^s$ with $s>p/2$ as the RKHS for estimation. If the target $f_0$ has $s_0$ derivatives, i.e. $f_0\in \H_2^{s_0}$, then $c=s_0/s$ and hence $\H_2^{s_0}=(\H_2^s)^c$ in the sense of (ref). Meanwhile, the effective dimension of $\H_2^s$ is $b=2s/p$ in the sense of (ref). Intuitively, $c$ measures the relative smoothness of the target to the estimator, while $b$ measures the relative smoothness of the estimator to the dimension. As $s\rightarrow \infty$, the Mat\'ern kernel converges to the widely used Gaussian kernel.
Rates in terms of $(b,c)$ reflect the effective dimension of the features, and the smoothness of the estimand relative to the features. The minimax optimal rate for regression in RKHS norm is $n^{-\frac{1}{2}\frac{c-1}{c+1/b}}$, achieved by kernel ridge regression with rate optimal regularization $\lambda=n^{-\frac{1}{c+1/b}}$. This rate approaches $n^{-1/4}$ when $b\rightarrow \infty$ and $c=2$, i.e. when the effective dimension is finite and the smoothness is high. This paper's results combine rates (and regularization paths) of this form. They apply to Sobolev spaces over $\R^p$ as a special case, where $n^{-\frac{1}{2}\frac{c-1}{c+1/b}}=n^{-\frac{s_0-s}{2s_0+p}}$. More generally, they allow variables to belong to Polish spaces.
We construct an RKHS for long term dose response curve estimation, in which we embed the distribution of the short term experimental data. We define RKHSs for the short term reward $S \in\cS$, action $D\in\cD$, and context $X \in\cX$. For example, for the short term reward, we define the RKHS $H_{\cS}$ with feature map $\phi_{\cS}(s)$ and kernel $k_{\cS}(s,s')=\langle \phi_{\cS}(s),\phi_{\cS}(s')\rangle_{H_{\cS}}$.
From these RKHSs for short term rewards and contexts, we construct an RKHS $H$ for the long term reward mechanism $\gamma_0^{\OBS}(S,X)=\gamma_0(S,G=\OBS,X)=\E(Y|S,G=\OBS,X)$. We assume that it belongs to the RKHS with the product kernel $k(s,x;s',x')=k_{\cS}(s,s')k_{\cX}(x,x')$. Equivalently, $\gamma_0^{\OBS}\in H=H_{\cS} \otimes H_{\cX}$, where $\otimes$ means tensor product.\footnote{For the alternative long term causal inference models in Remark (ref), $H=H_{\cS} \otimes H_{\cD}\otimes H_{\cX}$ instead.} The features for $\gamma_0^{\OBS}$ are the outer product of features for the individual RKHSs: $\gamma_0^{\OBS}(s,x)=\langle \gamma_0^{\OBS},\phi(s,x) \rangle_{H}=\langle \gamma_0^{\OBS},\phi(s)\otimes \phi(x) \rangle_{H}$.
Measurability, boundedness, and the characteristic property are weak regularity conditions satisfied by commonly used kernels, e.g. the Mat\'ern and Gaussian kernels. The characteristic property ensures that kernel mean embeddings of distributions are unique sriperumbudur2010relation.
The long term regression $\gamma_0^{\OBS}(s,x)$ allows for nonlinearity and heterogeneity in the link between the short term and long term. The short term kernel mean embedding $\mu^{\EXP}_s(d,x)$ allows for nonlinearity and heterogeneity in the counterfactual distribution of short term rewards. This embedding is a representation of the experiment, and we must learn it from the data. The “fused” embeddings $\mu_{s,x}(d)$, $\tilde{\mu}_{s,x}(d)$, $\mu'_{s,x}(d)$, and $\mu''_{s,x}(d)$ further adjust for the possibility that experiments are limited in scope, i.e. that the distribution of experimental contexts $\P(X|G=\EXP)$ is not representative.
For intuition, we interpret Theorem (ref) in terms of Example (ref). Here, $\mu_s^{\EXP}(d,x)$ encodes short term experimental data $\P(S|D,G=\EXP,X)$ as the conditional moments $\int s^2\d\P(s|D,G=\EXP,X)$, $\sqrt{2}\int s \d\P(s|D,G=\EXP,X)$, and $1$. The fused embedding $\mu_{s,x}(d)$ encodes the desired context distribution $\P(X)$ by interacting these conditional moments with the context moments.\footnote{Altogether, there are $3\times 3$ moments: $\int x^2\int s^2\d\P(s|d,G=\EXP,x) \d\P(x)$, $\sqrt{2}\int x^2\int s \d\P(s|d,G=\EXP,x) \d\P(x)$, and $\int x^2 \d\P(x)$; $\sqrt{2}\int x\int s^2\d\P(s|d,G=\EXP,x) \d\P(x)$, $2\int x\int s \d\P(s|d,G=\EXP,x) \d\P(x)$, and $\sqrt{2}\int x \d\P(x)$; and $\int\int s^2\d\P(s|d,G=\EXP,x) \d\P(x)$, $\sqrt{2}\int\int s \d\P(s|d,G=\EXP,x) \d\P(x)$, and $1$.} In this toy example, the long term dose response $\theta_0(d)$ is simply the long term regression $\gamma_0^{\OBS}$ evaluated at these interacted quadratic moments. Quadratic kernels only preserve the first and second moments of the data, so we use characteristic kernels to preserve all moments.
The product kernel is not necessary for Theorem (ref). For example, the same result holds for the sum kernel, replacing the tensor product $\otimes$ with the concatenation $\oplus$. However, the product kernel is interpretable as interactions that allow for heterogeneity in the link and the effects. It also simplifies the proof of Theorem (ref) below. Further products, besides $\phi(s,x)=\phi(s)\otimes \phi(x)$, are only for the sake of exposition. Moreover, measurability of $\phi(D)$ and boundedness of $k_{\cD}$ are not necessary for Theorem (ref) in the surrogate model, but they are invoked in the alternative long term models of Remark (ref).
Theorem (ref) provides a blueprint for a long term dose response curve algorithm that avoids density estimation and numerical integration. Our algorithm is of the form $\hat{\theta}(d)=\langle \hat{\gamma}^{\OBS}, \hat{\mu}_{s,x}(d) \rangle_H$ where $\hat{\gamma}^{\OBS}$ is a kernel ridge regression estimated from observational data, $\hat{\mu}^{\EXP}_s(d,x)$ is a generalized kernel ridge regression estimated from experimental data, and $\hat{\mu}_{s,x}(d)=\frac{1}{n}\sum_{i=1}^n \{\hat{\mu}_s^{\EXP}(d,X_i) \otimes \phi(X_i)\}$ averages over all data. We prove that this algorithm has a closed form solution, generalizing (ref).
Theorem (ref) gives theoretical values of $(\lambda^{\OBS},\lambda^{\EXP})$ to optimally balance bias and variance for the regressions. Appendix (ref) gives practical tuning procedures with closed form solutions to empirically balance bias and variance, one of which is asymptotically optimal for regression.
A mature literature on kernel methods for binary actions, with joint observations of randomized actions and final outcomes, emphasizes the interpretation of outcome weights; see e.g. bruns2023augmented for a recent review and many references, going back to the classical work of speckman1979minimax. We show that our estimators share some of these virtues despite the additional challenges of our setting: continuous actions and long term extrapolation.
These weights allow us to evaluate how trustworthy our estimator is in certain respects, e.g. its robustness and transparency. For example, we may compare different choices of the kernels $k_{\cS}$ and $k_{\cX}$ in the nonlinear model $\gamma_0^{\OBS}$ that links the short term and long term. For each choice, we can examine the magnitudes of the weights to assess which regions of the short term reward and context space receive greater weight. A larger $|\hat{\alpha}_i(d)|$ means that observation $i\in\OBS$ has more influence on the final estimate. For each choice, we can also examine the signs of the weights to assess how much extrapolation is taking place. A negative weight, i.e. $\hat{\alpha}_i(d)<0$, means that our counterfactual prediction of the long term reward does not simply interpolate among $Y_i$ for $i\in\OBS$.
Such weights would not exist in general, for long term dose response curve estimators outside of the RKHS. As shown in Remark (ref), the surrogate formula with continous actions is unbounded over $\L^2$ in general. As such, it does not have a Riesz representer over $\L^2$. The weights, if they exist, are empirical analogues of the Riesz representer.
Our algorithm keenly relies on the fact that such weights do exist for long term dose response curve estimators in the RKHS. As shown in Remark (ref), the surrogate formula with continuous actions is bounded over $H$. As such, it does have a Riesz representer over $H$, given by the fused embedding $\mu_{s,x}(d)$ in Theorem (ref). The weights $\hat{\alpha}(d)$ are calculated from its empirical analogue.
For identification in Section (ref), we placed the long term causal inference assumptions. To derive a closed form estimator in Section (ref), we assumed RKHS regularity (Assumption (ref)). To prove uniform consistency across actions, we now place three final assumptions: original space regularity (Assumption (ref)), low effective dimension and smoothness of the long term regression (Assumption (ref)) and likewise for the short term conditional expectation operator (Assumption (ref)).
For simplicity, the long term reward is a bounded scalar. Our results in Appendix (ref) relax this condition, allowing $\cY$ to be separable Hilbert space. We allow short term rewards, actions, and contexts to be discrete or continuous. This generality means our results can be applied to nonstandard data such as texts, images, and graphs
Next, we place our main approximation assumptions for the long term regression and short term conditional expectation operator. We assume that they have low effective dimensions and are smooth in the sense of (ref) and (ref), which are standard assumptions in the RKHS framework. Extending the notation of Section (ref), we specify the measure and space of the eigendecomposition.\footnote{For example, $\eta^{\OBS}_j(H)$ is the $j$th eigenvalue of the operator $L^{\OBS}:\gamma(\cdot )\mapsto \int k\{s,x;(\cdot)\}\gamma(s,x)\d\P(s,x|G=\OBS)$, where $k$ is the kernel of the RKHS $H=H_{\cS}\otimes H_{\cX}$.}
Under these conditions, we arrive at our main result: uniform consistency of long term dose response curves. This result appears to be the first rate in $\sup$ norm for a nonlinear, long term dose response. It accommodates general types of short term rewards, actions, and contexts.
Appendix (ref) gives nonasymptotic error bounds in more detail. The rates reflect the sample sizes $(n^{\OBS},n^{\EXP})$, effective dimension parameters $(b^{\OBS},b^{\EXP})$ and the smoothness parameters $(c^{\OBS},c^{\EXP})$ of the long term regression $\gamma_0^{\OBS}$ and short term embedding $\mu_s^{\EXP}(d,x)$. These rates approach $n^{-1/4}$ when $n^{\OBS}=n^{\EXP}$, $(b^{\OBS},b^{\EXP})\rightarrow \infty$, and $(c^{\OBS},c^{\EXP})=2$, i.e. when the sample sizes are of the same order, the effective dimensions are finite, and the targets are smooth. Each rate combines minimax optimal rates in RKHS norm for standard nonparametric regression, conditional mean embedding, and unconditional mean embedding tolstikhin2017minimax,fischer2017sobolev,li2022optimal. See Section (ref) for translation into mimimax Sobolev norm rates, which are a special case.
We prove uniform rates for an RKHS estimator of the long term dose response curve, under effective dimension and smoothness conditions on the regression and embedding. An interesting direction for future work is whether uniform rate improvements are possible by placing additional assumptions, perhaps building on the techniques from pointwise, mean square, or excess risk analysis kennedy2017nonparametric,nie2021quasi,foster2023orthogonal,zeng2024continuous.
To demonstrate that our proposed kernel methods are practical for empirical research, we evaluate their ability to recover long term dose response curves. Using short term experimental data and long term observational data, our methods measure similar long term effects as an oracle method that has access to long term experimental data. Our methods outperform some benchmarks from previous work that use only long term observational data.
Our exercise directly extends the exercise of athey2020combining. Whereas those authors study the long term effect of a binary action (“small” versus “large” classes), we study the long term effect of a continuous action (various class sizes). Figure (ref) shows that the randomized action $D$ takes values in $\cD=[12, 28]$. It also shows that the supports of the “small” and “large” classes are overlapping; some “small” classes were larger than some “large” classes. By modeling $D$ as continuous with a Gaussian kernel $k_{\cD}$, we smoothly share information across different class sizes.
As in previous work, we consider the third grade test score to be the short term reward $S$, and a subsequent test score to be the long term reward $Y$. By choosing different grades as different long term rewards, we evaluate how our methods perform over different horizons. Our variable definitions are identical to athey2020combining, except that we use a continuous action.
The oracle, visualized in red, is estimated from long term experimental data, i.e. joint observations of the randomized action $D$ and long term reward $Y$ in Project STAR. Our goal is to recover similar estimates without access to long term experimental data. Figure (ref) shows that the oracle curve is typically decreasing: larger class sizes appear to cause lower test scores, across horizons. In particular, the oracle estimates are nonlinearly decreasing, from positive counterfactual test scores (above average) to negative counterfactual test scores (below average). As the long term horizon increases, i.e. as the definition of $Y$ corresponds to later grades, the oracle curves flatten: the effect of kindergarten class size on test scores appears to attenuate over time.
To explore why the curves appear non-monotonic, we visualize in gray the ranges of class sizes targeted in the Project STAR protocol. There is a positive bump at $d=22$, which is the beginning of a targeted range, suggesting that there may be selection at this boundary.
The benchmarks, visualized in black, are estimated from joint observations of the nonrandomized action $D$ and long term reward $Y$ in observational data from New York City (NYC) public schools. We consider versions that do (dashed) or do not (solid) adjust for baseline covariates $X$: race, gender, and free and reduced price lunch status. We implement the benchmarks using an estimator from previous work based on kernel ridge regression singh2020kernel. The benchmark estimates are steeply increasing, whereas the oracle estimates are decreasing; a researcher using previous kernel methods for continuous actions would arrive at the opposite of the desired conclusion.
Finally, we implement our proposals, estimated using short term experimental data from Project STAR and long term observational data from NYC. We consider versions for the surrogate model in Lemma (ref) (light blue), and the missing-at-random model in Remark (ref) (dark blue). The former uses $(D,S)$ from Project STAR and $(S,Y)$ from NYC, while the latter uses $(D,S)$ from Project STAR and $(D,S,Y)$ from NYC. Both versions of our method recover the direction and nonlinear shape of the oracle. The latter version recovers the oracle particularly well, suggesting that the missing-at-random identification may be more applicable than the surrogacy identification in this setting. Appendix (ref) demonstrates that these trends continue over longer horizons.
Long term causal inference is a core challenge in both the development of artificial intelligence and in the evaluation of social programs; exploratory and experimental data are often short term, due to collection expenses, privacy concerns, or ethical considerations. We propose a kernel method that learns a representation of the short term experimental data distribution, in order to extrapolate long term dose response curves. This representation provides interpretable weights. We consider the additional challenge of distribution shift, i.e. of transfer learning.
As a contribution to long term causal inference, we propose what appears to be the first nonlinear, nonparametric estimator for long term dose response curves, by adapting techniques and assumptions from the RKHS statistical framework. We derive a closed form for the estimator using RKHS techniques, and prove that it uniformly consistent across actions under RKHS assumptions. The RKHS framework improves our scientific understanding of real data from Project STAR.
Future work may extend our results to neural network embeddings, or to reinforcement learning problems. Future work may also improve scalability via Nystr\"om approximations.
\spacingset{1}
\spacingset{1.9}