On the Existence and Information of Orthogonal Moments

On the Existence and Information of Orthogonal Moments

Introduction

A recent and growing literature in economics and machine learning recommends Locally Robust (LR)/Orthogonal/Debiased moments for inference on parameters of interest in the presence of high-dimensional first steps (see, e.g., Athey and Wager (2021), Belloni et al. (2012, 2017), Bravo, Escanciano and Van Keilegom (2020), Chernozhukov et al. (2018), Chernozhukov et al. (2016, 2022, henceforth CEINR), Farrell (2015), Nekipelov, Semenova, and Syrgkanis (2022), and Sasaki and Ura (2021), among many others). The literature on debiased machine learning has shown that LR moments have several advantages over plug-in approaches. Most notably, LR moments are useful to reduce model selection and regularization biases when machine learning estimators are used as first steps. Furthermore, orthogonal moments have a fast rate of convergence and provide valid confidence intervals for structural parameters under more general conditions than plug-in methods. However, a fundamental question that has not been addressed in the literature is the existence of such orthogonal moments for inference and for general parameters. If one equates reliable inference to orthogonal moments, we are asking:

Under what minimal conditions is reliable inference on a parameter of interest possible?

In this paper, we give a necessary and sufficient condition for the existence of LR moments. Furthermore, when orthogonal moments exist, we characterize when they are informative.\footnote{Throughout the paper, we use indistinguishably the terms informative and relevant to refer to moments for which the corresponding tests have non-trivial local power functions at the parametric rate.}

To introduce the main result, let us consider a semiparametric model in which the parameter of interest is fixed at a specified known value under the null hypothesis, referred to as the restricted model. If this restricted model is (locally) nonparametric, in the sense that the set of scores of the restricted model can approximate any zero-mean square-integrable function arbitrarily well, then we say the model satisfies Restricted Local Surjectivity (RLS). Otherwise, the model satisfies Restricted Local Non-Surjectivity (RLN). The first result of this paper is that, subject to regularity conditions, RLN is necessary and sufficient for the existence of an orthogonal moment function for inference on the parameter of interest.

We argue that RLN follows from mild conditions, which are generally weaker than those needed for relevance of the orthogonal moments, and which in turn are generally weaker than the ones required for (local and regular) identification of the parameter of interest. Lack of identification of the parameter of interest will affect the power properties of testing procedures based on orthogonal moments, as we show. A non-zero efficient Fisher Information matrix for the parameter of interest suffices for RLN and it is necessary and sufficient for informative orthogonal moments. In comparison, full rank of the efficient Fisher Information matrix is known to be required for regular identification (cf. Rothenberg (1971) and Escanciano (2022)). In summary, this discussion highlights that the mild conditions for the existence of orthogonal moments differ from the slightly stronger conditions for relevance. In this paper, we thoroughly investigate and characterize both sets of conditions.

Essentially, our results apply to any regular semiparametric model. To emphasize the broad applicability of our results, we work through two popular settings in econometrics: models with Unobserved Heterogeneity (UH) and models defined by conditional moment restrictions with possibly different conditioning variables. Orthogonal moments in these settings have been less explored in the debias/LR literature, and in particular, are not treated in CEINR. We understand UH as all those factors that are unobserved by the econometrician but whose presence cannot be disregarded. The existence of LR moments is particularly crucial in models with UH, and in the vast majority of models where nuisance parameters are not identified (as in, e.g., discrete choice models) or identified by ill-posed problems (see, e.g., Carrasco, Florens and Renault (2007)). Orthogonal moments are robust (at least locally) to identification failures of nuisance parameters, as well as to regularization biases typically present in ill-posed settings. This is precisely our motivation to study orthogonal moments in such models.

An important special case of models with UH is non-linear panel data. This literature has been particularly interested in carrying out flexible estimation and robust inference in the presence of UH. The usual and simplest approach to inference consists of constructing moments that do not depend on the nuisance parameter (here, the density of UH), referred to as Nuisance-Free (NF) moments in this paper. In important work, Bonhomme (2012) developed a systematic method to construct NF moments for fixed parameters by functional differencing. He found a condition, closely related to RLN, that is necessary but not sufficient for the estimation of fixed parameters with NF moments. We characterize LR in this setting, and we show that the support of UH plays an important role in this characterization. We spell out additional smoothness and support conditions under which the sufficient condition for NF moments in Bonhomme (2012) becomes necessary for LR, thereby showing that all orthogonal moments are NF under the derived conditions. Thus, the proposed characterization of LR when applied to nonlinear panel data and under some additional smoothness and support conditions, delivers functional differencing as a special case. The characterization of LR moments for general parameters of interests, such as moments of the distribution of UH, is more involved. We show that when orthogonal moments are informative, one can use existing functional differencing techniques to construct partially robust moments that depend only on the fixed parameter and the parameter of interest. We also explain how these partially robust moments can be converted into fully LR moments, without the need to estimate the distribution of UH.

In addition, we consider models of conditional moment restrictions with possibly different conditioning variables. Over-identification and estimation in these models have been studied in Chen and Santos (2018) and Ai and Chen (2007, 2012), respectively, but the existence and relevance of orthogonal moments for general parameters have not been derived under this generality, to the best of our knowledge. A main implication of our work with these models (and of our paper) is that orthogonality might hold in more general settings than previously recognized. In particular, we show that exclusion restrictions are not necessary for the existence of orthogonal moments of structural parameters in semiparametric sample selection models, but they are necessary for the commonly used partly linear model with an endogenous variable. As special novel applications, we give orthogonal moments for the fully saturated two-stage least squares (2SLS) of Angrist and Imbens (1995, Theorem 3), for heterogeneous parameters in treatment effect models, for sample selection models, and for the fixed parameters and moments of the nonparametric UH in the popular demand model for differentiated products in Berry, Levinsohn and Pakes (1995, hereafter BLP). For the influential 2SLS of Angrist and Imbens (1995), our orthogonal moments provide a machine learning data-driven alternative to the often criticized method of running separate first stage regressions for each covariate value, see S\l oczy\'{n}ski (2020). In the case of treatment effects, we propose a new and simple locally robust inference method for parameters associated with interactions terms between covariates and the treatment. These parameters are of great interest in applied work. Both the fully saturated 2SLS and the LR method for heterogenous parameters are examples of Orthogonal-Relevant (OR) moments, which we introduce in full generality in Section (ref). We apply our results to the Oregon Health Experiment to study the presence of heterogeneous treatment effects of Medicaid on several health outcomes of interest. We also develop a LR estimation method in this case allowing for high-dimensional control variables.

We stress that these applications are just special cases of a general theory that we develop in this paper. We contribute to the recent and existing literature on LR/Orthogonal/Debiased moments by extending its scope of applications to other regular semiparametric models (such as models with UH) and to general smooth functionals (parameters) of interest.

The rest of the paper is organized as follows. Section (ref) introduces the setting, concepts and the first main result. Section (ref) applies the general theory to the class of models with UH. These first two sections focus on the structural fixed parameter as the parameter of interest for ease of exposition, but the core and main contribution of the paper is Section (ref), which shows how the results are extended to general models and general parameters of interest. Section (ref) discusses the application to the class of conditional moment restrictions, including the application to heterogeneous treatment effect parameters. Finally, Section (ref) concludes. An Appendix contains the proofs of the main results. A Supplementary Appendix gathers the applications to the sample selection models and the BLP model, Monte Carlo simulations for the heterogeneous treatment effects, some descriptive statistics of our empirical application, and further discussion and extensions.

Restricted Local Surjectivity and Local Robustness

Preliminaries

Setting

The data is an independent and identically distributed (iid) sample $Z_{1},...,Z_{n}$ from a distribution $\mathbb{P}_{0}$ that belongs to a semiparametric model $\mathcal{P}=\{\mathbb{P}_{\lambda}:\lambda=(\theta ,\eta)\in\Lambda\equiv\Theta\times\Xi\},$ where $\Theta\subset\mathbb{R}^{p}$ and $\Xi$ is an arbitrary subset of a typically infinite-dimensional space. The parameters that generated the data are denoted by $\left( \theta_{0} ,\eta_{0}\right) \in\Theta\times\Xi,$ i.e. $\mathbb{P}_{0}=\mathbb{P} _{\theta_{0},\eta_{0}}.$ The parameter of interest is a vector of functionals $\psi(\lambda_{0})\in\mathbb{R}^{d_{\psi}},$ although for the sake of exposition we focus first on $\psi(\lambda_{0})=\theta_{0},$ denoted as the structural parameter, with $d_{\psi}=p,$ and where we treat $\eta_{0}$ as an unknown nuisance parameter. For example, $\eta_{0}$ could be the unknown density of UH or an unknown function of control variables.\footnote{The results on existence of orthogonal moments also hold for an infinite-dimensional parameter of interest. We focus on finite-dimensional parameters of interest for ease of exposition.} Consider the restricted model $\mathcal{P}_{0}=\{\mathbb{P}_{\theta_{0},\eta}:\eta\in\Xi\}$ that assumes $\theta_{0}$ is known. Surjectivity refers to the situation where the set of scores of the model $\mathcal{P}_{0},$ defined formally below, can approximate arbitrary well any zero-mean square-integrable function of the data $Z_{i}$. This spanning condition, which can be considered an appealing one from a fitting point of view, will have important implications for the existence of LR moments.

The problem we address in this paper is inference about $\theta_{0}$ (for the case of a general $\psi(\lambda_{0}),$ see Section (ref))$,$ in the presence of an unknown, possibly high-dimensional, nuisance parameter $\eta_{0}$ (respectively, $\lambda_{0}$). That is, we aim at testing the hypotheses \[ H_{0}:\theta_{0}=\bar{\theta}\qquad vs\qquad H_{1}:\theta_{0}\neq\bar{\theta }, \] for a known $\bar{\theta},$ e.g. $\bar{\theta}=0,$ when $\eta_{0}$ is unknown under both $H_{0}$ and $H_{1}.$

Inference is based on a $k$-dimensional vector of moments, $k\in\mathbb{N},$ constructed such that under the null hypothesis:

where $Z$ is an independent copy of $Z_{i},$ $\mathbb{E}$ denotes expectation under $\mathbb{P}_{0},$ and $g(\cdot)$ is a measurable function with finite variance. A natural example of $g$ is the score of the model $\mathcal{P}$ with respect to $\theta$ at $\bar{\theta},$ often computed as

where $f_{\theta,\eta}$ is the density of $\mathbb{P}_{\theta,\eta}$ with respect to a $\sigma$-finite measure $\mu$. The score satisfies ((ref)) with $k=p,$ but it will not be in general an orthogonal moment, as we define it below.

Informal Overview of the Results

Local robustness or orthogonality refers to the situation where the moment $\mathbb{E}\left[ g\left( Z,\theta_{0},\eta\right) \right] $ is locally insensitive to deviations of $\eta$ from the truth $\eta_{0}.$ We characterize when such moments exist and when they are informative about $\theta_{0}$ (or more generally, about a parameter of interest $\psi(\lambda_{0})).$ To demonstrate the utility of our results we work through two popular settings in econometrics: models with UH and models defined by conditional moment restrictions with possibly different conditioning variables. We next explain the main contributions within these two applications.

Consider first a model with nonparametric UH given by $\alpha$ and covariates $X,$ where the functional form of the conditional distribution of $Y$ given $\alpha$ and $X$ is known up to the parameter $\theta_{0}$, as in Bonhomme (2012). We show that an orthogonal function $g$ for $\theta_{0}$ necessarily solves

This condition illustrates the important role of the support of the distribution of UH. Under additional support conditions that we provide, we show that any solution to ((ref)) will only depend on $\theta_{0}$ and not on $\eta_{0},$ i.e. $g(Z,\theta_{0},\eta_{0})\equiv g(Z,\theta_{0}).$ Suppose now that the parameter of interest is a moment of covariates and the distribution of UH $\psi(\lambda_{0})=\mathbb{E}\left[ r(X,\alpha)\right] $.\footnote{The function $r(X,\alpha)$ does not depend on $\theta_{0}.$ For the general case, see Section (ref).} We show in Section (ref) that an orthogonal function $g$ for $\psi(\lambda_{0})$ satisfies, for a constant $C,$

Under the same support conditions as for $\theta_{0}$, any solution in $g$ to ((ref)) will depend on $\theta_{0},$ and on $\eta_{0}$ only through $\psi(\lambda_{0}),$ i.e. $g(Z,\theta_{0},\eta_{0})\equiv g(Z,\theta_{0} ,\psi(\lambda_{0})).$ If $\partial\mathbb{E}\left[ g(Z,\theta_{0} ,\psi(\lambda_{0}))\right] /\partial\theta=0,$ these moments are orthogonal for $\psi(\lambda_{0})=\mathbb{E}\left[ r(X,\alpha)\right] ,$ i.e., they will be insensitive to deviations from both $\theta_{0}$ and $\eta_{0}$ satisfying that $\psi(\lambda_{0})$ remains constant at the true value. Otherwise, they are only partially orthogonal. They may be useful in any case, as inference and estimation of $\psi(\lambda_{0})$ is facilitated by the moment depending only on finite-dimensional parameters. These moments can be combined with functional differencing moments for $\theta_{0}$ in Bonhomme (2012) to provide estimation and inference on the parameter of interest $\psi(\lambda_{0}),$ without the need to estimate the distribution of UH. These are special cases of a more general theory for smooth functionals $\psi(\lambda_{0})$ in Section (ref).

As another application of the general theory, consider now models defined by $J$ moment functions $\rho_{j}(Z,\theta_{0},\eta_{0})$ with possibly distinct conditioning variables $W_{j}$, $j=1...,J.$ This setting is quite general and includes models with endogeneity. Orthogonal functions in this setting are obtained from linear combinations of the moment functions with coefficients given by suitable transformations of the conditioning variables $W_{j},$ referred to as \textquotedblleft orthogonal instruments\textquotedblright\ in this paper when the $W_{j}^{\prime}s$ involve Instrumental Variables (IV), i.e. \[ g(Z,\theta_{0},\eta_{0})= {\textstyle\sum\nolimits_{j=1}^{J}} \rho_{j}(Z,\theta_{0},\eta_{0})\varphi_{j}(W_{j}), \] for suitable $\varphi_{j}^{\prime}s$ that we characterize and which depend on the parameter of interest $\psi(\lambda_{0}).$ We illustrate with several examples that relevance of the moment $\mathbb{E}\left[ g(Z,\theta_{0} ,\eta_{0})\right] $ does not require to calculate the efficient score in this model, which is a rather complicated task (cf. Ai and Chen (2012)). We introduce Orthogonal-Relevant IVs (OR-IVs) as those $\varphi_{j}^{\prime}s$ that are orthogonal and guarantee relevance under a minimal condition. We characterize OR-IVs in general, and show how they depend on the functional of interest. We find OR-IVs for some useful examples.

As concrete applications of these results, we provide new OR-IVs for the partial linear model with endogeneity, including an orthogonal version of the fully saturated 2SLS of Angrist and Imbens (1995) which reduces model selection biases from fitting models with high-dimensional controls (see ((ref))). We characterize orthogonal moments for heterogeneous parameters in treatment effect models and for sample selection models. Combining the results for UH and conditional moment models, we provide orthogonal moments for the fixed parameters and moments of the nonparametric UH in the popular demand model for differentiated products in BLP. An empirical application to the Oregon Health Experiment obtains locally robust estimation and inferences on heterogenous parameters on the use of Medicaid on several health outcomes.

These applications can be seen as specific instances of a broader theory discussed in Sections (ref) and (ref). Given the abstract nature of these results, we proceed from the simple to the more complicated settings. Following this logic, we begin with the well-known parametric setting.

LR moments in the parametric setting

For expositional purposes, we first motivate the general problem in the parametric case studied in the landmark contribution by Neyman (1959). Readers already familiar with these notions can skip this section. Consider the situation where $\Xi\subset\mathbb{R}^{q}$ is endowed with the Euclidean inner product $\langle u,v\rangle=u^{\prime}v$ $(A^{\prime}$ denotes the transpose of the matrix $A)$. Neyman (1959) investigated conditions under which, for a given $\sqrt{n}-$consistent estimator $\hat{\eta}$ of $\eta_{0},$ the asymptotic distribution of the sample moment $\mathbb{E}_{n}\left[ g\left( Z,\bar{\theta},\hat{\eta}\right) \right] $ will not depend on that of the first step, $\sqrt{n}(\hat{\eta}-\eta_{0}),$ where henceforth $\mathbb{E} _{n}\left[ h\left( Z\right) \right] =n^{-1}\sum_{i=1}^{n}h\left( Z_{i}\right) \ $denotes the sample mean operator. A simple Taylor argument yields, under standard regularity conditions, the expansion \[ \sqrt{n}\mathbb{E}_{n}\left[ g\left( Z,\bar{\theta},\hat{\eta}\right) \right] =\sqrt{n}\mathbb{E}_{n}\left[ g\left( Z,\bar{\theta},\eta _{0}\right) \right] +\frac{\partial\mathbb{E}\left[ g\left( Z,\bar{\theta },\eta_{0}\right) \right] }{\partial\eta^{\prime}}\sqrt{n}(\hat{\eta} -\eta_{0})+o_{P}(1). \] Neyman's (1959) sufficient condition for asymptotic invariance to the distribution of $\sqrt{n}(\hat{\eta}-\eta_{0})$ was local robustness, in the sense of \[ \frac{\partial\mathbb{E}\left[ g\left( Z,\bar{\theta},\eta_{0}\right) \right] }{\partial\eta}=0. \] To achieve this goal, Neyman (1959) used a moment function $g$ sufficiently regular (termed a Cramer function) so that the following generalized information equality holds for $j=1,...,q,$

where $s_{\eta,j}\left( Z,\bar{\theta},\eta_{0}\right) =\partial\log f_{\theta,\eta_{0}}\left( Z,\bar{\theta},\eta_{0}\right) /\partial\eta_{j}$ and $\eta_{j}$ is the $j-th$ component of $\eta.$ By ((ref)), locally robust moments are moments orthogonal to the scores $s_{\eta,j}$, and they can be constructed starting from any original moment function $m(\cdot)$ by performing a least squares projection to obtain the \textquotedblleft least squares residual\textquotedblright\

where $\beta_{m}=\mathbb{E}\left[ s_{\eta}s_{\eta}^{\prime}\right] ^{-1}\mathbb{E}\left[ s_{\eta}m^{\prime}\right] $ and $s_{\eta} \equiv(s_{\eta,1},...,s_{\eta,q})^{\prime}.$ The term $\beta_{m}^{\prime }s_{\eta}\left( Z,\bar{\theta},\eta_{0}\right) $ is the orthogonal projection of $m$ onto the linear space generated by the scores $s_{\eta,j},$ $j=1,...,q$---the so-called tangent space of nuisance parameters. Existence of orthogonal moments was not an issue in the parametric case of Neyman (1959), because the tangent space of nuisance parameters is finite dimensional (of dimension at most $q$)$,$ while the space of zero-mean square-integrable $k-$valued functions has infinite dimension for a continuous $Z,$ creating a surplus of orthogonal moments.\footnote{For a discrete $Z$ taking $r$ distinct values, the space of zero-mean $k-$valued square-integrable moments has dimension $k(r-1),$ see Tsiatis (2006, pg. 12). Thus, orthogonal moments exist as soon as $k(r-1)>q.$} The main insight of our paper is that this may not be the case for nonparametric models.\footnote{Semiparametric extensions of the C($\alpha$) test have been considered in Choi, Hall and Schick (1996), and more recently, in Lee (2022), for strongly identified nuisance parameters. However, these papers focus on likelihood settings and do not study the existence of orthogonal moments. Escanciano (2012) proposed a semiparametric C($\alpha$) test for general moment restriction models, but he did not investigate existence.}

Notation, regularity conditions, and relevant definitions

Let again $f_{\theta,\eta}$ denote the density of $\mathbb{P}_{\theta,\eta}$ with respect to a $\sigma$-finite measure $\mu$, and let $f_{0}\equiv f_{\theta_{0},\eta_{0}}$ be the true density. Let $L_{2}\equiv L_{2}(f_{0})$ denote the Hilbert space of $\mathbb{P}_{0}-$square integrable measurable functions with inner product $\langle h,f\rangle=\int hfd\mathbb{P} _{0}=\mathbb{E}\left[ h\left( Z\right) f\left( Z\right) \right] $ and norm $\left\Vert h\right\Vert ^{2}=\langle h,h\rangle=\mathbb{E}\left[ h^{2}\left( Z\right) \right] $. The set $L_{2}^{0}\equiv L_{2}^{0}(f_{0})$ is the subspace of zero-mean functions in $L_{2},$ i.e. $h\in L_{2}$ with $\mathbb{E}\left[ h\left( Z\right) \right] =0.$ More generally, $L_{2}(f)$ and $L_{2}^{0}(f)$ are defined analogously for any density $f.$

A parametric sub-model in the restricted model is a path $\tau\in \lbrack0,\varepsilon)\mapsto\mathbb{P}_{\tau}\equiv\mathbb{P}_{\theta_{0} ,\eta_{\tau}}\in\mathcal{P}_{0},$ $\varepsilon>0,$ which satisfies the so-called Differentiability in Quadratic Mean (DQM)

where $f_{\tau}$ is density of $\mathbb{P}_{\tau}$. We drop the dependence of the path on $s_{\eta}$ for simplicity of notation. The function $s_{\eta}$ is the score of the path $\mathbb{P}_{\tau}$, in most cases $s_{\eta}=d\log f_{\theta_{0},\eta_{\tau}}/d\tau,$ where henceforth $d/d\tau$ is the derivative from the right (i.e. for nonnegative values of $\tau$) at $\tau=0.$ An implication of ((ref)) is that $s_{\eta}\in L_{2}^{0}.$ Note that these are scores under the null hypothesis.

Define the tangent space of nuisance parameters as \[ T_{0}=\{s_{\eta}\in L_{2}^{0}:(\ref{msd})\text{ holds}\}. \] Let $\overline{T_{0}}$ denote the closure of $T_{0}$ in the mean-square norm topology. Our semiparametric models are regular in the sense that $T_{0}$ is a linear subspace. Regularity and DQM are standard assumptions in the semiparametric efficiency theory (see, e.g., Newey (1990)). Also, for future reference, we introduce the orthocomplement of $\overline{T_{0}}$ as follows\footnote{Note that if $\mathbb{E}\left[ g\left( Z\right) s_{\eta }\left( Z\right) \right] =0$ for all $s_{\eta}\in T_{0},$ then $\mathbb{E}\left[ g\left( Z\right) s_{\eta}\left( Z\right) \right] =0$ for all $s_{\eta}\in\overline{T_{0}},$ by continuity of the inner product.} \[ \overline{T_{0}}^{\perp}=\{g\in L_{2}:\mathbb{E}\left[ g\left( Z\right) s_{\eta}\left( Z\right) \right] =0\text{ for all }s_{\eta}\in T_{0}\}. \] As in Neyman (1959), we require regularity conditions on the moments to be able to interchange derivatives and moments. Let $\mathbb{E}_{\tau}$ denote the expectation under the path $\mathbb{P}_{\tau}.$

These regularity conditions are standard in the semiparametric literature, see, e.g., Newey (1990, 1994). Condition ((ref)) holds, for example, when $g$ is bounded. More generally, suppose \[ g(Z,\theta,\eta)=m(Z)- {\textstyle\int} m(z)f_{\theta,\eta}(z)d\mu(z), \] for a measurable moment $m.$ Then, $g\in\mathcal{G}_{0}$ provided for all paths ((ref)) \[ \lim\sup_{\tau\downarrow0} {\textstyle\int} m^{2}(z)f_{\tau}(z)d\mu(z)<\infty; \] see Ibragimov and Khasminskii (1981, Lemma 7.2, pg. 67). The following condition is a maintained regularity assumption throughout the paper when referring to moment functions.

Whether orthogonal moments exist or not in semiparametric models will depend on a condition that we introduce. We work under the null hypothesis, so all our definitions apply to $\theta_{0}=\bar{\theta}.$ We will investigate power in Section (ref).

The intersection with $\mathcal{G}_{0}$ in the definition of RLN refers to the aforementioned maintained regularity condition. We aim to relate the concept of non-surjectivity with that of LR/orthogonal\ moments, so we introduce formally the latter.\footnote{For a related definition of LR in CEINR and the relation to the present definition see Section (ref).}

Existence of Orthogonal Moments

The implications of RLN on robustness can be seen as follows. For any score function $g$ and parametric submodel under the null hypothesis it holds, for $\varepsilon>0,$

Differentiating this equation, we obtain by the chain rule and ((ref))

Local robustness further implies that the second term is zero, so the first term must be zero for all scores $s_{\eta}$ in $\overline{T_{0}}.$ From this equation it follows:

Equation ((ref)) explains the well-known link between orthogonality and LR (see, e.g. Newey (1990)). Theorem (ref) is novel in deriving a necessary and sufficient condition for existence of orthogonal moments based on this fundamental relation.

RLN and LR moments based on influence functions

Our definition of LR/orthogonality is model-based, while the construction of orthogonal moments in CEINR is estimation-based. These two approaches are related, as we now show. Suppose $\hat{\eta}$ is a consistent estimator for $\eta_{0}$ in the model $\mathcal{P}_{0}.$ Let $\eta(\mathbb{P})$ denote the probabilistic limit of $\hat{\eta}$ under general misspecification, i.e., when the data distribution is given by $\mathbb{P},$ consistent with $\eta_{0} =\eta(\mathbb{P}_{0})$. The First Step Influence Function (FSIF) $\phi^{LR}$ pertaining to an original moment function $m$ and $\eta(\mathbb{P})$ is defined in CEINR as the unique function $\phi^{LR}$ with zero-mean and finite variance satisfying

for all paths $\mathbb{P}_{\tau},$ $\eta_{\tau}=\eta(\mathbb{P}_{\tau}),$ with corresponding scores $s_{\eta}(Z)$ such that their linear span is dense in $L_{2}^{0}.$ Having scores with this property is the precise meaning of \textquotedblleft misspecification\textquotedblright\ considered first in Newey (1994). Under these conditions, CEINR showed in great generality that $g^{LR}=m+\phi^{LR}$ is a LR moment. Their definition is estimation-based because it is relative to the mapping $\eta(\mathbb{P})$ defined by the estimator $\hat{\eta},$ while ours is model-based because it is relative to $\mathcal{P}.$

A key observation of our paper is that as long as $\hat{\eta}$ is a consistent estimator for $\eta_{0}$ in a model satisfying RLS, the precise meaning of misspecification in Newey (1994) holds. Thus, replacing $g$ by $m$ in ((ref)) and using the definition of $\phi^{LR}$ in ((ref)), it follows that $\phi^{LR}=-m,$ thereby leading to a zero LR moment in the construction proposed by CEINR for all identifying moment functions. We state the result in the next Proposition.

We illustrate Proposition (ref) with a running example. This example serves to demonstrate how our results above are also applicable to models defined by conditional moment restrictions.

\noindentExample 1: Partly Linear Model with Endogeneity. Consider a data observation $Z=(Y,W)$ where $Y=(Y_{1},Y_{2})$ are endogenous variables and $W=(X,Z_{2})$ are exogenous variables, with $X\ $controls and $Z_{2}$ Instrumental Variables (IV), following the model \[ Y_{1}=\theta_{0}Y_{2}+\eta_{0}\left( X\right) +\varepsilon,\text{ }\mathbb{E}\left[ \left. \varepsilon\right\vert W\right] =0\text{ a.s.} \] This is the standard partly linear model (Robinson (1988)) commonly used in applied work, possibly with a flexible high-dimensional specification for $\eta_{0}$. Suppose the researcher applies CEINR with an original identifying moment function given by the IV moment function with instrument $Z_{2},$ i.e. $m(Z,\theta_{0},\eta_{0})=(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left( X\right) )Z_{2},$ and a consistent nonparametric first step estimator $\hat{\eta}$ for $\eta(\mathbb{P})=\mathbb{E}_{\mathbb{P}}\left[ \left. Y_{1}-\theta_{0} Y_{2}\right\vert X\right] $ to do inference on the structural parameter $\theta_{0}$. CEINR considered paths $\mathbb{P}_{\tau}$ with CDF $F_{\tau }=(1-\tau)F_{0}+\tau G$ for $\tau\in\lbrack0,1],$ where $F_{0}$ denotes the CDF of $\mathbb{P}_{0},$ and $G$ is some alternative distribution$.$ We assume that $G$ is chosen so that $\eta_{\tau}\left( X\right) =\mathbb{E}_{\tau }\left[ \left. Y_{1}-\theta_{0}Y_{2}\right\vert X\right] $ exists and possibly other regularity conditions are satisfied (such as existence and square-integrability of scores). The corresponding FSIF follows from taking derivatives in the orthogonality condition \[ \mathbb{E}_{\tau}\left[ (Y_{1}-\theta_{0}Y_{2}-\eta_{\tau}\left( X\right) )\mathbb{E}\left[ \left. Z_{2}\right\vert X\right] \right] =0, \] so that by $\mathbb{E}\left[ (Y_{1}-\theta_{0}Y_{2})\right] $ not depending on $\tau$ and iterated expectations

The FSIF of CEINR is thus $\phi^{LR}=-(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left( X\right) )\mathbb{E}\left[ \left. Z_{2}\right\vert X\right] $ and the LR moment of CEINR boils down to the Double-Machine-Learning (DML) approach of Chernozhukov et al. (2018), $g^{LR}=m+\phi^{LR},$ i.e. \[ g^{LR}(Z,\theta_{0},\eta_{0})=(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left( X\right) )\left( Z_{2}-\mathbb{E}\left[ \left. Z_{2}\right\vert X\right] \right) . \] If there are no exclusion restrictions, in the sense that $Z_{2}\subset X,$ then $\mathbb{E}\left[ \left. Z_{2}\right\vert X\right] =Z_{2},$ $\phi ^{LR}=-m$ and $g^{LR}\equiv0.$ Indeed, one can show, see Section (ref) for further discussion, that \[ \overline{T_{0}}^{\perp}=\{g\in L_{2}^{0}:g\left( Z\right) =(Y_{1} -\theta_{0}Y_{2}-\eta_{0}\left( X\right) )\left( \zeta(W)-\mathbb{E}\left[ \left. \zeta(W)\right\vert X\right] \right) ,\text{ }\zeta\in L_{2}\}. \] It is then clear that $\overline{T_{0}}^{\perp}=\{0\}$ iff $W=X.$ Exclusion restrictions, in the sense of existence of an exogenous variable distinct from $X,$ are sufficient and necessary for the existence of LR moments. Exclusion restrictions are a minimal requirement for inference on $\theta_{0}$ in this example. $\square$

Information of Orthogonal Moments

Next, we ask: when are orthogonal moments informative? The concepts of RLS and RLN involve only the null hypothesis. This corresponds to our desire to have tests with controlled size under a general set of circumstances such us with high-dimensional nuisance parameters. We now turn to local power considerations. To that end, we also need to incorporate paths that change the parameter of interest. We consider paths $\tau\in\lbrack0,\varepsilon )\mapsto\mathbb{P}_{\tau}\equiv\mathbb{P}_{\lambda_{\tau}}\in\mathcal{P},$ $\lambda_{\tau}=(\theta_{\tau},\eta_{\tau}),$ satisfying DQM

where again $f_{\tau}$ is density of $\mathbb{P}_{\tau}$ and $s\in L_{2}^{0}$ is the score of the path. By the chain rule, the score of the model $s$ has the representation $s=\delta^{\prime}s_{\theta}+s_{\eta},$ where $s_{\eta}$ is the score for the nuisance parameter $\eta,$ $\delta\in\mathbb{R}^{p}$ is a vector of constants, and $s_{\theta}$ is the score for the parameter of interest, often obtained as in ((ref)). Similarly to the restricted model, define the full tangent space of the model as \[ T=\{s\in L_{2}^{0}:(\ref{QMD})\text{ holds}\}. \] Let $\overline{T}$ denote the closure of $T$ in the mean-square norm topology.

We define Cramer moments with respect to the full parameter.

The power of score-based tests comes from the non-zero mean property at values $\theta_{0}\neq\bar{\theta}.$ For any score function $g$ and parametric submodel within the model it holds, for $\varepsilon>0,$

Taking derivatives in this equation and, if, in addition, $g$ is a Cramer orthogonal moment, then by $s=\delta^{\prime}s_{\theta}+s_{\eta}$ and orthogonality

where $\tilde{s}_{\theta}(Z)$ is the so-called efficient score, the projection of $s_{\theta}$ onto $\overline{T_{0}}^{\perp}$, i.e. \[ \tilde{s}_{\theta}(Z)=s_{\theta}(Z)-\Pi_{\overline{T_{0}}}s_{\theta}(Z), \] where $\Pi_{V}$ denotes the orthogonal projection operator onto the closed subspace $V.$ Several important implications follow from ((ref)). First, the slope in ((ref)) is maximized in absolute value (and subject to a normalization) by $g=\delta^{\prime}\tilde{s}_{\theta}.$ Under RLS, $\overline{T_{0}}^{\perp}=\{0\}$ and hence $\tilde{s}_{\theta}=0.$ Therefore, RLN is necessary for the local power to be non-trivial. Second, $\tilde {s}_{\theta}$ may be zero even under RLN. Thus, RLN, and hence existence of orthogonal moments, may not be sufficient for non-trivial local power, i.e. for orthogonal moments to be informative about the parameter of interest, as our running example illustrates.

\noindentExample 1: Partly Linear Model with Endogeneity, cont. Orthogonal moments are \[ g\left( Z,\theta_{0},\eta_{0}\right) =(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left( X\right) )\left( \zeta(W)-\mathbb{E}\left[ \left. \zeta(W)\right\vert X\right] \right) ,\text{ }\zeta\in L_{2}. \] For a path such that $\theta_{\tau}=\bar{\theta}+\tau\delta+o(1),$ the power of the orthogonal moment-based test is determined by

Note how RLN is necessary for the local power to be non-trivial: the slope is zero if there are no exclusion restrictions and $W=X.$ However, RLN is not sufficient for non-trivial local power. The slope will be also zero if, conditional on $X,$ $Y_{2}$ is mean independent of $Z_{2}.$ This is the classical IV relevance condition. It is important to note that we do not need to compute $\tilde{s}_{\theta}$ in this example to claim relevance of orthogonal moments (cf. ((ref))). The expression for $\tilde{s}_{\theta }$ is complicated even in the exogenous case, see Chamberlain (1992) and Section 2.2.4 in Chernozhukov et al. (2018). For the endogenous case see Ai and Chen (2003). $\square$

Our results can be used to select suitable orthogonal instruments.

\noindentExample 1: Partly Linear Model with Endogeneity, cont. From ((ref)), it follows that $\zeta(W)=\zeta^{\ast}(W)=\mathbb{E} \left[ \left. Y_{2}\right\vert W\right] -\mathbb{E}\left[ \left. Y_{2}\right\vert X\right] $ guarantees non-trivial local power whenever $\zeta^{\ast}(W)\neq0$. Furthermore, it is worthy to note that the commonly recommended choice $\zeta(W)=Z_{2}$ may not lead to informative orthogonal moments, while the IV $\zeta^{\ast}(W)$ may still do. As an illustration, when $Z_{2}$ is a binary IV, say $Z_{2}\in\{0,1\},$ then $\zeta^{\ast}(W)=0$ a.s. is equivalent to $Cov\left[ \left. Y_{2},Z_{2}\right\vert X\right] =0$ a.s. (see Proposition 3.1. in Caetano and Escanciano (2021)). That is, for identification the IV needs to be correlated with the endogenous variable conditional on the controls. Clearly, $Cov\left[ \left. Y_{2},Z_{2} \right\vert X\right] =\mathbb{E}\left[ \left. \zeta^{\ast}(W)Z_{2} \right\vert X\right] =0$ implies $\mathbb{E}\left[ (\zeta^{\ast} (W))Z_{2}\right] =\mathbb{E}\left[ (\mathbb{E}[Y_{2}|W]-\mathbb{E}\left[ Y_{2}|X\right] )Z_{2}\right] =0,$ but the reciprocal is generally not true. Hence, our results on relevance can be used to select informative orthogonal moments over commonly used IV procedures. The orthogonal instrument $\zeta^{\ast}(W)$ is relevant under the minimal identification condition, and as such, it is referred to as an OR-IV. For a general definition of OR-IV see Section (ref). $\square$

An important point of this paper is that the conditions for the existence of orthogonal moments (i.e., RLN) are separate from the conditions needed for orthogonal moments to be informative, i.e., corresponding moment-based tests to have non-trivial asymptotic local power. If $\tilde{s}_{\theta}=0$, orthogonal moments may exist, but they will have trivial local power (cf. ((ref))). For orthogonal moments to be informative, it is additionally required that $\tilde{s}_{\theta}\neq0$. Importantly, when $\theta$ is univariate, as in Example 1, relevance is equivalent to (regular and local) identification of $\theta_{0}$. Some estimators may achive identification under the minimal relevance condition, but may not be orthogonal. We recommend estimators achieving both properties.

\noindentExample 1: Partly Linear Model with Endogeneity, cont. In this model $\theta_{0}$ is identified iff $\zeta^{\ast}(W)\neq0.$ Indeed, two identifying results are

and

where for a generic variable $V,$ $\tilde{V}=V-\mathbb{E}\left[ \left. V\right\vert X\right] .$ Equation ((ref)) corresponds to the fully saturated 2SLS estimand of Angrist and Imbens (1995, Theorem 3), while Equation ((ref)) has been used in Syrgkanis et al. (2019, Section 2.1). However, the moments ((ref)) and ((ref)) are not LR. To propose a LR moment, we use our characterization, $\mathbb{E}\left[ \zeta^{\ast }(W)|X\right] =0$ a.s., and the identification result $\eta_{0}\left( X\right) =\mathbb{E}\left[ \left. Y_{1}-\theta_{0}Y_{2}\right\vert X\right] $ to obtain an orthogonal moment which achieves identification under the minimal relevance condition,

This orthogonal moment is simpler than optimal IV estimation with controls, in that it does not require to estimate conditional variances. The parameter $\theta_{0}$ from the orthogonal moment ((ref)) has a convenient nonparametric interpretation in terms of conditional LATEs under a weak monotonicity condition, see S\l oczy\'{n}ski (2020). An estimator based on ((ref)) provides a LR version of the fully saturated 2SLS of Angrist and Imbens (1995). We recommend inference based on this moment, as it can be easily implemented by machine learning methods, e.g. Lasso, Random Forest, Boosting, etc., for estimating the conditional means $r_{1}(X)=\mathbb{E} \left[ \left. Y_{1}\right\vert X\right] ,$ $r_{2}(X)=\mathbb{E}\left[ \left. Y_{2}\right\vert X\right] $ and the long regression $\mu (W)=\mathbb{E}\left[ \left. Y_{2}\right\vert W\right] $. Let $\hat{Y} _{j}=Y_{j}-\hat{r}_{j}(X),$ $j=1,2,$ and $\hat{\zeta}^{\ast}(W)=\hat{\mu }(W)-\hat{r}_{2}(X)$ denote cross-fitted machine learners of $\tilde{Y}_{j} \ $and $\zeta^{\ast}(W),$ respectively. Then, an estimate for $\theta_{0}$ is an IV estimation of $\hat{Y}_{1}$ on $\hat{Y}_{2}$ with IV $\hat{\zeta}^{\ast }(W).$ This IV estimator can be implemented as a DML-IV with generated instruments $\hat{Z}_{2}=\hat{\mu}(W)$ in off-the-shelf statistical software. Asymptotic theory and inference with such LR estimator is justified by a straightforward application of CEINR. $\square$

Sections (ref) and (ref) generalize the ideas of Example 1 by providing a necessary and sufficient condition for orthogonal moments to be informative for general functionals and models without the need to compute the efficient score of the functional.

To give further insights into the power of tests based on orthogonal moments, the following representation is useful \[ \overline{T_{0}}^{\perp}=\tilde{S}_{\theta}\oplus\overline{T}^{\perp}, \] where $\tilde{S}_{\theta}$ is the linear span of $\tilde{s}_{\theta}$ and $\oplus$ denotes the direct sum. This is the decomposition of orthogonal moments in the informative and non-informative parts, as we now show. By our characterization of orthogonal moments, any orthogonal moment $g$ can be written as $g=c_{g}^{\prime}\tilde{s}_{\theta}+\Pi_{\overline{T}^{\perp}}g,$ where $c_{g}^{\prime}\tilde{s}_{\theta}$ is the orthogonal projection of $g$ onto $\tilde{S}_{\theta}\ $and $\Pi_{\overline{T}^{\perp}}g$ that onto $\overline{T}^{\perp}.$ The local power of the corresponding score test is fully determined by the coefficients $c_{g},$ the efficient Fisher Information matrix $\tilde{I}_{\theta}=\mathbb{E}[\tilde{s}_{\theta}(Z)\tilde{s}_{\theta }^{\prime}(Z)]$, and the direction of departure $\delta,$ through the slope ((ref)): \[ \frac{d}{d\tau}\mathbb{E}\left[ g(Z,\theta_{\tau},\eta_{0})\right] =-c_{g}^{\prime}\tilde{I}_{\theta}\delta\equiv-\langle c_{g},\delta \rangle_{\tilde{I}_{\theta}}, \] where $\langle\cdot,\cdot\rangle_{\tilde{I}_{\theta}}$ is the efficient Fisher inner product.

Indeed, by Le Cam's Third Lemma and ((ref)), under local alternatives $H_{1n}:\theta_{0}=\bar{\theta}+\delta n^{-1/2}$ and for a scalar LR\ moment \[ \sqrt{n}\mathbb{E}_{n}\left[ g(Z,\bar{\theta},\eta_{0})\right] \longrightarrow_{d}N\left( c_{g}^{\prime}\tilde{I}_{\theta}\delta ,\mathbb{E}\left[ g^{2}(Z,\bar{\theta},\eta_{0})\right] \right) . \] LR moments that are orthogonal to the full model are not informative about $\theta_{0},$ because if $g\in\overline{T}^{\perp}$ then $c_{g}=0,$ and hence the drift $c_{g}^{\prime}\tilde{I}_{\theta}\delta=0,$ regardless of the direction $\delta$ and the rank of $\tilde{I}_{\theta}.$ The part $\Pi_{\overline{T}^{\perp}}g$ of the orthogonal moment $g$ does not contribute to the relevance (the drift or signal), while increasing the asymptotic variance (the noise) of $\sqrt{n}\mathbb{E}_{n}\left[ g(Z,\bar{\theta} ,\eta_{0})\right] $. The drift is also zero if $\tilde{I}_{\theta}=0,$ i.e. $\tilde{s}_{\theta}=0$ a.s. On the contrary, if $\tilde{s}_{\theta}\neq0$ then we can select a $\delta\neq0$ and $g=\delta^{\prime}\tilde{s}_{\theta}$ such that $\delta^{\prime}\tilde{I}_{\theta}\delta>0$ (choose $\delta$ as the eigenvector associated to a positive eigenvalue of $\tilde{I}_{\theta}).$ Thus, orthogonal moments are informative whenever $\tilde{s}_{\theta}\neq0.$ For a detailed analysis of local power in the general multivariate case, see Section A of the Supplementary Appendix.

This discussion also explains why identification of the structural parameter may not be necessary for the existence and relevance of orthogonal moments. Relevance requires $\tilde{I}_{\theta}\neq0,$ while local regular identification requires full rank of $\tilde{I}_{\theta},$ see Rothenberg (1971) and Escanciano (2022) for parametric and semiparametric models, respectively. In the scalar case, $p=1$, singular and zero matrix are the same, but in the multivariate case $p>1,$ they are not. We illustrate this situation in Section B in the Supplementary Appendix by showing that exclusion restrictions are not necessary in sample selection models for orthogonal moments to exist and being informative about the structural parameter. This example is representative of large class of partially identified models for which this situation arises. Lack of identification of the structural parameter will do have an impact on power, though, as directions of identification failure, i.e. $\delta$ such that $\delta^{\prime}\tilde {s}_{\theta}(Z)=0,$ will also have zero slope by ((ref)). These directions correspond to zeros of the efficient information matrix $\tilde {I}_{\theta}=\mathbb{E}\left[ \tilde{s}_{\theta}(Z)\tilde{s}_{\theta} ^{\prime}(Z)\right] .$

We also note that identification of nuisance parameters is not required for the existence and relevance of orthogonal moments. Indeed, orthogonal moments are defined to be robust, at least locally, to such identification failures. In an independent work, Lee (2022) has also documented the robustness to identification failures in $\theta_{0}$ of efficient score tests in likelihood settings with strongly identified nuisance parameters. We generalize these results to non-likelihood settings, general parameters of interest, and possibly non-identified nuisance parameters. The orthogonal moment $\mathbb{E}\left[ g(Z,\theta_{0},\eta_{0})\right] $ may be identified without $\eta_{0}$ being identified. Section B of the Supplementary Appendix further discusses the relation between identification failures and existence of orthogonal moments, illustrating these insights with examples in sample selection models and measurement error models.

Models with Unobserved Heterogeneity

Basic Model and Results

In this section, we specialize the previous findings to an important class of models in econometrics, namely, models with Unobserved Heterogeneity (UH). We focus on models with a particular structure that commonly arises in settings with UH. We assume an iid sample ${(Z_{i},\alpha_{i})}$, where $\alpha_{i}$ denotes UH, a random vector of arbitrary dimension $d_{\alpha}$. In this framework, $\eta_{0}$ is the density of UH with respect to a $\sigma$-finite measure $\nu$. We assume regularity of the model, in the sense that there is a linear tangent space $T(\eta_{0})\subset L_{2}^{0}(\eta_{0})$ for UH such that the nuisance score has the representation

Regularity and DQM (cf. ((ref))) of this model has been shown under great generality, see, e.g., Le Cam and Yang (1988) and Lemma 25.34 in Van der Vaart (1998).

The conditional mean structure of $s_{\eta}$ will have important implications for orthogonality, as we now show. By iterated expectations

If all $s_{\eta}$ are obtained as ((ref)) for different $b^{\prime}s,$ $b\in T(\eta_{0}),$ then, orthogonality of $g$ is equivalent to orthogonality of $\mathbb{E}\left[ \left. g(Z,\bar{\theta},\eta_{0})\right\vert \alpha\right] $ to $T(\eta_{0}).$ In particular, a sufficient condition for orthogonality is

Such condition will be also necessary if $\mathbb{E}\left[ \left. g(Z,\bar{\theta},\eta_{0})\right\vert \alpha\right] $ belongs to the mean square closure of $T(\eta_{0}),$ as stated in the following results. Henceforth, the symbol $\perp$ denotes orthogonality in the corresponding Hilbert space inner product$.$

The following result applies when UH is nonparametric, i.e., when $T(\eta _{0})$ is dense in $L_{2}^{0}(\eta_{0})$.

Mixture Models

The previous results apply to the following general class of semiparametric mixture models

where the conditional density $f_{z|\alpha}\left( z|\alpha;\theta_{0}\right) $ is known up to the parameter $\theta_{0}\in\Theta\subset\mathbb{R}^{p},$ and again $\eta_{0}$ is the unknown density of the UH (with some abuse of notation we use the same $\alpha$ for denoting the random vector and its realization). Under mild smoothness conditions, scores of this model satisfy ((ref)) with $b(\alpha)=d\log\eta_{\tau}/d\tau$. If UH is nonparametric, then by Corollary (ref), a necessary and sufficient condition for LR\ is existence of $0\neq g\in L_{2}$ such that

If such a function $g$ exists, it may depend on $\eta_{0}$ through its support only, since $f_{z|\alpha}\left( z|\alpha;\theta_{0}\right) $ does not depend on $\eta_{0}$. The following example illustrates this point.

\noindentExample 2: Normal means model with general UH. Consider a data observation $Z_{i}=\alpha_{i}+\sqrt{\theta_{0}}u_{i},$ $u_{i}\sim N(0,1),$ $\alpha_{i}$ independent of $u_{i},$ with a density given by $\eta_{0}.$ This model has been extensively studied in the literature (see, e.g., Fan (1991) and Hall and Meister (2007)), often assuming $\theta_{0} =1\ $and that $\eta_{0}$ is an absolutely continuous density with respect to the Lebesgue measure on $\mathbb{R}$. The necessary and sufficient condition for existence of a LR\ moment in a general version of this example where UH is not necessarily absolutely continuous is \[ \int g(z,\bar{\theta},\eta_{0})\phi(\left( z-\alpha\right) /\sqrt {\bar{\theta}})dz=0\text{ }\eta_{0}-a.s. \] where $\phi$ is the standard Gaussian PDF. If the true distribution of UH is discrete with, say, $J$ points of support $\alpha_{1},...,\alpha_{J}$, $J<\infty,$ then there exists a LR moment, but it will depend on $\eta_{0}$ through its support points. The construction of orthogonal moments follows as for the parametric case discussed in Section (ref). Moreover, we emphasize that it suffices for condition ((ref)) to hold $\eta_{0} -$a.s., in contrast to be valid for all $\alpha$. Indeed, as we show in Section B of the Supplementary Appendix, in this example there is no non-zero solution to ((ref)) when $\eta_{0}-a.s$ is replaced by all $\alpha \in\mathbb{R}$. $\square$

In many different settings, learning $\eta_{0}$ might not be trivial. Because of this, we now provide assumptions under which a LR moment $g$ will be free of the support of $\eta_{0}$. That is, assumptions under which the following stronger condition holds

where $\mathcal{A}$ is a known connected open set of $\mathbb{R}^{d_{\alpha} }.$

If there is a moment $g$ satisfying ((ref)), it will be necessarily NF since $f_{z|\alpha}\left( z|\alpha;\theta_{0}\right) $ does not depend on $\eta_{0}\ $and $\mathcal{A}$ is known. A real analytic function is an infinitely differentiable function such that its power expansion coincides with its Taylor series. As in the functional differencing literature, we assume in the next result that $\mathcal{A}$ contains the support of $\eta _{0}$, denoted by $\mathcal{S}.$

In the Supplementary Appendix (Section D) we give conditions for the left hand side of ((ref)) to be real analytic in exponential families. Many popularly used models satisfy the analytic property, including various forms of logit models, duration models, etc. Fox et al. (2012) also used analyticity, but for the different problem of identifying the distribution of UH in the mixed logit model. We note that in the univariate UH case the condition $U\subset\mathcal{S}$ can be relaxed to $\mathcal{S}$ containing a limit point rather than a whole open set (cf. Krantz and Parks (2002)).

Introducing covariates and functional differencing

The results of the previous section can be generalized to include covariates. Now, the data observation is $Z=(Y,X),$ where $Y$ is a vector of dependent variables, $X$ is a vector of covariates, and possibly other variables such as initial conditions. Let $\eta_{0}\left( \alpha|x\right) $ denote the conditional density of UH given covariates $X=x$, and let the density of the observed data be given by

In this specification $f_{\theta_{0},\eta_{0}}$ is the density of $\mathbb{P}_{0}$ with respect to $\mu=\mu_{Y}\times v_{X},$ where $\mu_{Y}$ is a $\sigma$-finite measure and $v_{X}$ is the probability measure of $X.$ The conditional density of heterogeneity $\eta_{0}\left( \alpha|x\right) $ depends on $x$ in an unrestricted way, as in a fixed effects approach.\footnote{This setting includes panel data applications with $Y_{i}=(Y_{i1},...,Y_{iT}),$ for a finite number of periods $T<\infty.$ The vector $X_{i}$ includes predetermined covariates $W_{i}=(W_{i1},...,W_{iT}),$ as well as possibly initial values, e.g., $Y_{i0}.$} Under mild smoothness conditions, scores of the nuisance parameter of this model satisfy

where $b(\alpha,x)=d\log\eta_{\tau}\left( \alpha|x\right) /d\tau$ is the conditional score and $f_{X}$ is the density of $v_{X}$ with respect to some $\sigma$-finite measure $\mu_{X}.$

As in the previous subsection, by iterated expectations \[ \mathbb{E}\left[ g(Z,\theta_{0},\eta_{0})s_{\eta}(Z)\right] =\mathbb{E} \left[ \Pi_{\overline{T(\eta_{0})}}\mathbb{E}\left[ \left. g(Z,\theta _{0},\eta_{0})\right\vert \alpha,X\right] b(\alpha,X)\right] . \] From this equality, an analogous result to Proposition (ref) with covariates holds. To save space, we only give the result for nonparametric UH, i.e. fixed effects setting, as the semiparametric UH, i.e. correlated effects case, follows mutatis mutandis as in the model without covariates.

We relate this result with the functional differencing approach of Bonhomme (2012). He starts with the apparently different goal of finding a NF\ moment $g_{FD}(z,\theta_{0})$ such that

where $L_{\theta_{0},x}$ is the linear mapping \[ L_{\theta_{0},x}f\left( y\right) =\int_{\mathcal{A}}f_{y|\alpha,x}\left( y|\alpha,x;\theta_{0}\right) f\left( \alpha\right) d\nu\left( \alpha\right) , \] which is defined on $L_{2}(\pi_{\alpha}),$ for a user-specific weight function $\pi_{\alpha},$ and where $\mathcal{A}$ is a known set that contains the support of $\eta_{0}\left( \alpha|x\right) ,$ for all $x.$ Bonhomme (2012, Theorem 2) then shows that if $g_{FD}$ satisfies ((ref)), then it necessarily satisfies \[ \mathbb{E}\left[ \left. \pi_{Y}(Y)g_{FD}(X,\theta_{0})\right\vert X\right] =0,\text{ }v_{X}-a.s., \] for some user-specific weights $\pi_{Y}$. We provide conditions in the Supplementary Appendix (Section C) under which $g=\pi_{Y}g_{FD}$ satisfies the moment restriction

for some $g\neq0.$ Whether this equation holds for all $\alpha\in\mathcal{A}$ or $\eta_{0}-$a.s. could be critical for the existence of orthogonal moments, as we previously showed with Example 2. If ((ref)) holds, the LR\ moment function $g(z,\theta_{0})$ will not depend on $\eta_{0},$ i.e., it will be NF.\footnote{For interesting applications of ((ref)) to dynamic logit discrete choice see Kitazawa (2013, 2016), and more prominently Honor\'{e} and Weidner (2021).} If the necessary condition ((ref)) holds, without ((ref)) being true, then $g(z,\theta_{0},\eta_{0})$ may still exist but it may depend on $\eta_{0}$ through its support.

In parallel with Proposition (ref), we now give conditions under which ((ref)) and ((ref)) are equivalent, and thus both are necessary and sufficient conditions for LR.

This result shows that commonly used sufficient conditions for NF moments in functional differencing, such as ((ref)), are also necessary for existence of LR moments under support conditions on UH and smoothness of the model. Thus, functional differencing is a special case of our characterization. Without these additional assumptions, the conditions of functional differencing are sufficient when $\mathcal{A}$ includes the support of UH, but they may not be necessary. Furthermore, if $\mathcal{A}$ does not include the support of UH, then the moments from functional differencing may be misspecified. A practical take away of this discussion is that (i) $\mathcal{A}$ should be chosen large enough; and (ii) although orthogonal moments might depend on the UH (e.g., trough its support), they are LR, which make them less sensitive to large regularization biases if UH is estimated. This observation might be useful for proposing efficient functional differencing methods through feasible versions of the efficient scores (cf. Bonhomme, (2012)).

Summarizing, for the models considered in this section the fundamental equation for orthogonality for fixed parameters is ((ref)), which shows the important role played by the support of UH. Under additional support conditions, orthogonal moments are NF. The conditions for existence of orthogonal moments are more general than previously recognized, and include partially identified settings and models with discrete UH.

For other parameters of interest, such as average marginal effects, finding NF moments seems much more difficult, if not impossible. We study these cases in Section (ref).

Random Coefficient Models

If UH is independent of the covariates, the density of the data is

Under nonparametric UH, the necessary and sufficient condition for LR\ simplifies to conditioning only on the UH, as shown in the next result.

Propositions (ref) and (ref) imply that having the independence assumption between UH and covariates substantially increases the possibility of existence of LR moments. It is clear that $\mathbb{E}\left[ \left. g(Z,\bar{\theta},\eta_{0})\right\vert \alpha\right] =0$ a.s. could hold with $\mathbb{E}\left[ \left. g(Z,\bar{\theta},\eta_{0})\right\vert X,\alpha\right] \neq0,$ corresponding to the case where $g$ is only orthogonal in the random coefficient model and not in a fixed effects setting. We illustrate this point with an example.

\noindentExample 3: Linear Random Coefficient Model. Consider the linear random coefficient model with $Y=\theta_{0}X_{1} +X_{2}^{\prime}\alpha$, where $X_{2}$ usually contains an intercept and $\alpha$ is independent of $X=(X_{1},X_{2}^{\prime})^{\prime}$. Thus, this model generalizes the classical linear regression model to some coefficients being random. Take the moment function $g(Z,\theta_{0},\eta_{0})=(\tilde {Y}-\theta_{0}\tilde{X}_{1})\tilde{X}_{1}$, where for a generic variable $V$, $\tilde{V}=V-\mathbb{E}\left[ \left. V\right\vert X_{2}\right] $. To see that this moment is an orthogonal moment, note that $\tilde{Y}=\theta _{0}\tilde{X}_{1}+X_{2}^{\prime}(\alpha-\mathbb{E}\left[ \alpha\right] )$, and check that our necessary and sufficient condition ((ref)) holds

In contrast, $\mathbb{E}\left[ \left. g(Z,\bar{\theta},\eta_{0})\right\vert X,\alpha\right] =\tilde{X}_{1}X_{2}^{\prime}(\alpha-\mathbb{E}\left[ \alpha\right] )\neq0,$ so this moment is not orthogonal in a fixed effects setting if $\mathbb{E}\left[ \tilde{X}_{1}^{2}\right] >0$ and $\alpha$ is not constant. $\square$

General Models and Functionals of Interest

In many applications, researchers are interested in functionals of the model parameters, such as marginal effects and counterfactual effects. We extend our previous results to general models and parameters of interest. Consider again a generic semiparametric model $\mathcal{P}=\{\mathbb{P}_{\theta,\eta} :\theta\in\Theta,\eta\in\Xi\},$ with parameter space $\Lambda=\{\lambda =(\theta,\eta):\theta\in\Theta,\eta\in\Xi\}.$ We assume, by the chain rule, that the scores of the model have the representation $s=S_{\lambda_{0}}h$, $h\in\mathbf{H},$ for a score operator $S_{\lambda_{0}}$ given by

and for a Hilbert space $\mathbf{H}=\mathbb{R}^{p}\times\mathcal{H},$ with inner product $\langle(\delta_{1},b_{1}),(\delta_{2},b_{2})\rangle _{\mathbf{H}}:=\delta_{1}^{\prime}\delta_{2}+\langle b_{1},b_{2} \rangle_{\mathcal{H}}$, and where $\mathcal{H}\ $is another Hilbert Space endowed with the inner product $\langle\cdot,\cdot\rangle_{\mathcal{H}}.$ Here, $s_{\theta}$ is the ordinary score function of $\theta,$ and $S_{\eta}$ is the nuisance score operator from $T(\eta_{0})\subset\mathcal{H}$ to $L_{2} $.\footnote{See, e.g., Begun, Hall, Huang and Wellner (1983) for an introduction to score operators.} In this section we consider a parameter of interest given by a smooth functional of $\lambda$, $\psi(\lambda )\in\mathbb{R}^{d_{\psi}},$ where the precise sense of smoothness is defined below. Our previous results correspond to the leading example of $\psi (\lambda)=\theta,$ but this current setting also covers other parameters of interest such as average marginal effects or moments of the UH, as, for example, in \[ \psi(\lambda_{0})=\mathbb{E}_{\eta_{0}}\left[ r(\alpha)\right] ,\text{ }r\in L_{2}(\eta_{0}). \] The previous analysis with UH has a $S_{\eta}$ given by $S_{\eta} b=\mathbb{E}\left[ \left. b(\alpha)\right\vert Z\right] $. It is often straightforward to find the expression for $S_{\eta}$ in each application (simple inspection of $d\log f_{\theta_{0},\eta_{\tau}}/d\tau$ suffices).

We now consider paths $\lambda_{\tau}$ such that $d\lambda_{\tau} /d\tau=(\delta,b\eta_{0}),$ and smooth functionals $\psi(\cdot)$ such that their derivative $\dot{\psi}(h)=d\psi(\lambda_{\tau})/d\tau\ $is linear and continuous in $h=(\delta,b)$ over $\overline{T(\lambda_{0})}\subseteq \mathbf{H.}$\footnote{This parametrization of paths in terms of $b\eta_{0}$ rather than $b$ is convenient for cases where $\eta_{0}$ is a density, and fits our previous discussion. For other cases, we often use paths with tangents $(\delta,b)$ such that $d\lambda_{\tau}/d\tau=(\delta,b)$.} Linearity and continuity of $\dot{\psi}(\cdot)$ guarantees, by the Riesz representation theorem, the existence of a vector $r_{\psi}=(r_{\psi,j})_{j=1}^{d_{\psi}},$ with $r_{\psi,j}\in\overline{T(\lambda_{0})}$ for all $j=1,...,d_{\psi},$ such that \[ \dot{\psi}(h)=\langle r_{\psi},h\rangle_{\mathbf{H}}. \] For multivariate functionals this representation is understood componentwise. For example, for $\psi(\lambda)=\theta\in\mathbb{R}^{p}$ and $T(\lambda _{0})=\mathbb{R}^{p}\times T(\eta_{0}),$ the corresponding Riesz presenter $r_{\psi}$ is such that $r_{\psi,j}=(e_{j},0)\in\mathbb{R}^{p}\times \mathcal{H},$ with $e_{j}$ denoting the $j-$th canonical vector with a one in the $j-th$ coordinate and zero everywhere else, $j=1,...,p.$ That is, if $\theta_{\tau,j}=\theta_{0,j}+\tau\delta_{j}+o(1),$ then $d\theta_{\tau ,j}/d\tau=\delta_{j}=e_{j}^{\prime}\delta,$ where $\theta_{0}=(\theta _{0,1},...,\theta_{0,p})^{\prime}$ and $\delta=(\delta_{1},...,\delta _{p})^{\prime}.$

We show how all the previous theory of the paper can be extended to this more general setting. The null and alternative hypotheses are now \[ H_{0}:\psi(\lambda_{0})=\bar{\psi}\qquad vs\qquad H_{1}:\psi(\lambda_{0} )\neq\bar{\psi}, \] for a known $\bar{\psi}\in\mathbb{R}^{d_{\psi}},$ and with other aspects of $\lambda_{0}$ unknown under both $H_{0}$ and $H_{1}.$ The concepts of RLS and RLN are the same, but now the restricted tangent space is the space of scores from paths satisfying ((ref)) such that $\psi(\lambda_{\tau})=\bar{\psi}$ for all $\tau$ sufficiently small. A key object is the nuisance score operator pertaining to $\psi(\lambda_{0}),$ which we define as \[ S_{\psi_{0}^{\perp}}=S_{\lambda_{0}}\circ\Pi_{r_{\psi}^{\perp}}, \] where $\circ$ denotes composition of operators, $\Pi_{r_{\psi}^{\perp}}$ is the orthogonal projection onto the orthocomplement of the linear span of $(r_{\psi,j})_{j=1}^{d_{\psi}},$ and $S_{\lambda_{0}}$ is defined in ((ref)). The corresponding adjoint operator to $S_{\psi_{0}^{\perp} }$ satisfies \[ \langle S_{\psi_{0}^{\perp}}h,g\rangle=\langle h,S_{\psi_{0}^{\perp}}^{\ast }g\rangle_{\mathbf{H}},\text{ for all }h\in\mathbf{H},\text{ }g\in L_{2}^{0}, \] and it is given by $S_{\psi_{0}^{\perp}}^{\ast}=\Pi_{r_{\psi}^{\perp}}\circ S_{\lambda_{0}}^{\ast},$ where $S_{\lambda_{0}}^{\ast}$ is the adjoint operator of $S_{\lambda_{0}}$ defined by

As expected, for $\psi(\lambda)=\theta,$ $S_{\psi_{0}^{\perp}}h=S_{\lambda _{0}}(0,b)=S_{\eta_{0}}b$ and $S_{\psi_{0}^{\perp}}^{\ast}=S_{\eta_{0}}^{\ast }.$

We characterize orthogonal moments for inference on $\psi(\lambda_{0}).$ Henceforth, for a linear operator $K$ between the Hilbert Spaces $\mathcal{H}_{1}$ and $\mathcal{H}_{2},$ define the range of $K$ as $\mathcal{R}(K):=\{f\in\mathcal{H}_{2}:$ $f=Kb$ some $b\in\mathcal{H}_{1}\}$ and the kernel of $K$ as $\mathcal{N}(K):=\{f\in\mathcal{H}_{1}:$ $Kf=0\}.$ For multivariate functionals, we define $C^{\prime}r_{\psi}=\sum _{j=1}^{d_{\psi}}C_{j}r_{\psi,j}\in\mathbf{H},$ where $C=(C_{1},...,C_{d_{\psi }})^{\prime}\in\mathbb{R}^{d_{\psi}}.$

This result relies on a novel characterization of the tangent space of the restricted model based on the Riesz representer and duality.

For the structural parameter $\psi(\lambda)=\theta,$ $S_{\lambda_{0}}^{\ast }g=C^{\prime}r_{\psi}$ implies, by ((ref)) and $r_{\psi,j}=(e_{j},0),$ that $S_{\eta_{0}}^{\ast}g=0$, which corresponds to the functional differencing moments ((ref)) and ((ref)) for models with UH (without and with covariates, respectively). In a setting where the conditional likelihood given UH is parametric, the nuisance score operator $S_{\eta_{0}}^{\ast}g=\mathbb{E}\left[ \left. g(Z)\right\vert \alpha\right] $ is known under the null hypothesis (up to support conditions), which gives rise to the possibility of functional differencing (NF moments). Are there functional differencing moments for other functionals different from fixed parameters? As an illustration, let us now consider the case $\psi (\lambda)=\mathbb{E}_{\eta}\left[ r(\alpha)\right] $ in the model ((ref)) with a known function $r\in L_{2}(\eta_{0})\equiv\mathcal{H}$ and nonparametric UH. The case with covariates follows analogously. Then, for $\psi(\lambda)=\mathbb{E}_{\eta}\left[ r(\alpha)\right] $ and $h=(\delta ,b),$ \[ \dot{\psi}(h)=\int r(\alpha)b\left( \alpha\right) \eta_{0}\left( \alpha\right) d\nu\left( \alpha\right) =\langle r,b\rangle_{\mathcal{H}}. \] Therefore, the corresponding Riesz representer is $r_{\psi}=(0,r-\psi (\lambda_{0}))\in\mathbf{H}.$\footnote{The centering in $r_{\psi}$ is necessary for $r_{\psi}\in\overline{T(\lambda_{0})}=\mathbb{R}^{p}\times L_{2}^{0}(\eta_{0}).$} Then, from Theorem (ref) and ((ref)), an orthogonal moment exists if we can find $0\neq g\in L_{2}^{0}$ such that

and for a constant $C$ (possibly zero)

The distinct cases $C=0$ and $C\neq0$ represent two fundamentally different situations. The case $C=0$ allows for the possibility that the parameter of interest, here $\psi(\lambda_{0}),$ does not have a finite efficiency bound, as we show in ((ref)) below. In this situation orthogonal moments are not informative. Intuitively, a $g$ satisfying ((ref)) with $C=0$ does not depend on $\psi(\lambda_{0}).$ In this case $\psi(\lambda_{0})$ cannot be estimated at root-$n$ rate (see Bonhomme (2011), Escanciano (2023)). When $C\neq0$ orthogonal moments are informative, with a finite efficiency bound for $\psi(\lambda_{0})$ (as shown in Escanciano (2022)). Under the support conditions used in the functional differencing literature, the solution $g(Z,\lambda_{0})$ to ((ref)) with $C\neq0$ necessarily depends only on $\theta_{0}$ and $\psi(\lambda_{0}),$ i.e. $g(Z,\lambda_{0})=g(Z,\theta _{0},\psi(\lambda_{0})).$ If, in addition, $\partial\mathbb{E}\left[ g(Z,\theta_{0},\psi(\lambda_{0}))\right] /\partial\theta=0,$ then ((ref)) holds and $g$ is an orthogonal moment for $\psi(\lambda_{0}).$ Otherwise, $g$ is only partially robust, as it will sensitive to deviations of $\theta_{0}$ (((ref)) holds, but ((ref)) does not hold). The partially robust case is still useful, as inference can be based on $g(Z,\theta_{0},\psi(\lambda_{0}))$ and combined with moments from functional differencing for $\theta_{0}.$ An algorithm to compute partially robust moments solves \[ \mathbb{E}\left[ \left. \tilde{g}(Z,\theta_{0},\psi(\lambda_{0}))\right\vert \alpha\right] =r(\alpha)-\psi(\lambda_{0}), \] using the specification $\tilde{g}(Z,\theta_{0},\psi(\lambda_{0}))=\tilde {g}_{0}(Z,\theta_{0})-\psi(\lambda_{0})$ and the same tools from the functional differencing literature to solve for $\tilde{g}_{0}$ in $\mathbb{E}\left[ \left. \tilde{g}_{0}(Z,\theta_{0})\right\vert \alpha\right] =r(\alpha)$; see, e.g., Bonhomme (2012), Aguirregabiria and Carro (2021) and Honor\'{e} and Weidner (2021) for the construction of $\tilde{g}_{0}$. Then, an estimator for $\psi(\lambda_{0})$ can be based on $\psi(\lambda_{0})=\mathbb{E}\left[ \tilde{g}_{0}(Z,\theta_{0})\right] ,$ where $\theta_{0}$ can be estimated by functional differencing (see Bonhomme 2012)$\ $based on a $p$ dimensional NF moment $m(Z,\theta_{0}).$ Fully robust moments for $\psi(\lambda_{0})$ can be then constructed by

assuming the non-singularity of the Jacobian $\mathbb{E}\left[ \partial m(Z,\theta_{0})/\partial\theta^{\prime}\right] .$ Estimation of this Jacobian and the related $\mathbb{E}\left[ \partial\tilde{g}(Z,\theta_{0},\psi (\lambda_{0}))/\partial\theta^{\prime}\right] $ is discussed in Bonhomme (2012, p. 1366). Inference and estimation based on the fully orthogonal moment $g(Z,\theta_{0},\psi(\lambda_{0}))$ does not require estimation of the distribution of UH and it is robust to the estimation of $\theta_{0}.$

An algorithm that extends ((ref)) to a general setting with any smooth $\psi(\lambda_{0})\in\mathbb{R}$ and any regular semiparametric model is given as follows:

The feasibility of this algorithm depends on the feasibility of solving equations such as $S_{\eta_{0}}^{\ast}\tilde{g}=r_{\psi}^{(2)}$ or $S_{\eta_{0}}^{\ast}m=0.$ As in the functional differencing literature, these equations are easier to verify for discrete observations (e.g. discrete choice models), where they imply a system of equations with a finite number of unknowns. To see this, suppose $Z$ takes $M$ distinct values $\{z_{1} ,...,z_{m}\},$ so $\tilde{g}(z)=\sum_{j=1}^{M}1(z=z_{j})g(z_{j},\lambda_{0})$. Then, \[ S_{\eta_{0}}^{\ast}\tilde{g}=\sum_{j=1}^{M}S_{\eta_{0}}^{\ast}1(\cdot =z_{j})g(z_{j},\lambda_{0}), \] which is in the linear span of $\left\{ S_{\eta_{0}}^{\ast}1(\cdot =z_{j})\right\} _{j=1}^{M},$ a space of dimension at most $M.$ For discrete data, the number of functionals satisfying $S_{\eta_{0}}^{\ast}\tilde {g}=r_{\psi}^{(2)}$ is limited, as the set of such $r_{\psi}^{(2)}$ is finite-dimensional.

For continuous observations the number of possible functionals satisfying $S_{\eta_{0}}^{\ast}\tilde{g}=r_{\psi}^{(2)}$ is larger, but it also becomes more challenging to solve for $\tilde{g}.$ In many applications $S_{\eta_{0} }^{\ast}\tilde{g}=r_{\psi}^{(2)}$ becomes an integral equation of the form \[ S_{\eta_{0}}^{\ast}\tilde{g}(\cdot)=\int\tilde{g}(z,\lambda_{0})K(z,\cdot )d\mu(z)=r_{\psi}^{(2)}(\cdot), \] for a suitable kernel $K$ and measure $\mu.$ The set of Riesz representers $r_{\psi}^{(2)}(\cdot)$ satisfying this equation has been well-characterized in the mathematical literature (see, e.g., Saitoh (1997)). For functionals with a $r_{\psi}^{(2)}(\cdot)$ for which a solution to $S_{\eta_{0}}^{\ast }\tilde{g}=r_{\psi}^{(2)}$ exists, there are several methods available to solve for $\tilde{g},$ see Carrasco, Florens and Renault (2007) for a review of these results. In many cases, simple differentiation rules allow for explicit solutions of $\tilde{g}$. We illustrate some of these results with the average marginal effect parameter in Altonji and Matzkin (2005).

\noindentExample 4: Average Marginal Effects. Consider the model studied in Altonji and Matzkin (2005), where the observed data $Z=(Y,X,Z_{2})$ is such that $Y=m(X,\alpha),$ $X$ is a continuous random variable, and $\alpha$ denotes UH independent of $X$, conditional on $Z_{2}.$ The parameter of interest is the average marginal effect \[ \psi(\lambda_{0})=\mathbb{E}\left[ \frac{\partial m(X,\alpha)}{\partial x}\right] , \] where $\lambda_{0}=(\theta_{0},\eta_{0}),$ $\theta_{0}$ is the conditional density of $X$ given $Z_{2}$ and $\eta_{0}$ is the density of $\alpha$ conditional on $Z_{2}.$ In this parametrization, $f_{\theta_{0},\eta_{0}}$ is the density of $\mathbb{P}_{0}$ with respect to $\mu=\mu_{Y}\times\mu _{X}\times v_{Z_{2}},$ where $\mu_{Y}$ and $\mu_{X}$ are $\sigma$-finite measures and $v_{Z_{2}}$ is the probability measure of $Z_{2}.$ In this example $\theta_{0}$ is infinite-dimensional. Assume regularity conditions so that all derivatives and moments are well-defined and the classical integration by parts can be applied. The functional $\psi(\lambda_{0})$ is nonlinear and smooth, and $\dot{\psi}(\cdot)$ has a Riesz representer $r_{\psi}=(r_{\psi}^{(1)},r_{\psi}^{(2)})\in\overline{T(\lambda_{0})} =L_{2}^{0}(\theta_{0}\times dv_{Z_{2}})\times L_{2}^{0}(\eta_{0}\times dv_{Z_{2}})$ given by

From our results with UH, the adjoint score operator $S_{\eta_{0}}^{\ast }g(\cdot)$ is \[ S_{\eta_{0}}^{\ast}g(\cdot)=\mathbb{E}\left[ \left. g(Z,\lambda _{0})\right\vert \alpha=\cdot,Z_{2}=\cdot\right] . \] Following the general algorithm above, we need to find a solution to $S_{\eta_{0}}^{\ast}\tilde{g}=r_{\psi}^{(2)}\ $(Step 2), which can be easily solved by integration by parts, as \[ \tilde{g}(Z,\lambda_{0})=-Y\frac{\partial\theta_{0}(X,Z_{2})}{\partial x} \frac{1}{\theta_{0}(X,Z_{2})}-\psi(\lambda_{0}). \] However, the moment $\tilde{g}$ is only partially orthogonal (to deviations of $\eta_{0}),$ and not fully orthogonal for $\psi(\lambda_{0}),$ as we show next. The adjoint score operator $S_{\theta_{0}}^{\ast}g(\cdot)$ is given by \[ S_{\theta_{0}}^{\ast}g(\cdot)=\mathbb{E}\left[ \left. g(Z,\lambda _{0})\right\vert X=\cdot,Z_{2}=\cdot\right] , \] and satisfies \[ S_{\theta_{0}}^{\ast}\tilde{g}=-\mu(X,Z_{2})\frac{\partial\theta_{0}(X,Z_{2} )}{\partial x}\frac{1}{\theta_{0}(X,Z_{2})}-\psi(\lambda_{0})\neq r_{\psi }^{(1)}(\cdot), \] where $\mu(\cdot)=\mathbb{E}\left[ \left. Y\right\vert X=\cdot,Z_{2} =\cdot\right] .$ To see the last inequality, use the independence to show \[ r_{\psi}^{(1)}(\cdot)=\frac{\partial\mu(X,Z_{2})}{\partial x}-\psi(\lambda _{0}). \] Therefore, if we define \[ m(Z,\lambda_{0})=\frac{\partial\mu(X,Z_{2})}{\partial x}+\mu(X,Z_{2} )\frac{\partial\theta_{0}(X,Z_{2})}{\partial x}\frac{1}{\theta_{0}(X,Z_{2})}, \] we have $S_{\eta_{0}}^{\ast}m=0,$ since by integration by parts, \[ \mathbb{E}\left[ \left. \frac{\partial\mu(X,Z_{2})}{\partial x}\right\vert \alpha,Z_{2}\right] =-\mathbb{E}\left[ \left. \mu(X,Z_{2})\frac {\partial\theta_{0}(X,Z_{2})}{\partial x}\frac{1}{\theta_{0}(X,Z_{2} )}\right\vert \alpha,Z_{2}\right] . \] Take $A=1$ in the general algorithm (Step 3) and define the moment \[ g(Z,\lambda_{0})=\tilde{g}(Z,\lambda_{0})-m(Z,\lambda_{0}). \] Then, $S_{\theta_{0}}^{\ast}g=\partial\mu(X,Z_{2})/\partial x-\psi(\lambda _{0})=r_{\psi}^{(1)}$ and $S_{\eta_{0}}^{\ast}g(\cdot)=r_{\psi}^{(2)},$ verifying our sufficient and necessary conditions for orthogonality. The identification result of Altonji and Matzkin (2005) is \[ \psi(\lambda_{0})=\mathbb{E}\left[ \frac{\partial\mu(X,Z_{2})}{\partial x}\right] , \] but this moment is not LR. Our results permit a systematic approach based on orthogonal moments for inference on general parameters in models with UH. $\square$

When are orthogonal moments informative in this general setting? The insights from the local power investigation, our characterization of orthogonal moments and duality reveal that orthogonal moments are not informative when the score of the parameter of interest belongs to the closure of the restricted tangent space. To see this, when the parameter of interest is the structural parameter $\psi(\lambda)=\theta,$ from ((ref)) and $S_{\eta_{0}}^{\ast}g=0$ (i.e. $g\in\mathcal{N}(S_{\eta_{0}}^{\ast})$), if $s_{\theta}\in\overline {\mathcal{R}(S_{\eta_{0}})}=\mathcal{N}(S_{\eta_{0}}^{\ast})^{\perp},$ then \[ \frac{d}{d\tau}\mathbb{E}\left[ g(Z,\theta_{\tau},\eta_{0})\right] =-\mathbb{E}\left[ g(Z,\theta_{0},\eta_{0})s_{\theta}^{\prime}(Z)\right] \delta=0. \] We generalize this insight to the general case. Define $s_{\psi} :=S_{\lambda_{0}}r_{\psi},$ which plays the role of the score of the parameter of interest in this more general setting. Here, the application of $S_{\lambda_{0}}$ is componentwise when $\psi(\cdot)$ is multivariate.

We can interpret the condition $s_{\psi}\notin\mathcal{N}(S_{\psi_{0}^{\perp} }^{\ast})^{\perp}$ as a general relevance condition. It generalizes the conditional relevance condition of IV to any smooth parameter and any regular semiparametric model. For the structural parameter $\psi(\lambda)=\theta,$ $s_{\psi}=s_{\theta},$ $S_{\psi_{0}^{\perp}}^{\ast}=S_{\eta_{0}}^{\ast},$ the condition $s_{\theta}\notin\mathcal{N}(S_{\eta_{0}}^{\ast})^{\perp}$ means $\tilde{s}_{\theta}\neq0$ (the efficient Fisher information for the parameter is not a matrix of zeros). This is the general interpretation of relevance.

As an illustration, we can answer a question that has remained open in the literature of functional differencing: when is a functional differencing moment informative? Theorem (ref) provides the answer: a LR moment $g(Z,\theta_{0},\eta_{0})$ for $\psi(\lambda)=\theta$ is informative if $\mathbb{E}[g(Z,\theta_{0},\eta_{0})s_{\theta}(Z)]\neq0.$ This condition is generally weaker than local identification of $\theta_{0}$. Theorem (ref) generalizes this result to other functionals. As an example, consider $\psi(\lambda)=\mathbb{E}_{\eta}\left[ r(\alpha)\right] $ and an orthogonal $g$ satisfying ((ref)) and ((ref)), so $r_{\psi} =(0,r-\psi(\lambda_{0}))\ $and $s_{\psi}=S_{\lambda_{0}}r_{\psi}=S_{\eta_{0} }(r-\psi(\lambda_{0}))=\mathbb{E}\left[ \left. r(\alpha)-\psi(\lambda _{0})\right\vert Z\right] .$ By iterated expectations and ((ref)), the slope of the local power function for an orthogonal moment for $\psi (\lambda_{0})$ is

Taking $g=\tilde{s}_{\psi}=\Pi_{\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast} )}s_{\psi}\ $in ((ref)), it follows that $C=0$ corresponds to zero efficient Fisher information for $\psi(\lambda_{0}),$ i.e., $\mathbb{E} [\tilde{s}_{\psi}^{2}(Z)]=0.$ Thus, $C\neq0$ is necessary and sufficient for the orthogonal moment to be informative.

The next section considers a practically important class of settings in econometrics: models defined by Conditional Moment Restrictions (CMR), for which Example 1 is a special case. In these models, the previous analysis carries over, but with score operators which can be replaced by simpler conditional moment derivatives.

Models with Conditional Moment Restrictions

Let us consider the semiparametric model such that

where $\Theta\subset\mathbb{R}^{p}$ and $\Xi$ is possibly infinite-dimensional. These models have been studied in Ai and Chen (2007, 2012) and Chen and Santos (2018). The data observation is $Z=(Y,X,W),$ where $W$ denotes the union of distinct random elements of the conditioning variables $W_{j}.$

Define \[ m_{j}(W_{j},\theta,\eta)=\mathbb{E}_{\mathbb{P}}\left[ \left. \rho _{j}(Y,X,\theta,\eta)\right\vert W_{j}\right] \text{ a.s.} \] and the derivatives $\nabla m(W,\theta,\eta)[h]=(\nabla m_{1}(W_{1} ,\theta,\eta)[h],...,\nabla m_{J}(W_{J},\theta,\eta)[h]),$ where \[ \nabla m_{j}(W_{j},\theta,\eta)[h]=\frac{d}{d\tau}m_{j}(W_{j},\theta +\tau\delta,\eta+\tau b),\text{ }h=(\delta,b)\in T(\lambda_{0})\subseteq \mathbf{H}, \] and $\mathbf{H}=\mathbb{R}^{p}\times\mathcal{H},$ for a Hilbert space $\mathcal{H}$ endowed with the inner product $\langle\cdot,\cdot \rangle_{\mathcal{H}}.$ As in previous sections, when $\eta$ is a density we replace $\eta+\tau b$ by $\eta(1+\tau b)$ in the definition of $\nabla m_{j}$ for better interpretation. Let $L_{2}(W_{j})$ denote the Hilbert space $L_{2}(f)$ when $f$ is the density of $W_{j},$ and define the linear mapping \[ \nabla m:h\in\mathbf{H}\rightarrow\nabla m(W,\theta,\eta)[h]\in {\textstyle\bigotimes\nolimits_{j=1}^{J}} L_{2}(W_{j}). \] Define also \[ M_{\psi_{0}^{\perp}}=\left( \nabla m\right) \circ\Pi_{r_{\psi}^{\perp} }\qquad\text{and}\qquad M_{\psi_{0}^{\perp}}^{\ast}=\Pi_{r_{\psi}^{\perp} }\circ\left( \nabla m\right) ^{\ast}, \] where $\left( \nabla m\right) ^{\ast}$ is the adjoint operator of $\nabla m.$ The following theorem combines Theorem 4.1 in Chen and Santos (2018), which provides conditions for regularity of the model and characterizes the tangent space of the full model, with our results on orthogonal moments. Let $\psi(\lambda_{0})$ be a smooth functional, with Riesz representer $r_{\psi }\in\overline{T(\lambda_{0})}$ such that, for all $h\in\mathbf{H},$ $\dot {\psi}(h)=\langle r_{\psi},h\rangle_{\mathbf{H}}.$

Assumptions 4.1 and 4.2 in Chen and Santos (2018) are sufficient conditions for regularity of the model ((ref)). As a special case of functional consider the structural parameter $\psi(\lambda)=\theta,$ corresponding to $r_{\psi,j}=(e_{j},0).$ In this case the deviation $h$ has $\delta=0,$ so it suffices to consider $\nabla_{\eta}m_{j}(W_{j},\theta_{0},\eta)[b]=\frac {d}{d\tau}m_{j}(W_{j},\theta_{0},\eta+\tau b),$ and RLS holds if \[ \mathcal{R}(\nabla_{\eta}m)=\left\{ \nabla_{\eta}m(W_{j},\theta,\eta)[b]:b\in T(\eta_{0})\right\} \] is dense in $ {\textstyle\bigotimes\nolimits_{j=1}^{J}} L_{2}(W_{j}),$ where $\nabla_{\eta}m$ is the vector of conditional means derivatives with respect to $\eta,$ keeping $\theta_{0}$ fixed. The condition $\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})\neq\{0\}$ can be interpreted as the extension of exclusion restrictions in Example 1. We refer to the set $\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})$ as the set of \textquotedblleft orthogonal instruments\textquotedblright\ when the $W_{j}$ involve IVs, as they are transformations of the conditioning variables leading to orthogonal moments. This setting significantly generalizes Example 1.

\noindentExample 1: Partly Linear Model with Endogeneity, cont. This model corresponds to ((ref)) with $J=1,$ and \[ \rho_{1}(Z,\theta_{0},\eta_{0})=Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left( X\right) . \] Here, $M_{\psi_{0}^{\perp}}h=\nabla_{\eta}m(W,\theta_{0},\eta)[b]=-b(x)$ and $\overline{\mathcal{R}}(M_{\psi_{0}^{\perp}})=L_{2}(X).$ Therefore, \[ \mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})=\left( L_{2}(X)\right) ^{\perp }=\{\varphi(W)=\zeta(W)-\mathbb{E}\left[ \left. \zeta(W)\right\vert X\right] ,\text{ }\zeta\in L_{2}\}. \] In particular, RLS holds iff $L_{2}(X)=L_{2}(W),$ i.e., iff $Z_{2}\subset X.$ $\square$

To see when orthogonal scores are informative on the functional of interest in ((ref)), we specialize Theorem (ref) to this setting. Define

where again the application of $\nabla m$ is componentwise when $\psi(\cdot)$ is multivariate.

An important insight here is that it is not necessary to compute $\tilde {s}_{\theta}$ to check for the relevance condition. This is particularly useful for cases where computing $\tilde{s}_{\theta}$ is complicated (as in most of the examples considered in the paper). The definition of an OR-IV in the general case is $\Pi_{\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})}\nabla m_{\psi},$ where $\nabla m_{\psi}$ is given in ((ref)).

\noindentExample 1: Partly Linear Model with Endogeneity, cont. For $\psi(\lambda)=\theta,$ corresponding to $r_{\psi}=(1,0),$ we have \[ \nabla m_{\psi}=\frac{d}{d\tau}m(W,\theta_{0}+\tau,\eta_{0})=-\mathbb{E} \left[ \left. Y_{2}\right\vert W\right] . \] Therefore, for relevance $\nabla m_{\psi}\notin\mathcal{N}(M_{\psi_{0}^{\perp }}^{\ast})^{\perp}=L_{2}(X)$, or equivalently, $\mathbb{E}\left[ \left. -Y_{2}\right\vert W\right] \notin L_{2}(X),$ which is precisely the IV relevance condition$.$ Here $\Pi_{\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast} )}\nabla m_{\psi}=-\zeta^{\ast}(W)=-(\mathbb{E}\left[ \left. Y_{2} \right\vert W\right] -\mathbb{E}\left[ \left. Y_{2}\right\vert X\right] ).$ The complicated expression for $\tilde{s}_{\theta}$ is given in Ai and Chen (2003), and involves the inverse of conditional variances and solving a weighted least squares problem. $\square$

In Theorem (ref) and Theorem (ref) we deal with conditional moment restrictions. However, in some applications researchers are only willing to assume orthogonality restrictions on linear subspaces of $L_{2} (W)$, such as in high-dimensional linear regressions. We generalize the previous results to this case in the following remark.

\noindentExample 1: Partly Linear Model with Endogeneity, cont. So far, in this example we have considered $\Gamma=L_{2}(W),$ the set of all functions of $W$ with finite second moment. Instead $\Gamma$ could, for example, be restricted to be a linear combination of a dictionary, corresponding a high-dimensional regression. As an illustration, consider the case where $Z_{2}$ is a binary instrument, $Z_{2}\in\{0,1\},$ and let $\Gamma_{x}$ denote the closure of the linear span of a dictionary $b(X)=(b_{1}(X),b_{2}(X),...),$ meaning that any $\delta\in\Gamma_{x}$ can be well approximated in mean-square by a linear combination of $b_{j}(X).$ This model corresponds to our previous remark with $J=1$ and $\Gamma=\{\alpha (X)+Z_{2}\beta(X):\alpha,\beta\in\Gamma_{x}\}.$ Existence of orthogonal moments follows under the mild condition that $\Pi_{\Gamma}(L_{2}(X))$ is not dense in $\Gamma.$ For relevance, we recommend using the orthogonal instrument \[ \zeta^{\ast}(W)=\Pi_{\Gamma}Y_{2}-\Pi_{\Gamma_{x}}Y_{2}, \] where $\Pi_{\Gamma}Y_{2}$ and $\Pi_{\Gamma_{x}}Y_{2}$ denote, possibly high-dimensional, linear projections of $Y_{2}$ onto $\tilde{Z}_{2}\otimes b(X)$ and $b(X),$ respectively, where $\tilde{Z}_{2}=(1,Z_{2})$ and $\otimes$ denotes the Kronecker product.

Kolesar (2013) has investigated the properties of a 2SLS estimator based on a low-dimensional version of the instrument $\zeta^{\ast}$. Most notably, he obtained consistency under many instruments asymptotics for a version the jackknife IV estimator. A LR version of the moment based on $\zeta^{\ast}$ is given by

where $\tilde{Y}_{j}=Y_{j}-\Pi_{\Gamma_{x}}Y_{j},$ $j=1,2.$ A cross-fitting IV estimator with high-dimensional methods can be obtained from the moment ((ref)). This method requires three high-dimensional regressions $\Pi_{\Gamma_{x}}Y_{j},$ for $j=1,2,$ and $\Pi_{\Gamma}Y_{2}$, which can be obtained by, e.g., Lasso. Let $\hat{Z}_{2}$ denote a machine learner of $\Pi_{\Gamma}Y_{2},$ and $\hat{r}_{j}$ that of $\Pi_{\Gamma_{x}}Y_{j},$ for $j=1,2.$ Then, compute machine learning residual estimates for $\tilde{Y} _{j}\ $and $\zeta^{\ast}(W),$ say $\hat{Y}_{j}=Y_{j}-\hat{r}_{j}$ and $\hat{\zeta}^{\ast}(W)=\hat{Z}_{2}-\hat{r}_{2},$ respectively. Finally, an estimate for $\theta_{0}$ is an IV estimation of $\hat{Y}_{1}$ on $\hat{Y} _{2}$ with IV $\hat{\zeta}^{\ast}(W).$ This high-dimensional LR IV method addresses the critism on the practical use of the 2SLS of Angrist and Imbens (1995) mentioned in the Introduction (see S\l oczy\'{n}ski (2020)). The IV estimator can be easily implemented as a DML-IV with generated instruments $\hat{Z}_{2}\ $using off-the-shelf statistical software. This follows from the observation that $\Gamma_{x}\subset\Gamma,$ and therefore, by iterated projections $\Pi_{\Gamma_{x}}Y_{2}=\Pi_{\Gamma_{x}}\left( \Pi_{\Gamma} Y_{2}\right) .$ The asymptotic distribution for this estimator and the validity of inference with high-dimensional methods follows straightforwardly from the general results in CEIRN. $\square$

As further applications of these results, we consider heterogeneous parameters in treatment effects. Additionally, in the Supplemental Appendix we work with sample selection models and the popular demand model for differentiated products in BLP.

Heterogenous Parameters in Treatment Effects

Consider the following generalization of Example 1 to a model with heterogenous parameters (i.e. interactions),

where $W=(X,Z_{2})$, with $X=(X_{1},...,X_{d_{X}})^{\prime}\ $possibly high-dimensional, i.e. the dimension $d_{X}$ of $X$ can be large, much larger than the sample size, and the function $\eta_{0}(\cdot)$ has the representation \[ \eta_{0}\left( Y_{2},X\right) =\eta_{01}+\eta_{02}^{\prime}\left( X-\eta_{03}\right) + {\displaystyle\sum\limits_{l=1}^{d_{X}}} \eta_{04,l}Y_{2}\left( X_{l}-\eta_{03,l}\right) . \] The parameters $\eta_{03}$ are the means of $X$, i.e,

Models with interactions are commonly used in applied work, particularly when the interest is in understanding heterogenous treatment effects. This example falls under the setting of the previous section, with $\rho_{1}(Z,\theta _{0},\eta_{0})=Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left( Y_{2},X\right) ,$ $W_{1}=W,$ $\rho_{1+l}(Z,\theta_{0},\eta_{0})=X_{l}-\eta_{03,l},$ $W_{1+l} \ $empty, $l=1,...,d_{X},$ and $J=1+d_{X}$. As parameter of interest, we take $\psi(\lambda_{0})=\eta_{04,l}$, for some $l.$ For instance, we may be interested in testing if the conditional causal response function depends on the covariate $X_{l},$ i.e. testing $H_{0}:\eta_{04,l}=0$ vs $H_{1} :\eta_{04,l}\neq0$.

Our next result characterizes orthogonal moments for $\psi(\lambda_{0} )=\eta_{04,l}$ in this example. Define the random vector \[ Q_{l}=(Y_{2},1,X-\eta_{03},Y_{2}\left( X_{-l}-\eta_{03,-l}\right) ^{\prime })^{\prime}, \] and its projection onto the exogenous variables \[ \xi_{l}\equiv\xi_{l}(W)=\mathbb{E}\left[ \left. Q_{l}\right\vert W\right] =(p_{0}(W),1,X-\eta_{03},p_{0}(W)\left( X_{-l}-\eta_{03,-l}\right) ^{\prime })^{\prime}, \] where $p_{0}(W)=\mathbb{E}\left[ \left. Y_{2}\right\vert W\right] ,$ and $X_{-l}$ and $\eta_{03,-l}$ denote all coordinates of $X$ and $\eta_{03}$ but the $l-th.$

This proposition implies that for the heterogenous parameter $\psi(\lambda _{0})=\eta_{04,l},$ relevance can be achieved with just one instrument, provided ((ref)) holds. This relevance condition means that the partial correlation between $Y_{2}\left( X_{l}-\eta_{03,l}\right) $ and $\zeta$, after removing the effect of $\xi_{l},$ must be non-zero. When this condition holds, we can identify $\eta_{04,l}$ from the orthogonal moment \[ \mathbb{E}\left[ \left( Y_{1}-\gamma_{l}^{\prime}Q_{l}-\eta_{04,l} Y_{2}\left( X_{l}-\eta_{03,l}\right) \right) \varphi(W)\right] =0, \] where $\gamma_{l}=(\theta_{0},\eta_{01},\eta_{02}^{\prime},\eta_{04,-l} ^{\prime})^{\prime}.$ To implement inference based on a LR moment, let $\gamma_{l}$ solve the normal equations

Such $\gamma_{l}$ can be found under general conditions, including high-dimensional settings, as only the projection needs to be identified (see Step 2 below).

Set $\varphi(W)=\varphi^{\ast}(W)=\zeta^{\ast}(W)-\Pi_{\xi_{l}}\zeta^{\ast }(W),$ where $\zeta_{l}^{\ast}(W)=p_{0}(W)\left( X_{l}-\eta_{03,l}\right) .$ For this choice, under the minimal relevance condition \[ \eta_{04,l}=\frac{\mathbb{E}\left[ \left( Y_{1}-\gamma_{l}^{\prime} Q_{l}\right) \varphi^{\ast}(W)\right] }{\mathbb{E}\left[ Y_{2}\left( X_{l}-\eta_{03,l}\right) \varphi^{\ast}(W)\right] }, \] To implement inference in this example (testing $H_{0}:\eta_{04,l}=\bar{\eta }_{4,l}$ vs $H_{1}:\eta_{04,l}\neq\bar{\eta}_{4,l})$ based on orthogonal moments, we can use the following DML algorithm (we assume that prior to apply this algorithm we have centered $X$):

If the goal is estimation of $\eta_{04,l}$ instead, we can use the following variation of the previous algorithm, where Steps 1 and 3 remain the same, but Steps 2 and 4 change to:

For high-dimensional settings, we also recommend using cross-fitting in both algorithms, as in Chernozhukov et al. (2018). Asymptotic distribution theory for these procedures can be obtained by a routine application of CEINR. Section F of the Supplemental Appendix explores the finite sample performance of the above inference procedure in terms of size and power. Overall, the test presents size figures close to the usual nominal values and satisfactory power, consistent with the theory.

There are several papers proposing inference on heterogeneous parameters, but our procedure is different. Nekipelov, Semenova and Syrgkanis (2022) considered a binary choice model with interactions and instrumental variables. Qiu et al. (2021) consider a setting where the treatment $Y_{2}$ and the instrument $Z_{2}$ are binary. We do not impose any restriction on the support of the treatment variable or the instrument (i.e. these can take on a finite number of values or even be continuous) and also we allow for multiple instruments. Given that they model the treatment effect on the subpopulation of compliers (allowing for many regressors), the program they need to solve, which is a Lasso-type problem, is non-convex. This makes the inference procedure a more difficult task than in our setting, where the estimation and inference can be accomplished through an IV procedure with generated variables from machine learning fits. Syrgkanis et al. (2019) is the closest work to ours. They aim at estimating the Conditional Average Treatment Effect (CATE)$,$ although they also estimate projections of this quantity. The procedure of Syrgkanis et al. (2019) requires dealing with two different estimators of the CATE in order to obtain fully orthogonal moments. In our case, this is accomplished directly by using the orthogonal moments that we characterize.\footnote{We base inference on the orthogonal moment $\mathbb{E}\left[ \left( Y_{1}-\gamma_{l}^{\prime}Q_{l}-\eta_{04,l} Y_{2}X_{l}\right) \varphi(W)\right] =0$ rather than the non-orthogonal moment $\mathbb{E}\left[ \left( Y_{1}-\gamma_{l}^{\prime}\xi_{l}-\eta _{04,l}Y_{2}X_{l}\right) \varphi(W)\right] =0$. The latter fails to be orthogonal to estimates of $\xi_{l},$ as recognized by Syrgkanis et al. (2019).} Section F of the Supplementary Appendix compares the performance of our inference algorithm with that of Syrgkanis et al. (2019). Importantly, we derive conditions under which the orthogonal moments that we obtain are informative with respect to heterogeneous parameter of interest, which also seems to be a novelty in the literature of heterogeneous treatment.

In order to illustrate the usefulness of our previous theoretical results, we consider an application to the Oregon Health Experiment to study the presence of heterogeneous treatment effects of Medicaid on several health outcomes of interest.

The Oregon Health Experiment

In 2008, the state of Oregon expanded its coverage for Medicaid (the U.S. social program that provides health insurance to disadvantaged people who cannot afford private insurance). The state conducted lottery drawings to randomly select names from a waiting list of almost 90,000 uninsured adults, as demand far exceeded supply. Selected participants were given the opportunity (for themselves and any household member) to apply for Medicaid and, conditional on having their application approved, enroll in the program; for a detailed description of the experiment see Finkelstein et al. (2012).

Not all participants who were selected through the lottery were ultimately enrolled in Medicaid. There were two primary reasons for this: either they decided not to submit the application form in the end or they did so but failed to meet some of the requirements. Consequently, we observe a binary treatment variable $Y_{2}$ that equals 1 if the individual was successfully enrolled in Medicaid and 0 otherwise, and an instrument $Z_{2}$ that equals 1 if the individual was randomly selected from the lottery and 0 otherwise. As recognized by the literature, since individuals could choose to enroll in the program or not, selection into the treatment is potentially endogenous.

We focus on learning how individuals with different observable characteristics are benefited from the treatment, considering several outcomes of interest. Particularly, we are interested in health care utilization, out-of-pocket costs for medical care, a measure of overall health and signs of depression, as reported by individuals.\footnote{See the note in TABLE G.I in the Supplementary Appendix for a detailed description of the outcomes.} Previous studies have dealt with similar outcomes. In particular, Finkelstein et al. (2012) and Baicker et al. (2013) found that relative to the control group without insurance, treated individuals have higher health utilization, lower out-of-pocket medical expenditure, and better self-reported physical and mental health. A more recent study in the context of machine learning can be found in Qiu et al. (2021). There were different follow-ups during the experiment (conducted through different surveys). Here, we focus on the results obtained approximately one year after the experiment, as in Finkelstein et al. (2012).\footnote{In contrast, Baicker et al. (2013) and Qiu et al. (2021) work with data obtained approximately two years after the experiment.} This is our main database which we complemented using demographics characteristics that were recorded at the time individuals signed up for the lottery and lottery selection. TABLE G.I and TABLE G.II of the Supplementary Appendix present some descriptive statistics of the outcomes that we will be interested in and the main characteristics of our sample, respectively. We do observe differences, in terms of means, between the treated and the control group for all the outcomes, except for overall health, considering the usual nominal levels. Furthermore, there exist statistical significant differences in terms of observable characteristics. For instance, the treated have a higher proportion of women, individuals with an income less than half of the federal poverty line, people that did not finish high school, and unemployed subjects. It is worth noting that while there are observed differences in demographic characteristics between the treated and control groups, the random assignment of individuals to the treatment and through the lottery provides us with a source of exogenous variation to identify the causal effects of the Medicaid expansion.

We consider a causal analysis to estimate the effect of the treatment (i.e. being enrolled in Medicaid) once it is instrumented by winning the lottery on the above health outcomes. We have $69$ covariates which include the ones in TABLE G.II plus others and some quadratic terms (e.g. household income and age are also squared). In this analysis, we focus on age as the main source of observable heterogeneity between the treated and control groups. Previous studies have also recognized that age is an important variable to explain heterogeneous effects (e.g. Baicker et al. (2013) and Qiu et al. (2021)). Therefore, to better exploit heterogeneity of the treatment, we conduct the analysis for the overall sample but also for subsamples of individuals with age between 19 and 34, 35 and 49, and 50 and 64 years old.

To study the potential relevance of age regarding how the treatment affects the outcomes of interest, we now consider our linear model ((ref)) with interactions. This model has 138 regressors. In terms of our discussion in Section (ref), we take $X_{l}$ = Age. We then question if the treatment effect changes with such a variable, i.e., we test $H_{0} :\eta_{04,l}=0$, vs $H_{1}:\eta_{04,l}\neq0$, using our proposed algorithm that exploits our construction of LR moments.\footnote{Step 1 uses Categorical Boosting, and Step 3 employs Random Forest, using two different subsamples for estimation and prediction, while Step 2 uses Lasso with a penalization parameter selected by Cross-Validation (based on 4 folds).} The result of this inference exercise can be found in Table (ref). We report whether our algorithm rejects or not the null hypothesis at the $5\%$ level for the four outcomes and the different subsamples. The analysis indicates that there is evidence that the treatment changes with the age of individuals for the youngest and the oldest subsamples. Particularly, we reject $H_{0}$ for any of the outcomes when the subsample 19-34 yr is considered, while for the oldest group there exist heterogeneous effects in terms of out-of-pocket expenditures and overall health only. The slopes are observed to be small, though, as indicated by the 95$\%$ Confidence Intervals obtained by inverting our test statistic. No effects were found for the whole sample, possibly because the treatment was homogeneous among the middle-age individuals, which represents the most important group in our sample.

Conclusions

In the recent decade, there has been an increasing interest in orthogonal/debiased/LR moments, due to their convenient properties in contexts with machine learning first step estimators. The existence problem of such orthogonal moments for general parameters has not been studied in detail prior to this work. This paper contributes by providing a necessary and sufficient condition for the existence of such moments, denoted as Restricted Local Non-Surjectivity, to conduct inference on general parameters in regular semiparametric models. Our work delivers a positive result: existence of LR moments follows under quite weak general conditions. In particular, RLN does not require identification of the parameters of interest, identification of the nuisance parameters, or non-singularity of the Fisher Information matrix. Additionally, in our analysis we characterize when orthogonal moments are informative in general.

To demonstrate the utility of our results, we work through two popular settings in econometrics: models with UH and models defined by conditional moment restrictions with possibly different conditioning variables. For the former, our results characterize orthogonal moments and partially orthogonal moments for parameters such as average marginal effects. For models defined by conditional moment restrictions, we give orthogonal moments and illustrate how to select orthogonal moments to guarantee relevance, without the need to compute efficiency bounds. Several applications illustrate the wide applicability of our methods, including the fully saturated 2SLS, heterogeneous parameters in treatment effect models, sample selection models, and the popular BLP.

Our research has some limitations that should be addressed in future studies. Firstly, the practical implementation of constructing LR moments and tests derived from them for general models and parameters was not within the scope of this paper (though we obtain those for some useful examples). In many cases, functional forms for such moments may be unknown, such as for models with UH, making the construction challenging. Secondly, we focus on the inference problem and not on the estimation problem. However, our results on inference are fundamental for the estimation problem: if orthogonal moments exist and identify the structural parameters of interest, LR estimators can be proposed using a Generalized Methods of Moments (GMM) approach applied to the orthogonal moments, as in CEINR. The conditions derived for the existence and relevance of orthogonal moments are an essential part for such constructions. Therefore, an important research question is how to design an automatic construction of such moments for estimation and inference in models with UH and other settings not treated in CEINR. We believe that the progress made in this paper establishes the basis for advancing in this direction. We leave this promising avenue of research for future investigations.

\setcounter{secnumdepth}{0} \setcounter{table}{0}