Is completeness necessary? Estimation in nonidentified linear models

Is Completeness Necessary? Estimation in Nonidentified Linear Models

{ Keywords: machine learning, high-dimensional regressions, ridge, gradient descent, PCA, nonparametric IV, nonidentified models, weak identification. \\ JEL Classifications: C14, C26}

\thispagestyle{empty}

\setcounter{page}{1}

Introduction

Structural nonparametric and high-dimensional models are often ill-posed. Notable examples include nonparametric instrumental variable (IV) regression, various high-dimensional regressions, measurement error models, and random coefficient models. These cases commonly lead to an ill-posed functional equation

where $\varphi$ is the structural parameter of interest, $r$ is a known function, and $K$ is a linear operator. The classical literature on numerical ill-posed inverse problems (see engl1996regularization) addresses deterministic problems, where $K$ is known and where $r$ is observed with a deterministic numerical error. However, in econometric applications, both $K$ and $r$ must be estimated from data, posing a statistically ill-posed inverse problem.

Identification has long been a central concept in econometrics, as demonstrated by foundational works such as koopmans1949identification, koopmans1950identification, and rothenberg1971identification for parametric models. In nonparametric and high-dimensional settings, $K$ and $r$ are typically derived from the data-generating process, with $\varphi$ uniquely identified if $K\varphi = r$ has a unique solution. Uniqueness is achieved when $K$ is a one-to-one (or nonsingular) operator, meaning that $K\phi = 0 \implies \phi = 0$ for all $\phi$ in the domain of $K$. However, in econometric applications, $K$ is often unknown, and the estimated operator $\hat{K}$, being finite-rank, lacks one-to-one properties in finite samples.

In cases of nonidentification, the maximum likelihood estimator can yield a flat likelihood in some parameter space regions, leading to an ambiguously defined maximum. It is thus important to characterize the asymptotic behavior of estimators in potentially nonidentified models. In ill-posed models without identification, the identified set is a linear manifold, $\phi + \mathcal{N}(K)$, where $\phi$ solves $K\phi = r$ and where $\mathcal{N}(K)$ is the null space of $K$. This identified set, which is generally unbounded, lacks informativeness on $\varphi$ without additional constraints.

Given that $K$ or $\hat{K}$ may fail to be one-to-one and often has a discontinuous generalized inverse, regularization is essential for estimating the structural parameter consistently. In this paper, we focus on spectral regularization methods that modify the spectrum of $\hat{K}$. A prominent example is Tikhonov regularization (functional ridge regression); see tikhonov1963solution. Other methods include iterated Tikhonov, spectral cutoff (or principal component analysis, see yao2005functional), and Landweber iteration (or functional gradient descent).\footnote{Functional gradient descent is also foundational in boosting estimators; see friedman2001greedy.} We show that estimators based on spectral regularization are uniformly consistent for the best approximation of the structural parameter in the orthogonal complement of the null space of $K$, and we explore the distributional implications of identification failures.

In some cases, the best approximation may coincide with, or closely approximate, the structural parameter even if $K$ is singular and has a nontrivial null space. This provides an insightful interpretation of nonparametric IV regression under identification failure, akin to the least-squares best approximation under misspecification; see angrist2008mostly, Chapter 3.\footnote{In contrast, the parametric 2SLS estimator lacks this best approximation property; see escanciano2018optimal for an example of an IV estimator with this interpretation.} Moreover, the best approximation can also be used to construct set estimators in partial identification frameworks.

\paragraph{Contribution and related literature.} Our paper contributes to several research areas. First, it advances the literature on regularization in statistical inverse problems, as discussed in newey2003instrumental, blundell2007semi, chen2012estimation, chen2018optimal, carrasco2007linear,carrasco2013asymptotic, darolles2011nonparametric, florens2011identification, gagliardini2012tikhonov, hall2005nonparametric, babii2020honest,babii2021high, and other studies. canay2013testability demonstrate that the completeness condition is untestable in these models; see also d2010completeness, hu2017nonparametric, and andrews2017examples for more on this condition. Our contribution is the derivation of convergence rates in Hilbert space and $L_\infty$ norms for estimators based on generic spectral regularization schemes, including Tikhonov, iterated Tikhonov, Landweber iteration, and spectral cutoff. These results, which do not require the completeness condition, are illustrated for nonparametric IV and high-dimensional regressions.\footnote{This generalizes florens2011identification, which covers only $L_2$ rates for Tikhonov regularization under certain high-level conditions.}

Second, we add to the literature on identifying linear functionals of a structural function $\varphi$. For instance, severini2006some,severini2012efficiency show that linear functionals of $\varphi$ can be identified under conditions weaker than the completeness condition and derive the corresponding efficiency bounds when $\sqrt{n}$ estimation is possible. santos2011instrumental establishes the asymptotic normality of linear functionals of nonparametric IV regression without the completeness condition, whereas escanciano2018optimal provides necessary and sufficient conditions for the identification and $\sqrt{n}$-estimability of the best linear approximation to the nonparametric IV structural function. We contribute by showing that the asymptotic distribution of linear functionals of $\varphi$ may include a chi-square component when the completeness condition fails and the functional is estimable at a rate slower than $\sqrt{n}$.

Third, our work connects to the partial identification literature in nonparametric IV regression; see santos2012inference, freyberger2017completeness, and chernozhukov2023constrained. Our findings on consistency for the best approximation in the $L_\infty$ norm, alongside freyberger2017completeness, have applications in partial identification.

The remainder of this paper is structured as follows. Section (ref) introduces the concept of identification in linear ill-posed models. Section (ref) derives nonasymptotic risk bounds in Hilbert space and $L_\infty$ norms for Tikhonov-regularized estimators, generalizing these results to other spectral regularization schemes in Appendix Section (ref). Section (ref) illustrates the transition between Gaussian and weighted chi-square asymptotics under identification failure. Section (ref) presents a Monte Carlo study, while Section (ref) concludes the paper. All proofs are presented in Appendix (ref). Appendix (ref) discusses partial identification, and Appendix (ref) examines Tikhonov-regularized estimators under extreme identification failures. Finally, Online Supplementary Material, Sections (ref) and (ref) review relevant results from generalized inverse theory and Hilbert space-valued U-statistics.

Identification

Consider the functional linear equation

where $K:\mathcal{E}\to\mathcal{H}$ is a compact linear operator between Hilbert spaces $(\mathcal{E},\langle.,.\rangle_{\mathcal{E}})$ and $(\mathcal{H},\langle.,.\rangle_{\mathcal{H}})$ and where $\varphi\in\mathcal{E}$ represents the structural parameter of interest. For $\varphi$ to be point identified, $K$ must be one-to-one (or nonsingular), meaning that the null space of $K$, denoted by $\mathcal{N}(K) = \left\{\phi\in\mathcal{E}:\; K\phi = 0\right\}$, is reduced to $\{0\}$. Thus, for any $\phi\in\mathcal{E}$,

When the completeness condition fails, the identified set becomes a closed linear manifold, which is denoted by $I_0 = \varphi + \mathcal{N}(K)$. Since $\mathcal{N}(K)$ forms a closed linear subspace in $\mathcal{E}$, we can decompose $\varphi = \varphi_1 + \varphi_0$, where $\varphi_0$ is the unique orthogonal projection of $\varphi$ on $\mathcal{N}(K)$ and where $\varphi_1$ is the projection of $\varphi$ on its orthogonal complement, denoted by $\mathcal{N}(K)^\perp$. We let $K^*:\mathcal{H}\to\mathcal{E}$ denote the Hilbert adjoint operator of $K:\mathcal{E}\to\mathcal{H}$, defined by the relation\footnote{To reduce visual clutter, we omit subscripts and use $\langle .,.\rangle$ and $\|.\|$ to denote the inner products and norms in Hilbert spaces. For example, it is clear that the inner product on the left side is $\langle.,.\rangle_\mathcal{H}$, whereas the inner product on the right side is $\langle.,.\rangle_{\mathcal{E}}$.}

Since $\mathcal{N}(K)^\perp=\overline{\mathcal{R}(K^*)}$ (see luenberger1997optimization, p.157), the best approximation $\varphi_1$ equals the structural parameter $\varphi$ whenever $\varphi\in \overline{\mathcal{R}(K^*)}$, which is the closure of the range of the adjoint operator $K$. Note that $\mathcal{R}(K^*) = \mathcal{R}(K^*K)^{1/2}$, as shown in engl1996regularization, Proposition 2.18. Therefore, the identification of $\varphi$ can be related to a regularity condition known as the source condition, where $\varphi\in\mathcal{R}(K^*K)^{\beta/2}$ with $\beta\geq 1$; see carrasco2007linear. In other words, additional regularity of $\varphi$ can compensate for the failure of the completeness condition.\footnote{A similar observation holds for high-dimensional sparse regressions, where sparse coefficients can be identified under the restricted eigenvalue condition; see tsybakov2009simultaneous.} The following two examples further illustrate this point; see also bontemps2024functional for further applications to ecological inference.

Nonasymptotic Risk Bounds

In this section, we derive risk bounds for the Tikhonov estimator, which solves

where $\alpha_n > 0$ is a regularization parameter. The solution to this problem is

which is straightforward to verify. Note that the estimator $\hat{\varphi}$ is well defined even if $K$ or $\hat{K}$ is not one-to-one.

Hilbert Space Risk

We first describe the relevant class of structural functions and operators:

To illustrate this assumption, we let $(\sigma_j,e_j,h_j)_{j\geq 1}$ be the spectral decomposition of $K:\mathcal{E}\to\mathcal{H}$; see kress2013linear, Theorem 15.16. Then, $\varphi_1 = \sum_{j\geq 1}\langle \varphi_1,e_j\rangle e_j$, and by Parseval's identity, since $\psi = (K^*K)^{-\beta/2}\varphi_1$, we have

Assumption (ref) restricts the relative rates of decline of singular values $(\sigma_j)_{j\geq 1}$, which describe the ill-posedness, and Fourier coefficients $\langle\varphi_1,e_j\rangle_{j\geq 1}$, which describe the regularity of $\varphi_1$.

We estimate $(r,K)$ with $(\hat r,\hat K)$ and impose the following assumption:

Assumption (ref) restricts the convergence rate of the residuals and the estimated operator. Note that florens2011identification, Theorem 2.2, imposes a stronger high-level condition:

and certain assumptions on the tuning parameters that have not been verified for specific models.

The following result applies to the Hilbert space norm, $\|.\|$:

Theorem (ref) shows that the convergence rate in the Hilbert space norm is driven by the rates of

Notably, a stronger version of Assumption (ref) is commonly assumed: $\varphi\in\mathcal{R}(K^*K)^{\beta/2}$ with $\beta\geq 1$; see carrasco2007linear. In this case, we actually have $\varphi_0=0$, and the Tikhonov regularized estimator can consistently estimate the structural parameter $\varphi$, despite the failure of the completeness condition. More generally, we distinguish the following possibilities:

In the identified case, the optimal choice of the regularization parameter is $\alpha_n \sim \delta_n^{1/(\beta+1)}$, which yields a convergence rate of $O\left(\delta_n^{\beta/(\beta+1)}\right)$ for $\beta \in (0,2]$. For $\beta > 2$, we demonstrate in Appendix Section (ref) that the optimal rate can be achieved with additional Tikhonov iterations or alternative regularization schemes.

Finally, it is worth mentioning that in the special case of $L_2$ spaces, Theorem (ref) provides a sharper result than does florens2011identification, Theorem 2.2, where the rate is always driven by $\rho_{1n}$.

$L_\infty$ Risk

We consider the space of continuous functions on a compact set $D\subset\ensuremath{\mathbf{R}}^p$, denoted by $(C(D),\|.\|_\infty)$, where $\|.\|_\infty$ is the uniform norm. We suppose that this space is embedded into $\mathcal{E}$ and that $\mathcal{R}(K^*)\subset C(D)$ and that $\varphi_1\in C(D)$. The mixed operator norm of $K^*$ is defined as $\|K^*\|_{2,\infty}=\sup_{\|\phi\|\leq 1}\|K^*\phi\|_\infty$. The following assumption captures the approximation quality of the operator $K^*$ by its estimate $\hat K^*$ in the $\|.\|_{2,\infty}$ norm.

The following result establishes a uniform convergence rate:

Theorem (ref) provides a uniform convergence rate for generic ill-posed inverse problems. Unlike Theorem (ref), the $L_\infty$ convergence rate is affected by the estimation error in $K^*$, assessed in the $\|.\|_{2,\infty}$ norm. For further details on the $L_\infty$ convergence rates of more general regularized estimators, see Appendix Section (ref).

Applications

High-Dimensional Regressions

We consider the model

where $(Y,Z,W)\in\ensuremath{\mathbf{R}}\times\mathcal{E}\times\mathcal{H}$; see florens2015instrumental.\footnote{For example, when $W=Z$ and $\mathcal{E}=L_2$ with a counting measure, this yields a high-dimensional regression model for nonsparse data $$Y = \sum_{j\geq 1}\varphi_jZ_j + U,\qquad \ensuremath{\mathds{E}}[UZ_j]=0,\qquad \forall j\geq 1.$$ When $\mathcal{E}=L_2$ with the Lebesgue measure, this model is also referred to as functional regression; see florens2015instrumental and babii2021high.} The exclusion restriction leads to the functional linear equation

where $K:\mathcal{E}\to\mathcal{H}$ is a covariance operator and where $\otimes$ denotes the tensor product. An econometrician observes an i.i.d. sample $(Y_i,Z_i,W_i)_{i=1}^n$.\footnote{This assumption can be relaxed to covariance stationarity and absolute summability of autocovariances; see babii2021high.} Then,

are estimated as

It can be shown that

We let $\mathcal{F}(\beta,C)$ represent the class of models satisfying Assumption (ref) and assume that $\ensuremath{\mathds{E}}\|UW\|^2\vee \ensuremath{\mathds{E}}\|ZW\|^2\leq C$ for all the models in this class. Then, $\delta_{n}=\rho_{1n}=n^{-1}$, and by Theorem (ref), the Hilbert space risk is of order

For high-dimensional regression, the model is either identified ($\varphi_0=0$) or nonidentified ($\varphi_0\ne 0$). The conditions $\alpha_n\to 0$ and $\alpha_nn\to\infty$ as $n\to\infty$ are sufficient for consistency in the Hilbert space norm. The optimal choice $\alpha_n\sim n^{-1/(\beta+1)}$ yields a convergence rate of order $O(n^{-\beta/(\beta+1)})$, which is optimal when $W=Z$; see babii2024functional, Theorem 2.2.

For uniform convergence, we suppose that $\mathcal{E}=L_2(D)$, where $D\subset\ensuremath{\mathbf{R}}^p$ is bounded, and that the functions in $L_2(D)$ are square integrable with respect to the Lebesgue measure. To satisfy Assumption (ref), we also assume that the models in $\mathcal{F}(\beta,C)$ are such that $\|Z\|_\infty\vee \|W\|_\infty\leq C<\infty$ and that stochastic processes $Z$ and $W$ belong to a H\"{o}lder ball with smoothness $s>d/2$. Using the Hoffman--J\o rgensen and moment inequalities (see gine2015mathematical, p. 129 and p. 202),

Therefore, Assumption (ref) holds with $\rho_{2n}=n^{-1}$, and by Theorem (ref),

The conditions $\alpha_n\to 0$ and $\alpha_n n^{1/2}\to \infty$ as $n\to\infty$ ensure the uniform consistency of $\hat\varphi$. The optimal choice $\alpha_n \sim n^{-1/(\beta+2)}$ leads to a convergence rate of order $O(n^{-\beta/{(2\beta+4)}})$. Determining the minimax-optimal convergence rate for the class $\mathcal{F}(\beta,C)$ remains an open problem.

Nonparametric IV Regression

Following Example (ref), we can reformulate the model as

where $K:L_2([0,1]^p)\to L_2([0,1]^q)$. To estimate $r$ and $K$, we use kernel smoothing:

where $K_w$ and $K_z$ are symmetric kernel functions and where $h_n\to 0$ is a bandwidth parameter. Under mild assumptions, Proposition (ref) shows that $\delta_{n} = \frac{1}{nh^q_n} + h^{2s}_n$ and $\rho_{1n} = \frac{1}{nh^{p+q}_n} + h^{2s}_n$, where $s$ denotes the H\"{o}lder smoothness of $f_{ZW}$. Thus, by Theorem (ref), the mean-integrated squared error has the following rate:

where the class $\mathcal{F}(\beta,C)$ includes additional moment restrictions; see babii2020honest. In the nonparametric IV model, all four identification cases are possible, depending on the regularity parameter $\beta$. For consistency of $\hat\varphi$, we require $\alpha_nnh_n^{q}\to\infty$, $\alpha_n^{(1-\beta)\wedge 0}nh_n^{p+q}\to\infty$, and $h_n^{2s}/\alpha_n\to 0$ as $n\to \infty,\alpha_n\to0$, and $h_n\to0$.

Moreover, it is known that $\rho_{2n}^{1/2} = \sqrt{\frac{\log h_n^{-1}}{nh^{p+q}_n}} + h^s_n$; see, for instance, babii2020honest, Proposition A.3.1. Thus, by Theorem (ref), we find

For uniform consistency of $\hat\varphi$, we need $\alpha_n^2nh_n^q\to \infty$, $\alpha_nnh_n^{p+q}\to\infty$, and $h_n^{s}/\alpha_n\to 0$ as $n\to\infty$, $\alpha_n\to0$, and $h_n\to 0$.

It remains an open question whether these convergence rates for the NPIV model are minimax-optimal for the class $\mathcal{F}(\beta,C)$. Note that this class does not require differentiability or specific H\"{o}lder smoothness of $\varphi_1$; see chen2018optimal, which established minimax-optimal convergence rates for the NPIV model over the H\"{o}lder ball.

Continuous Linear Functionals

In many economic applications, the primary focus is on a continuous linear functional of a structural function $\varphi$. This section demonstrates that the asymptotic distribution of such functionals can become nonstandard under identification failure. To simplify the exposition, we concentrate on the high-dimensional regression example in Section (ref); further results are provided in Appendix Section (ref).

According to the Riesz representation theorem, any continuous linear functional on a Hilbert space $\mathcal{E}$ can be represented as an inner product with some $\mu\in \mathcal{E}$. Thus, we focus on the asymptotic distribution of $\langle \hat\varphi,\mu\rangle$. We let $\mu = \mu_0+\mu_1$ denote the orthogonal decomposition of $\mu$, where $\mu_0$ and $\mu_1$ are the orthogonal projections on $\mathcal{N}(K)$ and $\mathcal{N}(K)^\perp$, respectively. Similarly, we decompose $W = W^0 + W^1$, where $W^0$ and $W^1$ are the orthogonal projections of $W$ on $\mathcal{N}(K^*)$ and $\mathcal{N}(K^*)^\perp$, respectively.

We define the variance operator $\Sigma=\ensuremath{\mathds{E}}[(Y-\langle Z,\varphi_1\rangle)^2W\otimes W]$ and the sequence

with $\mu_1=\mu-\mu_0$. The following assumptions place mild restrictions on the data distribution and tuning parameters.

Assumption (ref) (ii) is an undersmoothing condition. Its requirements are the most stringent when $\pi_n\sim n^{1/2}$, in which case $n\alpha_n^{(2\gamma + \beta)\wedge 2}\to 0$, $n\alpha_n^{2-2\gamma\wedge 1}\to \infty$, and $n\alpha_n^{1+\beta\wedge 1}\to 0$ are needed. The last two conditions are not binding whenever $\gamma\geq1/2$ and $\beta>0$, in which case the first condition is automatically satisfied.

We let $(\chi_{j}^2)_{j\geq 1}$ be i.i.d. chi-square random variables with 1 degree of freedom, and we let $(\lambda_j)_{j\geq 1}$ be the eigenvalues of the operator $T:\phi\mapsto \ensuremath{\mathds{E}}_{X}[\phi(X)h(X,X')]$ on $L_2(X)$ with

where $\ensuremath{\mathds{E}}_X$ denotes the expectation with respect to $X=(Y,Z,W)$ and where $X'$ is an independent copy of $X$.

The following result holds:

Since

the following corollary is a trivial consequence of Theorem (ref).

The normalizing sequence in Corollary (ref) can be either $\pi_n$ or $n\alpha_n$, and the asymptotic distribution may feature a Gaussian component, a chi-square component, or both, depending on the mapping properties of the operators $K$ and $\Sigma$. Specifically, if the completeness condition holds, the asymptotic distribution consists solely of a Gaussian component. However, if the completeness condition fails, a chi-square component may also appear.

To illustrate this, we let $W=(W(t))_{t\in[0,1]}$ be a Brownian bridge with a Karhunen--Lo\`{e}ve decomposition $W(t) = \sum_{j=1}^\infty\frac{\xi_j}{j\pi}h_j(t)$, where $\xi_j\sim_{i.i.d.}N(0,1)$ and $h_j(t)=\sqrt{2}\sin(\pi j t)$; see shorack1986empirical, p.33. We let $Z(s) = \sum_{j=1}^\infty\frac{\zeta_j}{j\pi}e_j(s)$ be a Gaussian process with $\zeta_j\sim_{i.i.d.}N(0,1)$, $\ensuremath{\mathrm{Cov}}(\zeta_j,\xi_k)=\rho\mathbf{1}_{j\ne k}$, and $e_j(s)=\sqrt{2}\cos(\pi js)$. In the high-dimensional IV regression, we have

with $\varphi(s) = \sum_{j=1}^\infty j^{-a}e_j(s),a>1/2$ and $e_j(s)=\sqrt{2}\cos(\pi j s)$. We suppose that the object of interest is the average structural function

for $\mu\in L_2[0,1]$ with $\mu(s)=1 + \sum_{j=1}^\infty j^{-b}e_j(s)$ and $b>1/2$.\footnote{Note that $\int_0^1\mu(s)\ensuremath{\mathrm{d}} s=1$ and that $\varphi\in L_2[0,1]$.} The covariance operator $K=\ensuremath{\mathds{E}}[W\otimes Z]:L_2[0,1]\to L_2[0,1]$ simplifies to $K=\sum_{j=1}^\infty\frac{\rho}{j^2\pi^2}h_j\otimes e_j$ and its adjoint to $K^* = \sum_{j=1}^\infty\frac{\rho}{j^2\pi^2}e_j\otimes h_j$. It can be observed that $1\in\mathcal{N}(K)$, $\varphi_0=0$, and $\mu_0=1$, so the structural function $\varphi$ and average structural function $\langle\varphi,\mu\rangle$ are identified even though $K$ has a nontrivial null space.

To verify Assumption (ref), we observe that by Parseval's identity,

and

cf. Section (ref). Under homoskedasticity, we have $\Sigma = \sigma^2\ensuremath{\mathds{E}}[W\otimes W]=\sum_{j=1}^\infty\frac{\sigma^2}{j^2\pi^2}h_j\otimes h_j$, which implies that

with $\sigma_j=\frac{\rho}{j^2\pi^2}$. Therefore, we obtain

For $\gamma=1/2$, we require that the function $\mu_1$ has sufficient regularity, with $b>5/2$. For larger values of $\beta$, more regularity in $\varphi_1$ is needed, which corresponds to larger values of $a$. In this case, $\pi_n^2\sim n$, and Assumption (ref) (ii) requires $n\alpha_n^2\to 0$, ensuring that $n\alpha_n=o(\pi_n)$. Consequently, Corollary (ref) shows that

Monte Carlo Experiments

In this section, we examine the validity of our asymptotic theory via Monte Carlo simulations. The data-generating process (DGP) is specified as follows:

where we use the basis $e_j(x)=\sqrt{2}\sin(j\pi x),j\geq 1$ and set $C_J$ such that the density integrates to 1; see also hall2005nonparametric. The parameter $J$ controls the degree of identification, with $K:L_2[0,1]\to L_2[0,1]$ having an infinite-dimensional null space whenever $J<\infty$. Specifically, $K$ has a $J$-dimensional range and

Accordingly, we consider three cases: $J\in\{1,2,\infty\}$, where $J=\infty$ corresponds to the fully identified model.

We simulate samples of size $n\in\{100,500\}$ with $5,000$ replications. For simplicity, we focus on the Tikhonov-regularized estimator with a product of Gaussian kernels to estimate the best approximation $\varphi_1$. The bandwidth parameter is selected via Silverman's rule of thumb, and the regularization parameter is chosen via leave-one-out cross-validation; see babii2020honest and centorrino2014data for details on the numerical implementation. All the integrals are approximated by Riemann sums on a uniform grid in $[0,1]$, and infinite sums are truncated at a large finite value beyond which the numerical results remain unchanged.

Table (ref) shows the empirical $L_2$ and $L_\infty$ errors for the three degrees of identification, as measured by $J\in\{1,2,\infty\}$. The estimation errors decrease as the dimension of the null space shrinks, for both $n=100$ and $n=500$. Larger sample sizes also result in smaller errors, which is consistent with our asymptotic theory. Figure (ref) displays the true structural function $\varphi$, the mean of the estimates $\ensuremath{\mathds{E}}\hat\varphi$, and the pointwise 90% empirical confidence bands based on 5,000 replications. The results align with those in Table (ref). Note that for $J=1$, we can only recover information related to the first basis vector, illustrating an extreme failure of the completeness condition. Nevertheless, even in this case, a significant amount of information about the structural function can be learned, especially when $n=500$.

Conclusions

This paper develops a theory for nonidentified linear models, using nonparametric IV regression and high-dimensional regressions as illustrative examples. Identification failures arise because of the noninjectivity of covariance or conditional expectation operators. We show that when these operators are noninjective, spectral-regularized estimators converge to the best approximation of the structural parameter in the orthogonal complement of the operator's null space, and we derive new bounds on the risk in both uniform and Hilbert space norms.

This framework offers an appealing projection interpretation for nonparametric IV regression under identification failure, which is analogous to that of ordinary least squares under misspecification. We describe cases where the best approximation coincides with or closely approximates the structural parameter, highlighting the usefulness of these results in partial identification settings. Additionally, incorporating smoothness constraints into the regularization procedure can yield more precise estimates in small samples; see babii2017unobservables for an approach using Sobolev-type penalties. Finally, under identification failure, the asymptotic distribution of a continuous linear functional can transition between weighted chi-square and Gaussian components.

{ To conclude, despite a large amount of research over the last two decades, there is only a small amount of empirical work using nonparametric IV regression. One reason is that it involves identifying conditions that are difficult to validate empirically and our paper shows that numerous results may be obtained without these conditions. Economic analysis often hinges on strong functional restrictions that may lead to erroneous counterfactual conclusions and policy recommendations. Nonparametric IV regression is useful for obtaining robust and reliable estimates; see e.g. compiani2022market.}

\setcounter{page}{1} \setcounter{section}{0} \setcounter{equation}{0} \setcounter{table}{0} \setcounter{figure}{0}

Spectral Regularization

In this section, we extend our results to encompass a range of estimators based on spectral regularization. This framework includes iterated ridge (Tikhonov) regression, functional gradient descent (Landweber-Fridman), and functional PCA (spectral cut-off) methods. Consider the linear operator equation:

where $K:\mathcal{E}\to\mathcal{H}$ is a linear operator between Hilbert spaces $\mathcal{E}$ and $\mathcal{H}$. We assume that $K$ is bounded, with $\|K\|^2\leq \Lambda$ for some $\Lambda<\infty$, though it may not be compact. Then $K^*K:\mathcal{E}\to\mathcal{E}$ is a normal operator with the spectral decomposition:

where $\sigma(K^*K)$ denotes the spectrum of $K^*K$ and $E$ is the resolution of the identity; see rudin1991functional, Theorem 12.23. For a bounded Borel function $g:[0,\Lambda]\to \ensuremath{\mathbf{R}}$, we define:

If $K$ is compact, the spectrum of $K^*K$ is countable, and we have

where $E_j=h_j\otimes e_j$ projects on the eigenspace corresponding to the eigenvalue $\sigma_j^2$.

We aim to recover the best approximation to the structural parameter $\varphi$ when estimates $(\hat K,\hat r)$ of $(K,r)$ are available with $\|\hat K\|^2\leq\Lambda$ almost surely. To do this, we consider an extended version of Assumption (ref):

The following two cases are of particular interest:

In the mildly ill-posed case, the eigenvalues of $K^*K$ can decline exponentially, provided that the Fourier coefficients of $\varphi_1$ decay at the same rate. In contrast, the severely ill-posed case accommodates less regular $\varphi_1$ and $K$.

The spectral regularization scheme is defined by a family of bounded Borel functions $g_\alpha: [0,\infty)\to\ensuremath{\mathbf{R}}$, where $\alpha>0$ is a regularization parameter such that $\lim_{\alpha\to 0}g_\alpha(\lambda)=\lambda^{-1}$. When $\hat K$ is bounded, the regularized estimator is defined by

Our theoretical results require the following assumption on the regularization scheme:

The following regularization schemes satisfy Assumption (ref) in both the mildly and severely ill-posed cases:\footnote{In the severely ill-posed case, the function $\lambda\mapsto s_\beta^2(\lambda)/\lambda^\beta$ is nonincreasing only on $(0,1/e]$ and $s_\beta$ is not defined at $\lambda=1$. To address these issues, we assume that the norm on $\mathcal{H}$ is scaled such that $\|K\|^2\leq 1/e$.}

The constant $\beta_0$ is known as the qualification of the regularization scheme. It is well-established that simple Tikhonov regularization experiences a saturation effect, limiting bias convergence to $O(\alpha^2)$. This issue can be resolved using iterated Tikhonov regularization, similar to higher-order kernels in nonparametric kernel estimation.

To provide a spectral interpretation of gradient descent in point 4, consider the minimization problem:

where the gradient is $\nabla Q(\phi):=K^*(K\phi - r)$. The gradient descent with a constant step size $c\in(0,1/\Lambda)$, leads to the update rule:

Iterating this update rule recursively, we obtain

Assuming that the initial value is $\phi_0=0$ and replacing $(K,r)$ by $(\hat K,\hat r)$, we obtain

with $g_{\alpha,c}$ and $\alpha=1/m$ as in equation ((ref)).

The following theorem establishes the convergence rate of $\hat\varphi$ to $\varphi_1$ for general regularization schemes:

Before presenting the proof, note that:

The following theorem establishes the uniform convergence rates for a broad class of regularized estimators:

Proofs of Main Results

The following proposition provides low-level conditions for Assumptions (ref) and (ref) in the context of nonparametric IV regression estimated with kernel smoothing. Let $C^s_M$ denote the H\"{o}lder class.

Partial Identification

In this section, we discuss how our results can be applied to partial identification, drawing on ideas from freyberger2017completeness and santos2012inference. This section is intended primarily for illustrative purposes and a detailed exploration of the partial identification approach is left for future research.

Define the $L_\infty$-diameter of the identified set as

Suppose that both $\varphi$ and $\varphi_1$ belong to the Lipschitz smoothness class

If the best approximation $\varphi_1$ and the diameter of the identified set $|I_0|$ are known, then $\varphi_1\pm|I_0|$ is a valid identified set for the structural function $\varphi$. Our results establish that we can estimate the best approximation consistently. The remaining task is to infer the diameter of the identified set.

To this end, for a fixed $\varepsilon>0$, we consider the following hypotheses:

following the approach in freyberger2017completeness to construct {a test statistic}. Under $H_0$, there exist functions $\phi_1,\phi_2\in I_0$ such that $\|\phi_1-\phi_2\|_\infty\geq \varepsilon$. For $\phi=\phi_1-\phi_2$, we have $K\phi = 0$, $\|\phi\|_s\leq 2C$, and $\|\phi\|_\infty\geq \varepsilon$. This motivates the following {statistic}

where $\mathscr{F}_\varepsilon = \left\{\phi\in L_\infty([0,1]^p):\; \|\phi\|_s \leq 2C/\varepsilon \right\}$. Clearly, under $H_0$, we have $T_\varepsilon=0$. Define the sample counterpart as

where $\hat K$ is a consistent estimator of $K$.

For NPIV regression, suppose that the density $\hat f_{ZW}$ is estimated using a series estimator, and the { statistic} $\hat T_\varepsilon$ is computed over a $J$-dimensional sieve. Theorem 2 in freyberger2017completeness shows that

consistently estimates the upper bound on $|I_0|$, where $\bar C$ is the largest $\varepsilon$ for which $\{\phi:\; \|\phi\|_s\leq 2C,\|\phi\|_\infty\geq \varepsilon \}\ne \emptyset$ and $q_{1-\alpha}$ is a critical value.

Combining this result with our result on the $L_\infty$ consistency for the best approximation, the interval $[\hat\varphi-\hat\varepsilon,\hat\varphi+\hat\varepsilon]$ serves as a valid set estimator for $\varphi$.\footnote{We thank the anonymous referee for highlighting this connection.}

Extreme Nonidentification

In this section, we derive an approximation of the large sample distribution of the Tikhonov-regularized estimators in cases of extreme nonidentification. Remarkably, we show that the asymptotic distribution is a weighted sum of independent chi-squared random variables. This result serves as a foundation for Section (ref), where we analyze a transition between chi-squared and Gaussian limits in intermediate cases.\footnote{Extreme nonidentification also relates to the weak instruments problem.}

High-dimensional Regressions

In high-dimensional regressions, the strength of identification is determined by the covariance operator of $Z$ and $W$. In cases of extreme nonidentification, this operator is degenerate, leading to the following result.

The theorem states weak convergence in the topology of the Hilbert space $\mathcal{E}$, which cannot be achieved in cases of regular identification. Furthermore, it can be shown that the distribution of the inner products of $J(h)$ with $\mu\in\mathcal{E}$ is a weighted sum of chi-squared random variables. Interestingly, Theorem (ref) does not require that $\alpha_n\to 0$ as $n\to\infty$.

Nonparametric IV Regression

In nonparametric IV regression, the strength of identification is described by the conditional expectation operator. In the extreme nonidentified case,

where $L_{2,0}(Z)=\left\{\phi\in L_2(Z):\; \ensuremath{\mathds{E}}\phi(Z)=0 \right\}$, and $K$ is a degenerate conditional expectation operator. Consider the operator $T:\phi\mapsto \ensuremath{\mathds{E}}_{X}[\phi(X)h(X,X')]$ on $L_2(X)$, where $\ensuremath{\mathds{E}}_X$ denotes expectation with respect to $X=(Y,Z,W)$ only, $X'$ is an independent copy of $X$, and

Here, $P_0$ is the projection operator on $L_{2,0}$ and $\bar K(v)=\int K_w(v-u)K_w(u)\ensuremath{\mathrm{d}} u$ is the convolution kernel. We assume the following mild conditions on the distribution of the data:

Let $h_z$ and $h_w$ be the bandwidth parameters for smoothing over $Z$ and $W$, respectively. The following result holds:

\setcounter{page}{1} \setcounter{section}{0} \setcounter{equation}{0} \setcounter{table}{0} \setcounter{figure}{0}

Generalized Inverse

In this section, we collect some facts about the generalized inverse operator from operator theory; see also carrasco2007linear for a comprehensive review of different aspects of the theory of ill-posed inverse models in econometrics. Let $\varphi\in\mathcal{E}$ be a structural parameter in a Hilbert space $\mathcal{E}$ and let $K:\mathcal{E}\to\mathcal{H}$ be a bounded linear operator mapping to a Hilbert {space} $\mathcal{H}$. Consider the functional equation

If the operator $K$ is not one-to-one, then the structural parameter $\varphi$ is not point identified, and the identified set is a closed linear manifold described as $\Phi^\mathrm{ID}=\varphi+\mathcal{N}(K)$, where $\mathcal{N}(K)=\{\phi:K\phi = 0\}$ is the null space of $K$; see Figure (ref). The following result offers equivalent characterizations of the identified set; see groetsch1977generalized, Theorem 3.1.1 for a formal proof.

The generalized inverse is formally defined below.

For nonidentified linear models, the generalized inverse maps $r$ to the unique minimal norm element of $I_0$. It follows from equation ((ref)) that $\varphi_1$ is a projection of $0$ on the identified set. Therefore, $\varphi_1$ is the projection of the structural parameter $\varphi$ on the orthogonal complement to the null space $\mathcal{N}(K)^\perp$, see Figure (ref), and we call $\varphi_1$ the best approximation to the structural parameter $\varphi$. The generalized inverse operator is typically a discontinuous map, as illustrated in the following proposition; see groetsch1977generalized, pp.117-118 for more details.

The following example illustrates this when $K$ is an integral operator on spaces of square-integrable functions.

It is worth stressing that in the NPIV model, the kernel function $k$ is typically a non-degenerate probability density function. Moreover, in econometric applications, $r$ is usually estimated from the data, so that $K^\dagger\hat r\xrightarrow{p} K^\dagger r=\varphi_1$ may not hold even when $\hat r\xrightarrow{p} r$ due to the discontinuity of $K^\dagger$.\footnote{In practice, the situation is even more complex because the operator $K$ is also estimated from the data.} In other words, we are faced with an ill-posed inverse problem. Tikhonov regularization can be understood as a method that smooths out the discontinuities of the generalized inverse $(K^*K)^\dagger$.\footnote{By Proposition (ref), solving $K\varphi = r$ is equivalent to solving $K^*K\varphi = K^*r$. The latter is more attractive to work with because the spectral theory of self-adjoint operators in Hilbert spaces applies to $K^*K$.}

Degenerate U-statistics in Hilbert Spaces

Wiener-It\^{o} Integral

This section reviews key results on the asymptotic distribution of degenerate U-statistics in Hilbert spaces. Let $(\mathcal{X},\Sigma,\mu)$ be a measure space, and let $H$ be a separable Hilbert space. We denote by $L_2(\mathcal{X}^m,H)$ the space of functions $f:\mathcal{X}^m\to H$ satisfying $\ensuremath{\mathds{E}}\|f(X_1,\dots,X_m)\|^2<\infty$. A stochastic process $\left\{\mathbb{W}(A),A\in\Sigma_\mu\right\}$, indexed by the $\sigma$-field $\Sigma_\mu=\{A\in\Sigma:\; \mu(A)<\infty\}$ is called a Gaussian random measure if:

Now, let $(A_k)_{k=1}^K$ be pairwise disjoint sets in $\Sigma_\mu$, and consider the set $S_m$ of simple functions $f\in L_2(\mathcal{X}^m,H)$ of the form

where $c_{i_1,\dots,i_m}=0$ if any two indices $i_1,\dots,i_m$ are equal, i.e., $f$ vanishes on the diagonal. For a Gaussian random measure $\mathbb{W}$ corresponding to $P$, we define the random operator $J_m:S_m\to H$ by

The operator $J_m$ has three notable properties:

Since $S_m$ is dense in $L_2(\mathcal{X}^m,H)$, $J_m$ can be extended to a continuous linear isometry on $L_2(\mathcal{X}^m,H)$, known as the Wiener-It\^{o} integral.

Central Limit Theorem

Consider a probability space $(\mathcal{X},\Sigma,P)$, where $\mathcal{X}$ is a separable metric space and $\Sigma$ is a Borel $\sigma$-algebra. Let $(X_i)_{i=1}^n$ be i.i.d. random variables taking values in $(\mathcal{X},\Sigma,P)$. Define a symmetric function $h:\mathcal{X}\times\mathcal{X}\to H$, where $H$ is a separable Hilbert space. The $H$-valued {$U$-statistic} of degree $2$ is given by

The {$U$-statistic} is called degenerate if $\ensuremath{\mathds{E}} h(x_1,X_2)=0$. The following theorem provides the limiting distribution of degenerate $H$-valued $U$-statistics; see korolyuk2013theory, Theorem 4.10.2 for a detailed proof.