Theory of Weak Identification in Semiparametric Models

Theory of Weak Identification in Semiparametric Models

\gdef\@thefnmark\@footnotetext{ I thank Anna Mikusheva, Isaiah Andrews, Victor Chernozhukov, Whitney Newey, Jerry Hausman, Hidehiko Ichimura, Kengo Kato, Rachael Meager, Peter Hull, and anonymous referees for helpful comments and suggestions. This work is supported by the Richard N.\ Rosett Faculty Fellowship and the Liew Family Faculty Fellowship at the University of Chicago Booth School of Business.}

Introduction

Weak identification arises in a wide range of empirical settings. A leading example is the linear instrumental variables (IV) model in which the instruments and endogenous regressors are barely correlated. When this happens, classical asymptotic theory is known to yield poor approximations to the behavior of familiar statistics ss1997. We encounter this problem in various other contexts: sw2000 analyze weak identification in generalized method of moments (GMM) models; ac2012 in extremum estimation models; am2016 in dynamic stochastic general equilibrium (DSGE) models. Many estimators of weakly identified parameters exhibit inconsistency and bias, and, as a consequence, standard inference procedures such as $t$- and Wald tests may have substantially distorted sizes d1997,hp2015. Following these problems, a vast amount of theoretical work has been published.

The theoretical literature on weak identification is largely confined to specific estimation and inference procedures in specific models. Many fundamental questions---such as what is the common cause of weak identification, what is a general guideline to find well-behaved statistics, and what is semiparametric efficiency in the presence of weak identification---are left unanswered. Such exploration is essential, however, not only to facilitate unified understanding of the phenomenon but to measure performance of different procedures and develop systematic methods for estimation and inference.

This paper studies weak identification from the perspective of semiparametric theory (\Citealp[Chapter 25]{bkrw1993,v1998}). This literature views parameters as functions defined on the probability manifold and relates their asymptotic properties to the functions' local behaviors in response to local perturbations of the probability. While strongly identified parameters often translate to differentiable functions, weakly identified parameters emerge as functions that are discontinuous at the probability to which we asymptote. We call such functions {\em weakly regular} in reference to differentiable functions called {\em regular}. As an immediate consequence of this discontinuity, we derive---without reference to a specific estimation or inference procedure---that there exists neither a consistent estimator, a consistent test, nor an equivariant (hence pivotal) estimator when the parameter is weakly regular. The local approximations of weakly regular parameters are homogeneous of degree zero and essentially nonlinear, becoming the root cause of non\hypGaussian nonpivotal asymptotic distributions witnessed throughout the literature ss1997,sw2000,ac2012. To circumvent the problem of nonlinearity, we explore weak regularity from the standpoint of {\em regular} parameters.

We show that every weakly regular parameter can be represented as a nonlinear function of the local parameter of some underlying regular parameter. Finding such a parameter lets us reformulate the model in a way that it consists only of regular parameters, providing a tractable foundation on which to discuss estimation and inference. This conforms with the repeated observation in the literature that reduction to regular parameters (usually referred to as “reduced\hypform parameters”) substantially simplifies the problems; we generalize this observation to arbitrary semiparametric models and show that there exists an underlying regular parameter for every weakly regular parameter. However, underlying regular parameters are not unique, and statistical analyses based on different underlying parameters may yield different performances. This gives rise to the need for criteria to choose an underlying parameter.

We introduce two desirable properties of underlying parameters. In semiparametric models, the space of probability distributions can be much bigger than the space of the parameter of interest. Consequently, there are many directions of perturbations that do not matter to the parameter of interest. Intuitively, a good underlying parameter would be sensitive to all perturbations that matter {\em and} not sensitive to any perturbations that are irrelevant. The first property, {\em sufficiency}, ensures that all perturbations that affect identification of the weakly regular parameter affect the underlying parameter; for example, reduced\hypform coefficients in linear IV that miss an instrument are not sufficient. The second property, {\em minimality}, guarantees that the underlying parameter does not include a nuisance parameter. In short, a best underlying regular parameter is minimal and sufficient. We show existence of minimal sufficient underlying parameters and present examples.

With these concepts, we define a notion of efficiency for weakly regular parameters. Efficient estimation under weak identification has received little treatment as non\hypGaussianity and nonpivotality of the asymptotic distributions render the classical efficiency concepts inapplicable in their direct forms. Our formulation helps decompose estimation of weakly regular parameters into estimation of the minimal sufficient underlying regular parameters and their transformation. Being regular, the underlying parameters are susceptible to the classical convolution theorem. Moreover, if the estimators of the underlying parameters contain noise, their transformations, too, would suffer from noise. Such noise can then be eliminated by taking expectation with respect to it. Conceptually, this corresponds to applying the Rao\hypBlackwell theorem to the local asymptotic representations of the estimators. The resulting conditional expectation estimators are more concentrated around the same means. We name this procedure {\em local asymptotic Rao\hypBlackwellization (LAR)}. If such improvement is impossible, we call the estimators {\em weakly efficient}. We put the qualifier “weakly” as weakly efficient estimators are not unique. We can interpret weak efficiency as a generalization of classical efficiency to accommodate non\hypdifferentiable transformations.

We apply our results to linear IV and nonlinear regression models and present weakly efficient estimators. In linear IV, the two\hypstage least squares (2SLS) and even the optimal IV estimators are shown to be inefficient in the presence of heteroskedasticity and, under the availability of an efficient estimator of the reduced\hypform coefficients, admit transformations into weakly efficient estimators. In nonlinear regression, a simple least\hypsquares estimator is shown to be inefficient under heteroskedasticity, and we obtain a weakly efficient estimator when the heteroskedastic structure can be estimated. In nonlinear GMM, the possibility of improvement depends on specificity of the model. Simulation shows how weakly efficient estimators behave under weak and strong identification asymptotics in a linear IV model.

There is a large body of literature that studies the optimality of statistical procedures under weak identification. To name a few, mw2015 study estimation that minimizes the weighted average risk when the asymptotic distribution of the statistics is known; ams2006 develop optimal conditional likelihood ratio tests in linear IV models with normal homoskedastic errors; m2011 studies efficient inference under a weak convergence assumption.

The rest of the paper is organized as follows. (ref) defines weak identification in semiparametric models, gives impossibility results, and introduces the notion of underlying regular parameters. (ref) introduces sufficiency and minimality of underlying regular parameters. (ref) derives LAR for the estimation of weakly regular parameters, whence we define weak efficiency. (ref) discusses application of LAR to heteroskedastic linear IV models and provides simulation results. (ref) concludes. The Appendix contains proofs and the local\hypto\hypsingularity linear IV model.

Weak Identification in Semiparametric Models

Weak identification is a problem that arises relative to the sample size. As such, it is modeled by one of two ways: (1) start with a primitive (global) model and consider a drifting sequence, “weak identification embedding”, toward identification failure ss1997; (2) skip the primitive model and start with a finite\hypsample situation equivalent to a local expansion around identification failure d1997. Both approahces are inherently local, although distinction may seem ambiguous in linear models where global and local settings coincide. To establish a framework for efficiency, it is essential to understand the type of local expansions that can emerge from a primitive model. Therefore, we develop a general theory for (1).

Suppose we observe i.i.d.\ random variables $X_1, \dots, X_n$ from the sample space $(\mathcal{X}, \mathscr{A})$. The set of possible distributions of $X_i$ is denoted by $\mathcal{P}$ and called the {\em model}. To obtain fruitful asymptotics around a distribution $P\in\mathcal{P}$, we consider a {\em path} of distributions $Q_t \in \mathcal{P}$ indexed by a real number $t\in(0,1]$ that is {\em differentiable in quadratic mean (DQM)} at $P$, that is, there exists a measurable function $g : \mathcal{X} \to \mathbb{R}$ such that \[ \int_{\mathcal{X}} \biggl[ \frac{dQ_t^{1/2}-dP^{1/2}}{t} - \frac{1}{2} g dP^{1/2} \biggr]^2 \operatorname*{\mathchoice{ \,\longrightarrow\,}{ \rightarrow}{ \rightarrow}{ \rightarrow} } 0 \qquad \text{as} \qquad t \to 0, \] where the integral is understood with respect to a $\sigma$\hypfinite measure dominating $P$ and $Q_t$, and $dP$ and $dQ_t$ are their Radon\hypNikodym derivatives. This convergence is denoted by $Q_t \operatorname*{\mathchoice{ \,\longrightarrow\,}{ \rightarrow}{ \rightarrow}{ \rightarrow} }^{\text{DQM}} P$, and we call $g$ the {\em (model) score} induced by the path $\{Q_t\}$. \footnote{Throughout the paper, dependence of $g$ on $\{Q_t\}$ will be implied by the context.} The idea behind asymptotic approximation theory is that the path of “alternatives" $\{Q_t\}$ that approaches $P$ at the same rate as the path of “samples” $\{\hat{P}_n\}$ is not deterministically distinguishable in the limit and hence yields an approximation that reflects finite sample uncertainty; therefore, $t=1/\sqrt{n}$ under local asymptotic normality \Citep[Lemma 25.14]{v1998}, and in a minor abuse of notation we denote $Q_{1/\sqrt{n}}$ by $Q_n$.

We often do not consider every possible path in $\mathcal{P}$; let $\mathscr{P}_P$ denote the set of paths we consider that tend to $P$ in DQM. Since there is little chance of misunderstanding, we denote $\{Q_t\}$ simply by $Q_t$, for example, $Q_t\in\mathscr{P}_P$; therefore, $Q_t$ can refer to the entire path $\{Q_t\}$ or an element $Q_t$ for a specific $t$, depending on the context. The set $\dot{\mathcal{P}}_P$ of scores $g$ induced by the paths in $\mathscr{P}_P$ is called the {\em tangent set} at $P$. It is clear that $\dot{\mathcal{P}}_P$ is a subset of zero\hypmean functions in $L_2(P)$. \footnote{In this sense, $\dot{\mathcal{P}}$ is the set of {\em equivalence classes} of scores, to be precise.} Depending on the structures of $\mathcal{P}$ and $\mathscr{P}_P$, the tangent set might be a linear space, a cone, \footnote{A subset $X$ of a linear space is called a {\em cone} if $x\in X$ implies $ax\in X$ for every $a>0$.} or a set without much structure; we assume that $\mathcal{P}$ and $\mathscr{P}_P$ are nice enough that the induced tangent set is linear. For this reason, we call the tangent set the {\em tangent space}. The tangent space can be considered the local approximation of the model by a linear vector space. Finally, a parameter $\psi : \mathcal{P} \to \mathbb{D}$ is defined as a map from the model $\mathcal{P}$ to a Banach space $\mathbb{D}$.

If the parameter $\psi:\mathcal{P}\to\mathbb{D}$ is differentiable in a suitable sense, we may approximate the change in the parameter along a path by a linear map from the tangent space $\dot{\mathcal{P}}_P$ to $\mathbb{D}$. Any infinitesimal perturbation of distribution $P$ then leads to a linear perturbation of the parameter $\psi$. Such a parameter is known to behave well and is said to be {\em regular}. The appropriate notion of differentiability is as follows.

Weakly Regular Parameters

Now we define a weakly identified parameter. Let $\mathcal{P}_\beta$ be a subset of $\mathcal{P}$ on which a parameter $\beta$ is uniquely defined. \footnote{$\mathcal{P}\setminus\mathcal{P}_\beta$ may contain distributions that we simply deem inconceivable as well as distributions that do not identify $\beta$.} As the problem of weak identification arises when the population distribution is close to a point of identification failure, we model the situation by a path that takes values in $\mathcal{P}_\beta$ and approaches a point outside of $\mathcal{P}_\beta$. However, not all such sequences are appropriate to consider. If the path approaches $\mathcal{P}\setminus\mathcal{P}_\beta$ too rapidly, $\beta$ may not be identified in the first\hyporder local expansion (tangent space) of $\mathcal{P}$. To avoid this, we focus on scores that are associated with unique limiting values of $\beta$.

Consequently, this paper does not cover faster\hypthan\hyp$\sqrt{n}$ weak identification. From the observation that $P$ is not in $\mathcal{P}_\beta$, we see that $\dot{\mathcal{P}}_{P,\beta}$ is only a cone.

If the set of paths $\mathscr{P}_P$ is much richer than $\mathscr{P}_{P,\beta}$ in a way that $\operatorname{Span}\dot{\mathcal{P}}_{P,\beta}$ is a strict subset of $\dot{\mathcal{P}}_P$, then there exists a superfluously rich side of the model on which $\beta$ is not even defined. Since it is meaningless to consider such parts of the model when one's focus is on the parameter $\beta$, we assume innocuously that $\overline{\operatorname{Span}}\,\dot{\mathcal{P}}_{P,\beta}=\overline{\dot{\mathcal{P}}_P}$. \footnote{Later on we define the underlying regular parameter on the whole of $\mathcal{P}$, so it is actually harmful to require that the parameter be regular on the unconsidered realm of the model.}

Now we define the weakly identified parameter under the name {\em weakly regular parameter}. We henceforth shun the use of the qualifier “weakly identified" since weak identification in the literature may not always exclude cases of in fact {\em no} identification. In this paper, we require that weakly regular parameters are identified at every fixed $n$ in that there exists a unique value of the parameter for any given $Q_n$ along a path. Moreover, we require that the parameters remain identified in the limit in the sense that there exists a unique limiting value of the parameter for each score $g$ in $\dot{\mathcal{P}}_{P,\beta}$. Let $\mathbb{B}$ be another Banach space on which a weakly regular parameter will be defined.

The definition says that the value to which a weakly regular parameter converges changes as we consider different paths. Moreover, this dependence is homogeneous of degree zero, hence essentially nonlinear and discontinuous at $g=0$. This makes consistent estimation impossible and asymptotic distribution nonstandard ((ref)).

Now we introduce examples and show how they satisfy our definition.

Fundamental Impossibility

The utility of our theoretical formalism can be readily harvested in the following theorem. It gives a formal proof to the conventional wisdom that a “weakly identified" parameter cannot be estimated consistently or pivotally ss1997,sw2000,ac2012---but not as a characteristic of a specific estimation method---as a direct consequence of the characteristic of the model. \footnote{Consistent estimation may be possible in linear IV models if the number of weak instruments tends to infinity and some other conditions are met. In this case, the structural parameter is not weakly regular.} This result can also be viewed as a generalized proof of nonexistence of a consistent test conjectured by hhm2011. \footnote{Their setup can be translated into ours by taking $\mathbb{B}$ to be the product space for two estimators compared in the Hausman test, observing that a regular parameter is trivially weakly regular.} Distinct but related are the impossibility results by d1997 and hp2015; their setup is a generalization of the weak linear IV structure whereas our setup is a generalization of the weak identification phenomenon. Indeed, d1997 shows nonexistence of bounded confidence sets (which is “stronger” than nonexistence of consistent estimators) while there exist weakly regular parameters that admit bounded confidence sets ; \footnote{A bounded weakly regular parameter trivially admits a bounded confidence set.} hp2015 show the impossibility of unbiased estimation while there exist weakly regular parameters that admit unbiased estimation aa2017.

Impossibility of equivariant estimation implies that the asymptotic distribution of any estimator of a weakly regular parameter, when centered at the true value, is nonpivotal and not consistently estimable. However, it does not preclude the possibility that there exist {\em test statistics} whose distributions are pivotal or consistently estimable. In fact, almost any reasonable inference procedure would be based on statistics whose asymptotic distributions are known or estimable; hence, the problem of estimation and the problem of inference bear quite distinct aspects when it comes to weakly regular parameters. This is in stark contrast to the classical context of regular parameters, in which efficient estimation and “efficient” inference are closely related to each other. \footnote{\Citet[Chapter 25]{v1998} states that “[s]emiparametric testing theory has little more to offer than the comforting conclusion that tests based on efficient estimators are efficient.”} This separation partly explains the specialty of current literature on inference problems regarding weak identification.

Underlying Regular Parameters

The idea to analyze weak regularity is that in many cases there exists another parameter that is regular and whose local parameter controls the limit behavior of the weakly regular parameter. In the literature, such a parameter is known as the “reduced\hypform parameter” and is utilized in various robust inference procedures under weak identification. \footnote{On the other hand, the weakly regular parameter is often referred to as the “structural parameter.”} Then, the weakly regular parameter acts by itself as a transformation of the local parameter of some “underlying" regular parameter; in other words, it is sufficient to know the value of (the local parameter of) the underlying regular parameter in order to infer the value of the weakly regular parameter around the point of identification failure. We now formalize this idea.

This definition requires that knowing the local parameter of the underlying regular parameter is enough to recover the value of the weakly regular parameter, that is, the reduction of information from knowing $g$ to knowing $\dot{\psi}_Pg$ does not impair the ability to discern $\beta$ in the limit.

As it turns out, it is straightforward to show that an underlying parameter always exists. If we regard the root likelihood ratio $Q\mapsto dQ^{1/2}/dP^{1/2}$ as a parameter, it is trivially an underlying regular parameter for any weakly regular parameter. However, whether there exists an underlying regular parameter that admits $\sqrt{n}$ estimation is a different matter. For this, we need to investigate each model separately.

Below, we see that the natural parameters that appear in examples constitute underlying regular parameters.

Minimal Sufficient Underlying Regular Parameters

The underlying regular parameters are not unique, and different parameters may exhibit different properties. This section develops two desirable properties for underlying parameters. The motivation is analogous to classical semiparametric efficiency. In a model that contains both a parameter of interest $\beta$ and a nuisance parameter, there is a variation of the data that is informative of $\beta$ and a variation that is not informative of $\beta$. The classical theory extracts “pure” variation for $\beta$ and makes an estimator depend only thereon. If $\beta$ is regular, such estimators share a unique efficient distribution by the virtue of differentiability. In other words, the following two observations hold simultaneously: (1) desirable estimators depend only on pure variation; (2) desirable estimators share a unique distribution.

In the context of weakly regular $\beta$, (2) is no longer attainable due to (ref) while (1) still is. In particular, we can make an estimator $\hat{\psi}$ depend only on pure variation for $\beta$ and then construct an estimator of $\beta$ using $\hat{\psi}$. Such $\psi$ plays the role of extracting pure Gaussian variation relevant to $\beta$. In the local expansion, this means that the local parameter $\dot{\psi}_P$ shares the same set of nuisance scores as $\beta_P$. Note that these concepts need to be developed in local terms since the nuisance scores depend on the point of local expansion $P$.

Nuisance Tangent Spaces

This section defines the space of nuisance scores for a weakly regular parameter. Two points deserve attention. First, a weakly regular parameter is not defined on some probability distributions. Therefore, we do not want to deem a score nuisance if it affects identification of the weakly regular parameter. Second, a weakly regular parameter is not linear in the local expansion. This calls for a way to discuss nuisance\hypness for nonlinear maps.

In the classical semiparametric theory, local parameters are linearly related to the score, leading to a very nice use of the theory of linear operators bkrw1993. The following definition extends the key notion to nonlinear maps defined on a cone.

Now we define the nuisance tangent space for $\beta$. Any score that does not affect identification or the value of $\beta$ only contains information about the path that is irrelevant to $\beta$, hence nuisance. Since the tangent space $\dot{\mathcal{P}}_P$ is linear, we separate the space into the space spanned by nuisance scores and its orthocomplement. That orthocomplement, by construction, only contains scores relevant to $\beta$.

The definition indicates that $\tilde{g}\in N(\beta_P)$ means $g+\tilde{g}\in\dot{\mathcal{P}}_{P,\beta}$ and $\beta_P(g+\tilde{g})=\beta_P(g)$ for every $g\in\dot{\mathcal{P}}_{P,\beta}$; the first condition is the preservation of identification and the second the preservation of the value of $\beta$. That is, the perturbation of $P$ in the direction of $\tilde{g}$ does not affect identification or distinction of $\beta$. The flip side is that if $\tilde{g}\notin N(\beta_P)$, then there exists $g\in\dot{\mathcal{P}}_{P,\beta}$ such that either $g+\tilde{g}\notin\dot{\mathcal{P}}_{P,\beta}$ or $\beta_P(g+\tilde{g})\neq\beta_P(g)$, so we want our statistical procedures to be sensitive to these directions.

This lemma shows that the nuisance tangent space is indeed a linear space.

Now we derive the nuisance tangent spaces and “efficient scores” in examples.

Sufficiency and Minimality of Underlying Regular Parameters

The underlying regular parameters are characterized by the span of their nuisance scores. The first property we want in the underlying regular parameter is that it contain all relevant information about $\beta$.

If $\psi$ is sufficient, then any perturbation that does not affect its local parameter does not affect $\beta$. The following example shows that an underlying regular parameter need not be sufficient.

Not surprisingly, the set of all reduced\hypform coefficients is sufficient.

The next property we want is that an underlying regular parameter captures only relevant information for the weakly regular parameter.

Minimality of $\psi$ requires the opposite of sufficiency. For a minimal $\psi$, any perturbation that does not affect $\beta$ does not affect $\psi$. In this sense, a minimal underlying parameter is free of nuisance parameters.

This theorem ensures that a minimal sufficient underlying parameter exists.

Minimal sufficiency {\em per se} is not strong enough to pin down the underlying parameter uniquely. However, minimal sufficient underlying parameters are almost equivalent to each other in terms of the efficient variation they can extract.

Let us look at examples of minimal sufficient underlying parameters.

Given a minimal sufficient underlying regular parameter, the problem of estimation or inference for a weakly regular parameter is translated without loss into a problem for the underlying parameter that is regular.

Weak Efficiency for Weakly Regular Parameters

This section defines a notion of efficiency for the estimators of a weakly regular parameter. The difficulty in defining efficiency is that their asymptotic distributions are nonstandard and nonpivotal ((ref)). Just as a weakly regular parameter being locally a nonlinear transformation of an underlying regular parameter, an estimator of a weakly regular parameter is often a nonlinear transformation of the estimator of an underlying regular parameter. Then, even when the estimator of the underlying parameter is Gaussian, its nonlinear transformation can in principle be anything.

A key observation is that if the estimator of the underlying parameter contains noise, its transformation also suffers from unnecessary noise. Our idea to define efficiency lies in consideration of such noise; if the estimator of the weakly regular parameter is asymptotically an appropriate transformation of an efficient estimator of a minimal underlying regular parameter, we call it {\em weakly efficient}. Here, the base estimator must be efficient for otherwise it is contaminated by noise, and the underlying parameter must be minimal for otherwise its transformation might be affected by the estimator of a nuisance component for $\beta$. If the underlying parameter is also sufficient, then the weakly efficient estimator may become efficient under strong identification asymptotics.

It is helpful to draw analogy with classical efficiency on regular parameters. Consider two {\em regular} parameters, $\nu\in\mathbb{B}$ and $\psi\in\mathbb{D}_\nu\subset\mathbb{D}$, related through a Hadamard differentiable map $\nu_\psi:\mathbb{D}_\nu\to\mathbb{B}$ by $\nu=\nu_\psi(\psi)$. Since a differentiable map can be approximated locally by a continuous linear map, we may assume that $\nu_\psi$ is continuous linear when two parameters admit consistent estimation. To construct an estimator of $\nu$ from an estimator of $\psi$ as $\hat{\nu}=T(\hat{\psi})$, there are two aspects to consider: (1) the efficiency of $\hat{\psi}$ and (2) the desirability of the map $T$. Note that when $\hat{\psi}$ takes values in $\mathbb{D}_\nu$, there is little motivation to choose $T$ other than $\nu_\psi$. Then, \Citet{v1991b} shows that if one has an efficient estimator $\hat{\psi}$ of $\psi$ that takes values in $\mathbb{D}_\nu$, the plug\hypin estimator $\nu_\psi(\hat{\psi})$ is efficient for $\nu$.

If, on the other hand, $\hat{\psi}$ takes values in a bigger space $\mathbb{D}$, we need to consider an optimal choice of $T$. Consider, for example, the strongly\hypidentified linear IV model with unconditional moment restrictions and overidentification, that is, $\mathbb{E}[z(y-x'\beta)]=0$ with $k>d$. Rewriting $\mathbb{E}[zz']^{-1}\mathbb{E}[z(y-x'\beta)]=\gamma-\pi\beta=0$, we can let $\nu=\beta$, $\psi=(\gamma,\pi)$, and $\nu_\psi(\psi)=\pi^\to\gamma$. Note that $\mathbb{B}=\mathbb{R}^d$, $\mathbb{D}=\mathbb{R}^{k\times d}\times\mathbb{R}^k$, and $\mathbb{D}_\nu$ is a subspace of $\mathbb{D}$ in which $\gamma$ is in the column space of $\pi$ and $\pi$ is of full column rank. The OLS estimator $(\hat{\gamma},\hat{\pi})$, which is efficient under unconditional moments, takes values outside of $\mathbb{D}_\nu$ in that $\hat{\gamma}$ falls outside the column space of $\hat{\pi}$ with probability $1$. Therefore, we have to extend $T$ that supports the bigger space $\mathbb{D}$. This can be considered as a problem of regressing $\hat{\gamma}$ on $\hat{\pi}$ for which the generalized least squares (GLS) estimation is possible since the variance of the error term $\hat{\gamma}-\hat{\pi}\beta$ can be consistently estimated, yielding an optimally weighted GMM estimator for $\beta$.

When $\beta$ is weakly regular, (ref) guarantees that one can find a minimal sufficient underlying regular parameter $\psi$ with which $\beta$ is related locally through $\beta_{P,\psi}$. The key difference is that $\beta_{P,\psi}$ is not a linear map; it is a continuous homogeneous\hypof\hypdegree\hypzero map. Nevertheless, we can construct an estimator of $\beta$ from an estimator of $\psi$ as $\hat{\beta}=T(\hat{\psi})$, taking into account the two aspects: (1) the effciency of $\hat{\psi}$ and (2) the desirability of $T$. The major consequence we must face, however, is that, since the relation of $\beta$ and $\psi$ is no longer linear, there are multiple choices of $T$ that are admissible from various perspectives. In fact, even when $\hat{\psi}$ takes values in $\mathbb{D}_{P,\beta}$, it may make sense to consider $T$ other than $\beta_{P,\beta}$, which we illustrate by an example.

In this section, we first define what it means for an estimator to be of the form $T(\hat{\psi})$. Then, we show that this estimator admits improvement via Rao\hypBlackwellization when an efficient estimator for $\psi$ is available. This allows us to define weak efficiency of an estimator among this class of estimators. Finally, we discuss estimators that are efficient under both strong and weak identification asymptotics.

Throughout this section, we assume that there exists a $\sqrt{n}$ efficient estimator of the minimal sufficient underlying regular parameter. Note, however, that not all regular parameters admit $\sqrt{n}$ consistent or efficient estimators in general.

Regular Estimators

We focus on estimators of $\beta$ that are transformations of regular estimators of $\psi$. First, recall the regular estimator for a regular parameter.

We restrict the class of estimators of $\beta$ to ones that asymptote to functions of estimators of a minimal underlying parameter. Considering a non\hypminimal underlying parameter per se is not a problem when such $\psi$ can be estimated efficiently, but we require that the estimator of $\beta$ depends only on its minimal component asymptotically.

The asymptotic distribution of a regular estimator follows straightforwardly from the continuous mapping theorem.

Weakly Efficient Estimators

We show that for any regular estimator of a weakly regular parameter, there exists another regular estimator that is weakly better in terms of convex loss. A strict improvement is possible unless the estimator is already a nonrandom transformation of an efficient estimator of the underlying parameter. For regular $\tilde{\beta}=S(\tilde{\psi})$, a particular improvement is given as the conditional expectation of $\tilde{\beta}$ conditional on an efficient estimator $\hat{\psi}$, that is, $\mathbb{E}[S(\tilde{\psi})\mid\hat{\psi}]$. To formalize the efficiency gain, however, we use the normalized expression $\tilde{\beta}=T(\sqrt{n}(\tilde{\psi}-\psi(P)))$ for unknown $T$ and $\psi(P)$.

Note that although $T$ and $\psi(P)$ are not known, we can construct a feasible estimator that is asymptotically equivalent to $\hat{\beta}$. Let $\tilde{\beta}=S(\tilde{\psi})$ where $\tilde{\psi}$ is an inefficient estimator of $\psi$ that is asymptotically normal. Then, the Rao\hypBlackwellized estimator for $\beta$ is calculated as follows: (1) compute efficient $\hat{\psi}$ and $\operatorname{Var}(\tilde{\psi}\mid\hat{\psi})$; (2) compute $\hat{\beta}=\mathbb{E}_m[S(\hat{\psi}+e_j)\mid\hat{\psi}]$ where $e_1,\dots,e_m$ are drawn i.i.d.\ from $N(0,\operatorname{Var}(\tilde{\psi}\mid\hat{\psi}))$. This feasible $\hat{\beta}$ has the same property as $\hat{\beta}$ in (ref).

Now we define our efficiency concept and introduce examples.

Weak efficiency generalizes classical efficiency through a differentiable map to an almost everywhere continuous map. It is therefore straightforward to construct estimators that are “efficient” under both strong and weak identification asymptotics. If $\psi$ is sufficient and $T_n$ asymptotes to a continuous map under weak regularity and to an efficient differentiable map under regularity, the estimator $\hat{\beta}=T_n(\hat{\psi})$ is weakly efficient under weak regularity and efficient under regularity. For example, the Rao\hypBlackwellized 2SLS exhibits this property. This is desirable since, often in practice, we do not know which asymptotics is a better approximation to the finite\hypsample situation in hand. Using such an estimator ensures maximal precision regardless of the “correct” asymptotics.

Finally, note that the point of weak efficiency is to exclude nuisance variation from an estimator, and the concept itself does not pin down a unique efficient distribution. For example, constant estimators or any linear transformations of weakly efficient estimators are weakly efficient. In some cases it is possible to impose additional restrictions to make a weakly efficient estimator unique. In linear IV with $d=k=1$ where we know the sign of $\pi$, the Rao\hypBlackwellized unbiased estimator is unique. Since many nonlinear functions of Gaussian means admit unique unbiased estimators, it may be possible to uniquely pin down an unbiased weakly efficient estimator in many models. However, unbiased estimators do not always exist (e.g., in linear IV when the sign of $\pi$ is not known).

Simulation of Weak Efficiency in Linear IV Models

To illustrate weak efficiency, we conduct simulation of linear IV models ((ref)) with overidentified conditional moment restrictions and heteroskedasticity. We consider discrete instruments so that we can estimate the heteroskedastic structure without imposing further assumptions. This enables us to compute the optimal IV and the feasible reduced\hypform GLS estimators. We focus on two estimators, 2SLS and optimal IV, under weak and strong identification asymptotics.

We let $d=1$ and $k=3$ so that 2SLS has a second moment. The sample size is chosen to be $n=1{,}000$. The instrument $z_i$ is uniformly distributed in $\{-1,1\}^3$, taking eight distinct combinations. The errors $(\varepsilon_i,v_i)$ are drawn from a normal distribution with mean $0$ and variance depending on $z_i$ as (ref); this dependence is determined randomly at the beginning of the simulation. The true parameters are given by $\beta=1$ and $\pi=(1,1,1)'/\sqrt{n}$ under weak identification (weakly regular $\beta$) and $\beta=1$ and $\pi=(1,1,1)'$ under strong identification (regular $\beta$). Simulation runs for 5,000 iterations. The heteroskedasticity\hypadjusted concentration parameter $\mathbb{E}[\mathbb{E}[vv'\mid z]^{-1/2}\pi'zz'\pi\mathbb{E}[vv'\mid z]^{-1/2}]$ is $0.0075$ for weak identification and $7.5484$ for strong identification.

To compute Rao\hypBlackwellization, we must derive the feasible GLS estimator for the reduced\hypform coefficients. A nontrivial aspect of this is that it consists of multiple equations. We handle this by combining them into one big equation: \[

=

+

=\vcentcolon

\psi+\tilde{U}. \] Consequently, the variance\hypcovariance matrix of $\tilde{U}$ has some nonzero off\hypdiagonal elements. We estimate it with initial OLS coefficients to compute the feasible GLS estimator for $(\gamma,\pi)$. Since GLS is efficient, by orthogonality we have $\operatorname{Var}(\hat{\psi}_{\text{OLS},n}-\hat{\psi}_{\text{GLS},n}\mid Z)=\operatorname{Var}(\hat{\psi}_{\text{OLS},n}\mid Z)-\operatorname{Var}(\hat{\psi}_{\text{GLS},n}\mid Z)$. With this, we compute the conditional expectation of 2SLS conditional on GLS using $100$ draws from $

\sim N\bigl(

,\,\operatorname{Var}(\hat{\psi}_{OLS,n}-\hat{\psi}_{GLS,n}\mid Z)\bigr)$. In particular, the RB 2SLS estimator of $\beta$ is given by \footnote{Note that $e_\gamma$ and $e_\pi$ are already denormalized by $\sqrt{n}$.} \[ \hat{\mathbb{E}}_e[((\hat{\pi}_{\text{FGLS},n}+e_\pi)'(Z'Z)(\hat{\pi}_{\text{FGLS},n}+e_\pi))^{-1}(\hat{\pi}_{\text{FGLS},n}+e_\pi)'(Z'Z)(\hat{\gamma}_{\text{FGLS},n}+e_\gamma)], \] where $\hat{\mathbb{E}}_e$ denotes numerical expectation with respect to $(e_\gamma,e_\pi)$.

Rao\hypBlackwellization of the optimal IV estimator requires a more elaborate procedure, as the optimal IV estimator involves two levels of noises. The first noise comes from the fact that $\varepsilon$, needed to compute $A(z)$, cannot be consistently estimated; if it is estimated with the 2SLS residuals, then it contains noise due to inefficiency of OLS used in 2SLS. The second noise comes from the fact that the optimal IV estimator is a function of the WLS estimator of $\bigl[

\bigr]$ and $\bigl[

\bigr]$, where the weights are given by the estimated $A(z)$. We use 50 draws to Rao\hypBlackwellize the first noise and 100 draws for each of the first noise to Rao\hypBlackwellize the second.

(ref) is the histograms of 2SLS and RB 2SLS estimators under weak regularity of $\beta$. The vertical dotted line indicates the true value, $\beta=1$. It shows that the distribution of RB 2SLS is more concentrated than 2SLS. Since Rao\hypBlackwellization does not affect its mean, both estimators share the same bias. (ref) is the histograms of optimal IV and RB optimal IV estimators for the same run, in which we observe similar results. To connect these histograms to (ref), we consider two loss functions: the mean squared error (MSE) $\ell(x)=x^2$ and the mean absolute error (MAE) $\ell(x)=|x|$, as summarized in (ref). The MSE of 2SLS decreases from $0.84$ to $0.20$ after Rao\hypBlackwellization; the MSE of optimal IV from $1.16$ to $0.17$. The MAE of 2SLS and optimal IV shows similar drop. We see substantial decrease in the losses in both estimators. LAR ((ref)) guarantees that the losses of the RB versions do not exceed those of the original ones, at least asymptotically. In this sense, it is preferable to use a weakly efficient estimator whenever available.

(ref) is the histograms of the same estimators under regularity of $\beta$. From classical results, we know that optimal IV is efficient and 2SLS is not. We see that both RB optimal IV and RB 2SLS coincide with optimal IV under strong identification asymptotics. This suggests that LAR does not alter an already efficient estimator while it transforms an inefficient estimator into an efficient one. The condition for this to hold can be understood using the analogy introduced at the beginning of (ref). For an estimator of the form $\hat{\beta}=T_n(\hat{\psi})$, if $T_n$ asymptotes to an almost everywhere continuous map under weak regularity of $\beta$ and to the optimal $T$ under regularity of $\beta$, then $\hat{\beta}$ is weakly efficient under weak identification asymptotics and efficient under strong identification asymptotics. This applies to most of known regular estimators.

Computational time of Rao\hypBlackwellization does not necessarily parallel computational time of the original estimator. There are two components that contribute to computational burden, $T_n$ and $\hat{\psi}_n$, and Rao\hypBlackwellization only repeats $T_n$. In our simulation, therefore, Rao\hypBlackwellization is done very quick. In fact, the most time\hypconsuming part of our simulation is the computation of the original optimal IV estimator, for which derivation of $\hat{\psi}_n$ requires a loop of matrix operations over observations. In our laptop, one iteration of the simulation (computation of 2SLS, optimal IV, their Rao\hypBlackwellization, and some auxiliary computation) takes less than 0.3 seconds. From a standpoint of strong identification asymptotics, our RB estimators (or the 2SLS with GLS estimators in place of OLS) give a much faster way to compute efficient estimators than to compute optimal IV.

Note that the conditional moment restrictions, $\mathbb{E}[u_i\mid z_i]=0$ and $\mathbb{E}[v_i\mid z_i]=0$, play a crucial role in this exercise. OLS is inefficient because of them. Relating thereto, another important assumption is the availability of feasible GLS. A notable example in which the form of heteroskedasticity is known {\em a priori} is when $y_i$ is binary and one has a conditional moment restriction, $\mathbb{E}[y_i\mid x_i]=f(x_i)$; the form of heteroskedasticity is uniquely determined by $f$ as $\mathbb{E}[(y_i-f(x_i))^2\mid x_i]=f(x_i)-f(x_i)^2$. If $f$ can be estimated, for example when $f$ belongs to some parametric family $\{f_\theta\}$, one may use feasible GLS with no additional loss of generality. In other linear models with an unknown form of heteroskedasticity, feasible GLS with a nonparametric estimator is available under various assumptions r1987,n1994.

Conclusion

This paper studies weak identification in semiparametric models and investigates efficient estimation. Weak identification is captured by the notion of {\em weak regularity}, with which the parameter is approximated by a homogeneous\hypof\hypdegree\hypzero map of the score. This nonlinearity implies impossibility of consistent estimation and inference and equivariant estimation. For each weakly regular parameter, there exists an underlying parameter that is regular and fully characterizes the weakly regular parameter locally. An underlying parameter that is minimal and sufficient shares the same nuisance tangent space with the weakly regular parameter, representing the exact amount of information relevant to the weakly regular parameter.

Decomposing estimation of the weakly regular parameter into estimation of the minimal underlying parameter and its transformation, efficiency is discussed in terms of the noise involved in the estimator of the underlying regular parameter. When the estimator of the underlying parameter is inefficient, we can improve the estimator of the weakly regular parameter by taking expectation conditional on the efficient estimator of the underlying parameter. The estimator is called {\em weakly efficient} if no further improvement is possible. This exploits the property that an efficient estimator of a regular parameter is “asymptotically sufficient", hence the name {\em local asymptotic Rao\hypBlackwellization}. Simulation of the linear IV model demonstrates that the 2SLS and optimal IV estimators can be improved if the feasible GLS estimator of the reduced\hypform coefficients is available.

Appendix