Nonparametric Analysis of Finite Mixtures

Nonparametric Analysis of Finite Mixtures

\address{Cowles Foundation for Research in Economics, Yale University, New Haven, CT 06520.} \email{[email removed]} \address{Cowles Foundation for Research in Economics, Yale University, New Haven, CT 06520.} \email{[email removed]}

Introduction

In empirical economics it is often crucially important to control for unobserved heterogeneity, and mixture models provide convenient ways to deal with it. This paper studies identification problems in the presence of unobserved heterogeneity under weak assumptions, by exploring identification in nonparametric finite mixture models. We then propose a fully nonparametric estimation method.

A generic mixture model takes the following form. Consider a probability distribution function $F_\alpha(\cdot)$, indexed by a random variable $\alpha$ that takes values on a sample space $\mathcal A$. $\alpha$ is sometimes called a mixing variable or a latent variable. It can be interpreted as a term representing unobserved heterogeneity. Let $G$ denote the probability distribution for $\alpha$. Define

The researcher observes $w$ distributed according to $F$. In other words, the mixture distribution $F(\cdot)$ is generated by mixing the component probability measures $F_\alpha(\cdot), \alpha \in \mathcal A$ according to the mixing distribution $G(\cdot)$. In an important special case where $G$ is discretely distributed and the space $\mathcal A$ is finite, ((ref)) becomes

For example, suppose there are $J$ types of economic agents that have type specific distributions $F_j(z), j = 1,...,J$. If type $j$ is drawn with probability $\lambda_j$, the resulting data obeys the finite mixture model ((ref)). The $F$ defined in ((ref)) is called a finite mixture distribution function. This is the main concern of the current paper. Since the paper presents various results with different models, a brief discussion of the overall nature of our contributions might be in order, as we now summarize in the following three points:

\ \

{\it (i) Relation to other identification results.} \quad As we mention below, currently available nonparametric identification strategies for finite mixtures often require either (A) multiple observations (a leading example being panel data) or (B) exclusion restriction and/or specific conditions on the shapes of the component distribution functions $F_\alpha, \alpha \in \mathcal A$. All the results in this paper concern identification in cross-section settings (i.e. the econometrician never observes an individual with a particular realization of the mixing/latent variable $\alpha$ more than once), therefore our identification strategy has little in common with the ones that belong to Category (A). Some papers in Category (B) assume exclusion restrictions, then invoke an identification-at-infinity type argument by focusing on observations at the tails of the component distributions. This paper does not rely on exclusion restrictions (and it even allows the mixture weights to depend on covariates in the model discussed in Section (ref)). Some other papers in Category (B) rely on symmetry of $F_\alpha$, which we do not assume either.

\ \

{\it (ii) Source of identification.} \quad The primal identification power in this paper comes from what may be called “componentwise shift-restriction" when a covariate is observed. That is, under an independence assumption, each component distribution generates a set of cross restrictions over a family indexed by the covariate values. Here the term “shift-restriction" is adopted from \citeasnoun{klein2002shift}, who consider semiparametric estimation of ordered response models (hence their paper is not about mixtures) though the identification strategy in the current paper not directly related to theirs: it is crucial to observe that in our case {\it for each component distribution} we obtain continuous limit analogues of shift-restrictions defined for a (possibly finite) set of covariate values. These componentwise shift-restrictions --- and equally importantly, the fact that after aggregating such latent distributions, the resulting mixture distribution function {\it lacks} the shift-restriction property under a “non-parallel condition" described later --- deliver fully nonparametric identification.

\ \

{\it (iii) On identification/estimation strategies.} \quad The “componentwise shift-restriction" described above can be usefully exploited after taking Fourier/Laplace transforms of the model. We then take limits in the Fourier/Laplace domains. As noted in {\it (i)} above, this is quite different from the approach based on exclusion restrictions together with nonparametric estimators with observations at the tails of the component distributions. Moreover, basing identification on the upper and lower tails generally limits the number of identifiable components, typically to the case with $J=2$, whereas our approach can be used to identify models with arbitrary $J$ (Section (ref)). The number of components $J$ itself will be identified in our approach as well. Alternatively, if we impose a large support restriction on covariates we can in principle establish identification in a straightforward manner. This would be a variant of the identification-at-infinity argument, and our approach does not share this feature either. As we shall see in Section (ref) it is possible to estimate the entire mixture model fully nonparametrically with standard polynomial convergence rates under mild assumptions. This desirable property is achieved without focusing on observations at the tails of the component distributions, nor a large support condition on the covariate.

\ \

We now mention some literature on the use of mixture models in general, followed by existing methods of identification for (finite) mixtures. As noted before, mixtures are commonly used in models with unobserved heterogeneity, especially in labor economics and industrial organization. See, for example, \citeasnoun{cameron1998life}, \citeasnoun{keane-wolpin}, \citeasnoun{berry1996airline}, \citeasnoun{arcidiacono2011conditional}, and \citeasnoun{aguirregabiria2013identification} for applications of finite mixture models in these fields. They are also used extensively in duration models with unobserved heterogeneity; see \citeasnoun{heckman1984method}, \citeasnoun{heckman1994econometric} and \citeasnoun{van2001duration}. A somewhat different use of mixtures can be found in models of regime changes, which can be viewed as finite mixture models. \citeasnoun{porter1983study}, for example, uses a switching simultaneous equations for an empirical IO model (see also \citeasnoun{ellison1994theories} and \citeasnoun{lee1984switching}). Some models with multiple equilibria can be regarded as mixtures as well (e.g. \citeasnoun{berry2006identification}, \citeasnoun{echenique2009testing}). Finally, contaminated models, as analyzed by \citeasnoun{horowitz1995identification} and \citeasnoun{manski2003partial} can be formulated as mixture models.

The most common estimation method for mixture models is parametric maximum likelihood (ML). In the notation introduced in ((ref)), ML requires parameterizing $F_\alpha(\cdot)$ and $G(\cdot)$ so that they are known up to a finite number of parameters. The EM algorithm often provides a convenient way to calculate the ML estimator for a mixture model.

This paper considers nonparametric identification problems in finite mixture models. The goal of the paper is to show that it is possible to treat the component distributions of a mixture model in a flexible manner. It should be noted that \citeasnoun{jewell1982mixtures} and \citeasnoun{heckman1984method} provide important identification results for mixture models in semiparametric settings. Again in the notation in ((ref)), these authors treat the component distributions $F_\alpha(\cdot)$ parametrically, (so that it is parameterized as $F_\alpha(\cdot,\theta)$, say, by a finite dimensional parameter $\theta$) while treating $G$ nonparametrically. They develop nonparametric ML estimators (NPMLE) for this type of models. Note that NPMLE, in actual applications, yields nonparametric estimates for $G$ that are typically discrete distributions with only a few support points. This fact may suggest that considering finite mixture distributions from the outset, as this paper does, is likely to be flexible enough for practical purposes.

Identification problems of finite mixtures have attracted much attention in the statistics literature. Teicher's pioneering work (Teicher \citeyear*{teicher1961identifiability}, \citeyear*{teicher1963identifiability}) initiated this research area. \citeasnoun{rao1992identifiability} provides a nice summary of this topic. See, also, \citeasnoun{lindsay1995mixture} for a comprehensive treatment of mixture models including their identification issues. Many results known in this area assume parametric component distributions. Indeed, as \citeasnoun{hall2003nonparametric} put it, “(v)ery little is known of the potential for consistent nonparametric inference in mixtures without training data.” Nevertheless, a number of papers have appeared on this subject, especially after the first version of the current paper was circulated. These include approaches based on multiple outcomes (e.g. \citeasnoun{bonhomme2016non}, \citeasnoun{bonhomme2016estimating}, \citeasnoun{d2015identification}, \citeasnoun{kasahara2009nonparametric}), or identification results based on exclusion restrictions, with/without tail restrictions on component distributions (e.g. \citeasnoun{adams2016finite}, \citeasnoun{compiani2016using}, \citeasnoun{henry2014partial}, \citeasnoun{henry2010identifying}, \citeasnoun{hohmann2013semiparametric}, \citeasnoun{jochmans2017inference}), or methods based on symmetry restrictions (e.g. \citeasnoun{butucea2014semiparametric}, \citeasnoun{hohmann2013two}).

The main result of the present paper is that nonparametric treatment of the component distributions of a finite mixture model is possible in a cross-sectional setting, if appropriate covariates are available.

Mixture Model with Covariates

Consider random vectors $z$ and $x$. Suppose the conditional distribution of $z$ given $x$ is given by a finite mixture model of the following form:

The main goal is to identify the mixing probability weights $\lambda_j, j = 1,...,J$ and the conditional component distributions $F_j(z|x)$ from the conditional mixture distribution $F(\cdot|x)$, using nonparametric restrictions. Sections (ref) - (ref) consider the case where $J = 2$. The above expression then becomes:

The case with $\lambda = 0$ is ruled out as we seek identification only up to labeling. Section (ref) considers an extension to the case with $J \geq 3$.

Regression

This section develops basic nonparametric identification results for ((ref)). Suppose $z$ and $x$ reside in $\mathbb{R}$ and $\mathbb{R}^k$, respectively. Define $$ m_j(x) = \int_\mathbb{R} z dF_j(z|x), j = 1,2, $$ i.e. the mean regression functions of the component distributions. Let $F_{\epsilon|x}^j, j = 1,2$ denote the distribution functions of the random variables \[ \epsilon_j = z_j - m_j(x), \quad j = 1,2. \] Note that by construction $\int \epsilon dF_{\epsilon|x}^j(\epsilon) = 0, j = 1,2$. With this notation $F_j(z|x) = F_{\epsilon|x}^j(z - m_i(x)), j = 1,2$, and the model ((ref)) can be written as

Our goal in this section is then to identify the elements of the right hand side of ((ref)) nonparametrically from the knowledge of $F(\cdot|x)$ evaluated at various $x$. Note that the model ((ref)) is further interpreted as a switching regression model:

Models as described above are conventionally estimated using parametric ML. That is, the researcher specifies (1) parametric functions for $m_1(x)$, $m_2(x)$, e.g. $m_1(x) = \beta_1^\top x$, $m_2(x) = \beta_2^\top x$, and (2) parametric distribution functions for $F_{\epsilon|x}^1$ and $F_{\epsilon|x}^2$, e.g. $\epsilon_1|x \sim N(0,\sigma_1^2)$, $\epsilon_2|x \sim N(0,\sigma_2^2)$. Examples of such methods can be found in \citeasnoun{quandt1972new} and \citeasnoun{kiefer1978discrete}; see also \citeasnoun{hamilton1989new} for application of ML in a time series context. The EM algorithm is often used in computing the ML estimator.

While the parametric approach is attractive and practical, the consistency of ML depends crucially on whether the parametric model is correctly specified or not. For example, even if $m_1$ and $m_2$ have the correct form, misspecifications in $F_{\epsilon|x}^1$ and $F_{\epsilon|x}^2$ would result in a failure of consistency. This is quite different from standard (possibly nonlinear) regression models, for which many distribution free estimators are available. This may discourage applied researchers from using mixture models. It also raises a more fundamental question: Is the model ((ref)) identified under weaker, non/semi-parametric assumptions? The results in this section provide a positive answer to this question.

Before discussing how nonparametric identification is possible, it may be helpful to see that a certain nonparametric restriction fails to generate identification in the model. Arguably the most common identification assumption for the standard regression model (without mixtures) is the conditional mean restriction. In our case, by the construction of $F_{\epsilon|x}^1$ and $F_{\epsilon|x}^2$ we have $\int_\mathbb{R} \epsilon dF_{\epsilon|x}^1(\epsilon) = 0$ and $\int_\mathbb{R} \epsilon dF_{\epsilon|x}^2(\epsilon) = 0$. The question is whether the knowledge of the conditional mixture distribution $F(z|x)$ at various $x$, combined with these “restrictions,” uniquely determine $F_{\epsilon|x}^1$, $F_{\epsilon|x}^2$, $m_1$, $m_2$, and $\lambda$. The answer is negative; at each $x$, we can split the mixture distribution $F(z|x)$ into increasing and right continuous $\mathbb{R}_+$-valued functions $a(z)$ and $b(z)$, say, so that $F(z|x) = a(z) + b(z)$. If we let $\lambda = \int da(z)$, $m_1(x)=\frac{1}{\lambda}\int z da(z),$ $m_2(x) = \frac{1}{1 - \lambda}\int z db(z)$, $F_{\epsilon|x}^1(\epsilon) = a(\epsilon + m_1(x))/\lambda$ and $F_{\epsilon|x}^2(\epsilon) = b(\epsilon + m_2(x))/(1-\lambda)$ they would satisfy all the available restrictions and information at all $x$. Even if $m_1$ and $m_2$ are completely parameterized, the model is not identified; “splitting” of $F(z|x)$ is not unique.

While it is straightforward to see the above identification failure, it highlights the fact the conditional mean zero condition allows “too many” ways to split the mixture distribution, thereby failing to deliver identification. Fortunately, however, there exists an alternative nonparametric restriction which identifies the model ((ref)). In what follows we focus on independence restrictions, i.e. independence of $(\epsilon_1,\epsilon_2)$ from $x$.

First identification result

Our first result is concerned with cases where at least one element of the vector of covariates $x = (x^1,...,x^k)^\top$ is continuous. Assume that the first $k^*$ elements $x^1,...,x^{k^*}$ are continuous covariates. We establish nonparametric identifiability at $x = x_0$ utilizing local variation in one of the $k^*$ continuous covariates. It is convenient to assume that the first element $x^1$ is such an element, which is assumed to be prior knowledge both for identification and estimation. The following notation is useful in considering local variations of $x^1$: for a point $x_0 = (x_0^1,...,x_0^k)^\top \in \mathbb{R}$, define $$ N^1(x_0,\delta) = \{(x^1,x_0^2,...,x_0^k)^\top \in \mathbb{R}^k| x^1 \in (x_0^1 - \delta,x_0^1 + \delta)\}. $$

With the notation above, ((ref)) is written as:

Note that the mixing distribution is allowed to be degenerate, i.e. $J = 1$. As a convention let $\lambda = 1$ if the mixing model is degenerate. That is, with degeneration ((ref)) becomes

The parameter space of $\lambda$ is therefore $(0,1]$.

We first discuss identification of the functions $m_1(\cdot), m_2(\cdot)$ in a neighborhood of the point $x_0 \in \mathbb{R}^k$. To this end a set of regularity conditions for nonparametric identification are stated in terms of moment generating functions. Let $$ M_i(t) = \int_\mathbb{R} e^{t\epsilon}dF_i(\epsilon), \quad i = 1,2, $$ for all $t$ such that this integral exists. $M_1$ and $M_2$ are the moment generating functions of the disturbance terms $\epsilon_1$ and $\epsilon_2$. Define $$ D(x) := m_2(x) - m_1(x) $$ on $\mathbb{R}^k$ and $$ h(c,t) : = e^{tD(x_0)(1 + c)} \frac {M_2(t)} {M_1(t)}, \quad c \in \mathbb{R}, t \in \mathbb{R}. $$ The following imposes a very weak regularity condition on the behavior of these moment generating functions.

Let $M(t|x)$ denote the moment generating function of $z$ conditional on $x$, that is, $$ M(t|x) := \int_\mathbb{R} e^{tz}dF(z|x), $$ whose domain, by ((ref)) and Assumption (ref)((ref)), is $(-\infty,\infty)$, and also let $$ R(t,x) := \frac{M(t|x)}{M(t|x_0)}. $$ Note that these functions are observable. The domain of these functions are $\mathbb{R}^k \times (-\infty,\infty)$ by Assumption (ref)((ref)).

Lemma (ref) suggests that the slopes of $m_1$ and $m_2$ are identified as far as the following condition holds. To state it, define $$ \mathbb{E}[z|x] = \int z d F(z|x) $$ and $$ \lambda_c := \frac { \mathbb{E}[z|x] - \mathbb{E}[z|x_0] - (1 + c) \lim_{t \rightarrow - \infty} \frac 1 t \log R(t,x) } { \lim_{t \rightarrow + \infty} \frac 1 t \log R(t,x) - (1 + c)\lim_{t \rightarrow - \infty} \frac 1 t \log R(t,x)}. $$ Note these are well defined under Assumption (ref) ((ref)). The constant $\delta$ in the following condition will be specified in the statements of Lemmas (ref) and (ref).

With this condition we have:

The next Lemma gives a full identification result. Let ${\mathcal F}(\mathbb{R}^p)$ denote the space of distribution functions on $\mathbb{R}^p$ for some $p \in \mathbb{N}$. Define $$ \bar{\mathcal F}(\mathbb{R}^p) = \{F: \int u F(du) = 0, F \in {\mathcal F}(\mathbb{R}^p)\}, $$ the set of distribution functions with mean zero. The parameter space of $(F_1(\cdot),F_2(\cdot))$ is given by $\bar {\mathcal F}(\mathbb{R})^2$. Also, for a set $\mathcal C \subset \mathbb{R}^k$ let $\mathcal V(\mathcal C)$ denote the space of all real valued functions on $\mathcal C$.

Second identification result

This section propose an alternative approach for identifying (ref). One advantage of this second identification result is that it is based on characteristic functions, so their existence is not an issue. Like the first identification result, the key sufficient condition, which differs from the MGF based condition in the previous section, is testable, Nonparametric identification holds under the following alternative set of sufficient conditions.

Assumption (ref) is similar to Assumption (ref), though here the continuity of $m_1$ and $m_2$ is not an issue. Next assumption imposes regularity conditions of the characteristic functions of $F_1$ and $F_2$, defined by $$ \phi_i(t) := \int_{\mathbb{R}} e^{it\epsilon} dF_i(\epsilon), \quad i = 1,2. $$

It is interesting to compare Assumption (ref) with Condition (ref). The former gives a sufficient condition in terms of the characteristic function, whereas the latter the moment generating function. It holds, for example, if $F_1$ and $F_2$ are the CDFs of $N(0,\sigma_1^2)$, $N(0,\sigma_2^2)$, $\sigma_1^2 \neq \sigma_2^2$. \citeasnoun{teicher1963identifiability} uses an assumption similar to this. Assumption (ref) rules out the case with $F_1 \equiv F_2$, which is allowed by Assumption (ref). Fortunately, just like Condition (ref), the new condition Assumption (ref) is verifiable through the observables, as is clear from the next lemma. This means which of the two identification strategies to be used can be determined by the observable features of the data. To state this more precisely, let $\phi(t|x)$ denote the characteristic functions of the conditional mixture distribution $F(z|x)$, that is, $$ \phi(t|x) := \int_\mathbb{R} e^{itz}dF(z|x), $$ and for $x_0 \in \mathbb{R}^k$ define $$ \rho(x,t) := \frac{\phi(t|x)}{\phi(t|x_0)}, \quad x \in \mathbb{R}^k. $$

Finally, assume

Note that the next lemma holds if the set of the regressors values includes at least three points. It therefore allows, for example, two regressors cases where one regressor is binary and the other is continuous.

Third identification result

We now propose an identification strategy that has an approach similar to the first identification result, though differs from it in some important ways. It uses one sided limit (e.g. $t$ tending to {\it positive} infinity) of MGFs and also characteristic functions. Unlike our first result, it for instance addresses the case where $F_1$ and $F_2$ are CDFs of $N(0,\sigma_1^2)$, $N(0,\sigma_2^2)$, $\sigma_1^2 \neq \sigma_2^2$. Moreover, the identification strategy for the distribution functions avoids Laplace inversion, a problematic step in practice. For these reasons it is the identification strategy in this section that will be used to construct our estimator in Section (ref).

Recall our definition of the function $h(\cdot,\cdot)$ (see Assumption (ref)) in the statements of the following assumption.

Note that this assumption does not demand the MGFs $M_1$ and $M_2$ to be defined on the whole real line, sometimes a restrictive assumption.

The constant $\delta$ in the following condition is specified in Assumption (ref).

The above condition is verifiable with information in the observables as $\rho$, $R$ and $\lambda_{\delta}$ are all observed.

Note that we once again needed the non-parallel regression function condition. Once the increments of the regression functions are identified, their levels as well as the mixture weight $\lambda$ are obtained using the same procedure as in the first identification result. Thus we have:

To identify the distribution functions $(F_1(.),F_2(.))$, we now propose another method which will be used to construct our estimator and avoids Laplace inversion. The main benefit of this is that it let us nonparametrically estimate the distribution functions without resorting to empirical MGF inversion, which is hard to handle in terms of obtaining polynomial rates of convergence. We will use previous identification of $\lambda$ and $m_1$ and $m_2$ evaluated at two points only, $x_1$ and $x_0$.

The idea is the following. Equation ((ref)) gives $F(z|x) = \lambda F_1(z - m_1(x)) + (1 -\lambda)F_2(z - m_2(x)), \forall (x,z) \in \mathbb{R}^{k+1},$ implying $ \forall (x,y) \in \mathbb{R}^{k+1},$

Applying Equation ((ref)) to $(x_0,y)$ and $(x_1,y)$ and taking the difference, we obtain

which means that $\forall y \in \mathbb{R},$ $ F_2(m_1(x_1)-m_2(x_1) + y) - F_2(m_1(x_0)-m_2(x_0) + y) $ is identified. Using recursively identification of this increment and the fact that the conditional cumulative distribution function $F_2$ converges to $1$ at infinity, we obtain identification of $F_2(z),$ $\forall z \in \mathbb{R}.$ Writing $g(x)=m_1(x)-m_2(x)$ and $\delta(x,x')=g(x)-g(x'),$ we assume that $\delta(x_1,x_0)>0.$ Note that $\delta(x_1,x_0) = \Delta - \nabla.$ Now, apply, for a given $z \in \mathbb{R},$ Equation ((ref)) to $y=z - g(x_0)$ to obtain $$F_2(z + \delta(x_1,x_0)) - F_2(z) = \frac{1}{1 -\lambda} (F(z + m_1(x_1) - g(x_0) |x_1) - F(z + m_2(x_0) |x_0)),$$ and, more generally, $\forall j \in \mathbb{N},$

Using $\lim_{j \to \infty} F_2(z + (j+1) \delta(x_1,x_0)) = 1,$ the identifying equation for $F_2(.)$ is

where the infinite sum is a convergent series of positive terms.

Finally the equation $F(z|x)= \lambda F_1(z - m_1(x)) + (1 - \lambda) F_2(z - m_2(x))$ identifies $F_1(.)$ as $$F_1(z) = \frac{1}{\lambda} \left[ F(z + m_1(x)) - (1 - \lambda) F_2(z + m_1(x) - m_2(x) \right]. $$

A model with “fixed effects”

The model we have focused on so far assumes that heterogeneity is exogenously determined. With $J=2$, a draw $(z,x)$ is generated from the first type of population or from the second with {\it fixed} probabilities $\lambda$ and $1 - \lambda$. This section relaxes this assumption. We assume that the binary probability distribution over the two types/population can depend on $x$ in a completely unrestricted, nonparametric manner. In terms of the switching regression formulation, this means:

where $x$ and $\epsilon_1$ ($\epsilon_2$) are, as before, assumed to be independent. Equivalently, we can write

The goals is now to identify the 5-tuple of functions $(\lambda(\cdot),m_1(\cdot),m_2(\cdot),F_1(\cdot),F_2(\cdot))$ from the joint distribution of $(z,x)$.

This model is of a particular interest in terms its implications. As in the rest of the paper, we often interpret the difference between $(m_1(\cdot),F_1(\cdot) )$ and $(m_1(\cdot),F_1(\cdot) )$ as a representation of unobserved heterogeneity. In a standard panel data regression model often such heterogeneity is represented by a scalar, and when it is assumed to be independent of the regressor it would be representing random effects, whereas if it is allowed to be correlated with the regressor in an arbitrary manner it becomes a fixed effects model. In certain applications fixed effects models are highly desirable. Panel data often offers approaches to deal with fixed effects, a leading case being a linear model with additive scalar-valued fixed effects. The model (ref) (or equivalently (ref)) is in this sense analogous to these fixed effects models. Unobserved heterogeneity in (ref) is function-valued (i.e. $m$ and $F$), as opposed to, say, an additive scalar. Its distribution, represented by $\lambda(x)$, is dependent on $x$ in a fully unrestricted way, accommodating arbitrary correlation between the unobserved heterogeneity and the regressor, so it resembles a panel data fixed effects model in this aspect. In this section we show that (ref) is nonparametrically identified, without requiring panel data, when the finite mixture modeling of unobserved heterogeneity is appropriate. Moreover, unlike in the standard panel data fixed effects model, the distribution of unobserved heterogeneity conditional on $x$ is identified fully nonparametrically. This means we identify the entire model, enabling the researcher to calculate desired counterfactuals.

We replace Assumption (ref) with

We maintain Assumption (ref), which, as noted before, is a weak regularity condition. Define $$ K_{+\infty,t}(x) := R(t,x)\exp \left( - t\lim_{s \rightarrow +\infty} \frac 1 s \log(R(s,x)) \right), $$ $$ K_{-\infty, t}(x) := R(t,x)\exp \left( - t\lim_{s \rightarrow -\infty} \frac 1 s \log(R(s,x)) \right), $$ $$ K_{+\infty}(x) := \lim_{t \rightarrow +\infty} K_{+\infty,t}(x) $$ and $$ K_{-\infty}(x) := \lim_{t \rightarrow -\infty} K_{ -\infty,t }(x). $$

Note that the limits in these definitions are well-defined over a neighborhood of $x_0$.

We replace Condition (ref) with:

The next result shows that the model that allows $\lambda$ to be arbitrarily dependent on $x$ is nonparametrically identified. Note that the mixture can be degenerate (i.e. $\lambda(x) = 1$) for some values of $x$, and this can be also inferred from the observables. As in the previous identification results presented in Lemmas (ref), (ref) and (ref), its main sufficient condition (i.e. Condition (ref)) is verifiable in terms of observables.

Instrumental Variables

The identification results developed in the preceding sections can be used to identify nonparametric finite mixture regression with endogenous regressors. Suppose we observe a triple of random variables $(y,w,x)$ taking its value in $\mathcal Y \times \mathcal W \times \mathcal X$ where $\mathcal Y \subset \mathbb{R}$, $\mathcal W \in \mathbb{R}^p$ and $\mathcal X \in \mathbb{R}^k$. Also let $$ z := \binom y w. $$ In a manner similar to Section (ref), consider a switching regression model:

Unlike in the previous sections, however, we no longer assume that $\eta$'s and $w$ are uncorrelated or independent. Instead, we assume

that is $$ \mathbb{E} [\eta_1|x] = \mathbb{E} [\eta_2|x] = 0. $$ Here and thereafter the notation $F_i(\star_1,\star_2,...)$ and $F_i(\star_1,\star_2,...|\star)$ denote the joint distribution of $\star_1,\star_2,...$ and the conditional distribution of $\star_1,\star_2,...$ given $\star$ when the joint distribution is given by $F_i, i = 1,2$. Consider linear operators

and assume that these operators are invertible.

The main goal is to identify $g_1$ and $g_2$. Here $x$ plays the role of instrumental variables. As before, define $m_1(x) = \int z dF_1(z|x)$ and $m_2(x) = \int z dF_2(z|x)$. Note that $m_1: \mathbb{R}^k \rightarrow \mathbb{R}^{p+1}$ and $m_2: \mathbb{R}^k \rightarrow \mathbb{R}^{p+1}$. For $j = 1,...,p+1$, let $m_{j,1}(\cdot)$ and $m_{j,2}(\cdot)$ denote the $j-$th elements of $m_1(\cdot)$ and $m_2(\cdot)$, respectively. Define the $p+1$-dimensional vectors of random variables $\epsilon_j = z - m_i(x), (z,x) \sim F_i(z,x), j = 1,2$. Consistent with the previous notation let $F_i(\epsilon_i|x), i = 1,2,$ denote the conditional distribution of $\epsilon_1$ and $\epsilon_2$ under $F_1$ and $F_2$.

By construction, $$ \int \epsilon dF_i(\epsilon|x) = 0, i = 1,2. $$ If we further assume that $F_i(\epsilon|x), j = 1,2$ do not depend on $x$, an appropriate extension of the theory developed in Section (ref) can be used to identify $m_{p+1,1}(x)$, $m_{p+1,2}(x)$, $F_1(z|x)$ and $F_2(z|x)$, which in turn, also identify the operators $T_1$ and $T_2$. By (ref), (ref) and (ref) we have $$ m_{p+1,1}(x) = T_1[g_1](x), \quad m_{p+1,2}(x) = T_2[g_2](x). $$ Then by their invertibility $g_1$ and $g_2$ are identified as . To formalize this idea, consider the following assumptions:

To state a multivariate extension of Assumption (ref), define the multivariate moment generating function $$ M_i({\bf t}) = \int e^{{\bf t}^\top\eta} dF_i(\eta), \quad i = 1,2, \quad {\bf t} \in \mathbb{R}^{p+1}. $$ Let ${\bf e}_j$ denote the unit vector whose $j-$th element is 1. Accommodating the identification strategy in Section (ref) require some modification as follows. Define $ {D(x)} := m_2(x) - m_1(x) $ as before, though now $D: \mathbb{R}^k \rightarrow \mathbb{R}^p $ is vector-valued. Also let $$ h_j(c,t) : = e^{ {t {\bf e}_j}' D(x_0)(1 - c)} \frac {M_2(t{\bf e}_j)} {M_1(t{\bf e}_j)}, \quad c \in \mathbb{R}_{++ }, t \in \mathbb{R} $$ and $$ R({\bf t},x) := \frac{M({\bf t} | x)}{M({\bf t} | x_0)}, \qquad {\bf t} \in \mathbb{R}^{p+1}. $$

By modifying the proofs of Lemmas (ref) and (ref) appropriately to deal with $\mathbb{R}^p$-valued random variables, we can show that $(\lambda,F_1^{\epsilon},F_2^\epsilon,m_1(x_0),m_2(x_0))$ is identified under Assumptions (ref) and (ref). Then, as noted in Remark (ref), $F(z|x), x \in {\mathcal X}, z \in \mathbb{R}^p$ uniquely determines $(\lambda,F_1^\epsilon,F_2^\epsilon,m_1(x),m_2(x))$ in $\mathbb{R} \times {\bar {\mathcal F}(\mathbb{R}^p)^2 \times \mathbb{R}^{2p}}$ for all $x \in \mathcal X$ up to labeling. Therefore each component distribution of $z$ is obtained by $$ F_1(z|x) = F_1^\epsilon(z - m_1(x)), \quad F_2(z|x) = F_2^\epsilon(z - m_2(x)). $$ We now have:

Mixtures with arbitrary $J$

Previous sections studied the identifiability for mixtures with $J = 2$. It is desirable, however, to be able to deal with mixtures with many components in some applications, especially when mixtures are used to represent unobserved heterogeneity. This section shows that nonparametric identification can be established for general $J$, possibly greater than 2, and moreover we show that the number $J$ itself is also identifiable.

The basic setup in this section is analogous to the one considered in Section (ref), though the conditional distribution of $z \in \mathbb{R}$ given $x \in \mathbb{R}^n$ consists of $J$ components, $J \in \mathbb{N}$, as in ((ref)). As before, define $$ m_j(x) = \int_\mathbb{R} z dF_j(z|x), j = 1,2,...,J. $$ Define also \[ \epsilon_j = z_j - m_j(x), \quad j = 1,2,...,J. \] Later we impose independence between $\epsilon_j, j = 1,...,J$ and $x$, which enables us to write $F(z|x)$ as

For later use, define $M_j(t) = \int e^{t\epsilon}F_j(d\epsilon), j = 1,...,J$. This section shows that the parameter $(\{\lambda_j\}_{j=1}^J,\{F_j(\cdot)\}_{j=1}^j,\{m_j(\cdot)\}_{j=1}^J)$ is identifiable under suitable conditions.

At an intuitive level, the argument developed in Section (ref) still offers a valid picture behind the identifiability result here. The independence of $\epsilon$ from $x$ leads to a shift restriction: the shapes of the distributions of $\{\epsilon_j\}_{j=1}^J$ have to remain invariant along the $J$ regression functions. This restriction, with other conditions, nails down the true parameters uniquely. Moving from $J = 2$ to $J \geq 3$, however, involves rather different theoretical arguments as developed subsequently. Recall that Section (ref) presented alternative conditions that guarantee the identifiability of two-component mixture models, as summarized by Lemma (ref), Lemma (ref) and Lemma (ref). This section proves the nonparametric identifiability of ((ref)) under conditions that are similar to the ones used in Lemma (ref), which seems least prohibiting of the three to generalize. Even so, this generalization calls for multistep identification argument with recursive procedures, as will be seen shortly.

To see how the treatment of general mixtures differs from the $J=2$ case, consider the case $J=3$. Instead of Equation ((ref)), we now have

Wlog, suppose $m_1(x_0) > m_2(x_0) > m_3(x_0)$ at a point $x_0$ in $\mathbb{R}^k$. Take a point $x'$ in the neighborhood of $x_0$ and consider the case $m_1(x') - m_1(x_0) \geq 0$ (if this term is negative, the roles of $m_1$ and $m_3$ get interchanged). The method used in the proof of Lemma (ref) to identify the $J=2$ model still works for the slopes of $m_1$ and $m_3$. Following the proof, take the ratio of the conditional moment generating functions at $x_0$ and a point in its neighborhood, $x'$, say, then take its logarithm followed by a normalization by $t$:

Suppose the ratios of $M_1(t)$, $M_2(t)$ and $M_3(t)$ do not explode exponentially, and $m_1$, $m_2$ and $m_3$ are continuous so that $m_2(x') - m_1(x_0)$ and $m_3(x') - m_1(x_0)$ are negative. Then as $t$ approaches to infinity, the above expression approaches to the slope $m_1(x') - m_1(x_0)$ if it is non-negative (though it yields the identical result if the slope is negative as well, as seen in the proof of Lemma (ref)). Similarly, by taking the limit $t \rightarrow - \infty$, the slope of $m_3$ is identified. This argument, however, leaves the slope of the middle term $m_2$ undetermined. And in the general case of $J \geq 3$, $J-2$ slopes remain to be determined. The approach in Lemma (ref) does fall short of achieving its goal when applied to models with $J \geq 3$.

It is, however, possible to identify the slope of $m_2$ by proceeding as follows. Suppose, evaluated at $x$, the regression functions satisfy the inequality $m_1(x) > m_2(x) > m_3(x)$. Pick a point $y$ in a neighborhood of $x$. Multiply ((ref)) by $e^{-t{[m_1(x) - m_1(y)]}}$ to obtain:

This purges $x$ out of the first term on the right hand side. Note $[m_1(x) - m_1(y)]$ can be identified by applying the argument in Lemma (ref) to the $J = 3$ model ((ref)), as demonstrated above. Therefore the left hand side of the above equation is known.

The above step enables us to eliminate all unknown parameters associated with the first mixture component. To see this, suppose $m_j, j = 1,2,3$ are differentiable in at least one of the $k$ elements of $x = (x^1,x^2,...,x^k)$. In what follows we assume that it is differentiable in the first element $x^1$ without loss of generality. As before, we assume that this is a prior knowledge. Let $D_x$ denote the partial differentiation operator with respect to the first component of x, i.e. $D_xf(x) = \frac \partial {\partial x^1}f(x)$. Differentiating both sides of the above equation by $x^1$ and rearranging,

Note that operating $D_x$ eliminates the unknown function $M_1(t)$ out of the right hand side of ((ref)). We now have $$ \frac {\partial}{\partial t} \log\left(D_x\left [e^{-t{[m_1(x) - m_1(y)]}}M(t|x)\right ]\right) = \frac {A_1}{A_2}, $$ say, where

and $$ A_2 = 1 + \frac {\lambda_3} {\lambda_2} \frac {[D_xm_3(x) - D_xm_1(x)]}{ [D_xm_2(x) - D_xm_1(x)]} e^{t[m_3(x) - m_2(x)]}\frac {M_3(t)}{M_2(t)}. $$ Note that the factor $D_xm_2(x) - D_xm_1(x)$ is non-zero if the two regression functions are not parallel at $x$, which makes the division by the factor valid. As far as $\frac {M_3}{M_2}$ and $\frac {D_xM_3}{M_2}$ do not explode exponentially, all the terms above except for the second and third terms of $A_1$ and the first term of $A_2$ converge to zero as $t \rightarrow \infty$. It follows that

The only unknown component in the above equation is $\frac {\frac {\partial}{\partial t} M_2(t)}{M_2(t)}$, but this term depends only on $t$, so it can be differenced out: repeat the above argument with replacing $x \in \mathbb{R}^k$ with a point $z \in \mathbb{R}^k$ so close to $x$ that $m_1(z) > m_2(z) > m_3(z)$. This yields $$ \lim_{t \rightarrow \infty} \left \{\frac {\partial}{\partial t} \log\left(D_z\left [e^{-t{[m_1(z) - m_1(y)]}}M(t|z)\right ]\right) \right \} = \{m_2(z) - [m_1(z) - m_1(y)]\} + \frac {\frac {\partial}{\partial t} M_2(t)}{M_2(t)}. $$ The slope of $m_2$ is $$ m_2(x) - m_2(z) = \lim_{t \rightarrow \infty} \frac {\partial}{\partial t} \log \left(\frac {D_x\left [e^{-t{[m_1(x) - m_1(y)]}}M(t|x)\right]}{D_z\left [e^{-t{[m_1(z) - m_1(y)]}}M(t|z)\right ]}\right) + (m_1(x) - m_1(z)). $$ The terms such as $m_1(x) - m_1(z)$ on the right hand side are identified by the method developed in Lemma (ref), as noted earlier. The equation above shows the identifiability of the slope of $m_2$.

We have already noted that the identifiability of the slope of $m_3$ basically follows from Lemma (ref). It is nevertheless instructive to present an alternative way to identify it by carrying on the foregoing analysis one step further. This will illustrate the basic idea behind our general identification theory for $J \in \mathbb{N}$.

Let us return to Equation ((ref)), changing the notation and writing $x_a$ for $x$, $x_b$ for $y$. As before, $\Delta_{ab}f$ stands for $f(x_a) - f(x_b)$. The first step is to purge $x_a$ from the first term on the right hand side, as we did in Equation ((ref)), as follows:

which yields

Notice that again this eliminates an unknown moment generating function, this time $M_2(t)$. Differentiating the above expression with respect to $t$ and following the line of argument presented above, the slope of $m_3$ is given by $$ \Delta_{ac}m_3 = \lim_{t \rightarrow \infty} \frac {\partial}{\partial t} \log \left[ \left( \frac { \frac {e^{-t[\Delta_{ab}m_2 - \Delta_{ab}m_1]}}{t[D_{x_a}m_2(x_a) - D_{x_a}m_1(x_a)]} D_{x_a}\left [e^{-t{\Delta_{ab}m_1}}M(t|x_a)\right ] } { \frac {e^{-t[\Delta_{cb}m_2 - \Delta_{cb}m_1]}}{t[D_{x_c}m_2(x_c) - D_{x_c}m_1(x_c)]} D_{x_c}\left [e^{-t{\Delta_{cb}m_1}}M(t|x_c)\right ] } \right) \right] + \Delta_{ac}m_2. $$

Let us now turn to the identifiability of the general model ((ref)) for a generic $J,$ at a point $x_a \in \mathbb{R}^k$. The general setting is the same as in Section (ref): the first $k*$ elements $x^1,...,x^{k*}$ of the vector of covariates $x$ are continuous covariates, and we will again use local variations in $x^1$.

Though Condition (ref) imposes $J$-th order differentiability in one argument for simplicity of presentation, this is not essential: it is sufficient to assume that there exists at least one multi-index $\alpha := (\alpha_1,...,\alpha_k) \in \mathbb{Z}^k, \alpha_1 + \cdots \alpha_k = J$ such that the derivative $D^\alpha m(x) = \left(\frac{\partial}{\partial x^1}\right)^{\alpha_1} \cdots \left(\frac{\partial}{\partial x^k}\right)^{\alpha_k} m(x) $ is well-defined for every $x$ in $B(x_a,\delta)$. See Remark (ref) for further discussions.

The independence assumption ((ref)) enables us to write the observable conditional distribution in the form ((ref)). The continuity assumption ((ref)) was also assumed in Lemma (ref). The differentiability condition ((ref)) may not be essential for the proof of the Lemma, though replacing derivatives in the proof with differences leads to extremely complex case-by-case analysis. Note that differentiability in only one element of $x$ suffices. Without loss of generality in what follows we assume that the $m_j, j=1,...,J$ are differentiable in the first element $x^1$. Recall that $D_1$ is the differentiation operator with respect to $x^1$. From now on, we will use the notation $$m_{k,j}(x)=m_k(x)-m_j(x).$$

Part ((ref)) of the assumption is not restrictive. As before, our goal is to establish identification up to labeling, so we can assume that

without loss of generality: this does not impact the validity of Assumption (ref). Part ((ref)) is an infinitesimal version of the non-parallel regression function conditions used in the previous sections.

Under these assumptions, we first prove identifiability of the slope $\Delta_{ab}m_1$, using the method developed in Section (ref) , for all $x_b$ in a chosen neighborhood of $x_a$. Note that we know $\lambda_1 \neq 0$. By the continuity and differentiability assumptions (Assumption (ref) ((ref)) and ((ref))), there exists $\delta' > 0$, $\delta' < \delta$, such that for all $x_b \in N^1(x_a, \delta')$ and for all $j=1,...,J$, $|m_j(x_b)-m_j(x_a)|<\frac{\epsilon}{2} $, and $D_1 m_j(x_b), j=1,...,J$ take J distinct values. Here we use the fact that twice differentiability of the regression functions implies that they are $\mathcal{C}^1$. Then, as in the proof of Lemma (ref), in the case $ m_1(x_b)-m_1(x_a)>0$, we write $$ \frac 1 t \log \left ( \frac {M(t|x_b)} {M(t|x_a)} \right ) = \frac 1 t \log \left ( \frac { e^{t[m_1(x_b) - m_1(x_a)]} + \sum_{j=2}^{J} \frac{\lambda_j}{\lambda_1}\frac{M_j(t)}{M_1(t)}e^{t[m_j(x_b) - m_1(x_a)]}} {1 + \sum_{j=2}^{J} \frac{\lambda_j}{\lambda_1}\frac{M_j(t)}{M_1(t)}e^{t[m_j(x_a) - m_1(x_a)]}} \right), $$ and in the case $ m_1(x_b)-m_1(x_a)<0$ , we write $$ \frac 1 t \log \left ( \frac {M(t|x_b)} {M(t|x_a)} \right ) = \frac 1 t \log \left ( \frac { 1 + \sum_{j=2}^{J} \frac{\lambda_j}{\lambda_1}\frac{M_j(t)}{M_1(t)}e^{t[m_j(x_b) - m_1(x_b)]}} {e^{t[m_1(x_a) - m_1(x_b)]} + \sum_{j=2}^{J} \frac{\lambda_j}{\lambda_1}\frac{M_j(t)}{M_1(t)}e^{t[m_j(x_a) - m_1(x_b)]}} \right). $$ Similarly, since $m_j(x_b) - m_1(x_a)$, $m_j(x_a) - m_1(x_a)$, $m_j(x_b) - m_1(x_b)$, and $m_j(x_a) - m_1(x_b)$ are less than $\epsilon - \Delta$, this gives in both cases, $$ \forall x_b \in U, \lim_{t \rightarrow \infty} \frac 1 t \log \left ( \frac {M(t|x_b)} {M(t|x_a)} \right ) = \Delta_{ba}m_1. $$ Hence the slope $\Delta_{ab}m_1$ is identifiable for all $x_b \in N^1(x_a, \delta')$. \\

Now we focus on the identifiability of the slopes $\Delta_{ab}m_j$ for all $j=2,...,J$ and $x_b$ in an appropriate neighborhood of $x_a$.

Pick a point $x_b \neq x_a$ in $\mathbb{R}^k$. For notational convenience, define the operator $A(x_a,x_b,t,k)$

where $f: \mathbb{R}^k \rightarrow \mathbb{R}$ is a function that is differentiable in its first argument, and $R_k(t,x)$ is a (rational) function in $t$. Its precise definition will be given shortly. The operator $A(x_a,x_b,t,k)$ generalizes the procedure performed on $D_{x_a}\left [e^{-t{\Delta_{ab}m_1}}M(t|x_a)\right ]$ in Equation ((ref)) to eliminate unknown parameters in ((ref)). Operate $A(x_a,x_b,t,k)$, $k = 2,3,...$ sequentially on $D_{x_a}\left [e^{-t{\Delta_{ab}m_1}}M(t|x_a)\right ]$ to define the expressions

By construction $Q_k(x_a,t)$ satisfies the following recursive formula:

The definition of the operator $A(x_a,x_b,t,k)$, as explained further later, is motivated by two facts: (i) the factor $e^{-t[\Delta_{ab}m_k - \Delta_{ab}m_{k-1}]}$ purges $x_a$ out of the exponent in the leading term of $Q_{k}(x_a,t)$ and (ii) division by the polynomial $R_k(t,x_a)$ then makes the leading term $\lambda_ke^{-tm_k(x_b)}M_k(t)$, which is completely free from $x_a$ and therefore eliminated by $D_{x_a}$. Once this is done, taking the log-derivative with respect to $t$ as in ((ref)) terms and taking the limit $t \rightarrow \infty$ yields $\Delta_{ab}m_{k}$ up to an unknown additive factor $\frac{\frac{\partial}{\partial t}M_k(t)}{M_k(t)}$, which can be differenced out.

Subsequent arguments establish the identifiability of $\Delta_{ab}m_k, k = 2,...,J$ for all $x_b$ in a neighborhood of $x_a$. We proceed in two steps. Step 1 shows that, with an appropriate choice of $R_k(t,x_a)$ in ((ref)), $Q_k(x_a,t), k = 2,3,...,J$ have following representations:

where $R_k^j(t,x_a), k = 2,3...,J, j = k,k+1,...,J$ are polynomials in $t$ with the property that $R_k^k(t,x_a) = R_k(t,x_a)$; a formal definition of these polynomials are provided later. The representations ((ref)) are useful, partly because the unknown functions $M_j(t), j = 1,..,k-1$ do not appear in $Q_{k}(x_a,t)$. Step 2 uses the representations ((ref)) to show that it is possible to identify the slope $\Delta_{ab}m_k, k = 2,...,J$ using the knowledge of $\Delta_{ab}m_1$, $Q_k(x_a,t)$ and $Q_k(x_b,t)$, $k = 2,...,J$ for all $x_b$ in a neighborhood of $x_a$.

The identifiability of the rest of the model (at $x_a$) is then established using the knowledge of $\Delta_{ab}m_k, k = 1,2,...,J$ and conditional moments of $z$ given $x_a$.

Let us start with Step 1, which derives the representation ((ref)) and will be summarized in Lemma (ref). Note that the definitions of the polynomials $R_k(t,x_a), k = 2,...,J$ and $R_k^j(t,x_a), k = 2,...,J, j = k,k+1,...,J$ are given in the course of our derivation.

{\bf Step 1:} \quad Start from $k = 2$. Define $$ R_2^j(t,x_a) = tD_{x_a}(m_j(x_a) - m_1(x_a)), j = 2,...,J, $$ then

yielding the desired representation for the case of $k = 2$. Let $R_2(t,x_a)$ (used in the definition of $A(x_a,x_b,t,2)$) be $R_2^2(t,x_a) = tD_{x_a}[m_2(x_a) - m_1(x_a)]$. With this choice

and the $j=2$ term in the summation drops out. Moreover, this result implies that $R_3(x_a,t)$ should be $$ R_3(x_a,t) = R_3^3(x_a,t) = D_{x_a}\frac{R_2^3(t,x_a)}{R_2(t,x_a)} + t \frac{R_2^3(t,x_a)}{R_2(t,x_a)}D_{x_a}[m_3(x_a) - m_2(x_a)]. $$ Note that the above step requires that $R_2(t,x_a)$ is non-zero: this issue will be discussed shortly.

The fact that the rest of $Q_{k}(x_a,t), k = 4,...,J$ have the representations as in ((ref)) can be shown by induction: suppose ((ref)) holds for $k = h$, that is

Define $$ R_{h+1}^j(t,x_a) = D_{x_a^1}\left(\frac{R_h^j(t,x_a)}{R_h^h(t,x_a)}\right) + t\frac{R_h^j(t,x_a)}{R_h^h(t,x_a)}D_{x_a^1}[m_j(x_a) - m_h(x_a)], \qquad j = h+1,...,J. $$ In what follows we sometimes write $$ R_k^j := R_k^j(t,x) $$ and $$ m_{k,l} := m_k(x) - m_l(x). $$ as short hand. Let $R_h(t,x_a) = R_h^h(t,x_a)$, then using this and the definition of the operator $A(x_a,x_b,t,h)$ in ((ref)), obtain

which is the desired result. The next lemma summarizes the foregoing argument. Notice that it relies on the assumption that $R_k(t,x_a) = R_k^k(t,x_a), k = 2,3,...J$ are non-zero, and later we show that the set

is non-empty.

{\bf{Step 2:}} This step shows that the knowledge of the function $Q_k(x,t)$ at $x = x_a$ and $x = x_b$ identifies $\Delta_{ab}m_k - \Delta_{ab}m_{k-1}$. The main result is:

Lemmas (ref) and (ref) will then be useful to prove the identifiability of $\Delta_{ab}m_k, k = 2,...,J$, for all $x_b$ in a neighborhood of $x_a$, since we already identified $\Delta_{ab}m_1$. The following propositions are useful in proving Lemma (ref). In what follows $\deg_t(f)$ and $\text{lc}_t(f)$ denote the degree and the leading coefficients of a polynomial $f(t)$ with respect to $t$.

The proof of the proposition is given in the Appendix.

Now we are ready to prove the main result in Step 2, that is, Lemma (ref).

To sum up, Lemma (ref) together with the proof of identifiability of $\Delta_{ab}m_1$ allow, by induction, the identifiability of the slopes $\Delta_{ab}m_k$ for all $x_b \in N^1(x_a, \delta')$ and for all $k=1,...,J$:

$$\Delta_{ab}m_1 = \lim_{t \rightarrow \infty} \frac 1 t \log \left ( \frac {M(t|x_a)} {M(t|x_b)} \right ),$$ $$\Delta_{ab}m_k = \sum_{j=2}^k \lim_{t \rightarrow \infty} \frac{\partial}{\partial t} \log(\frac{Q_k(x_a,t)}{Q_k(x_b,t)} ) + \Delta_{ab}m_1. $$

We now state the complete identification result. For the sake of clarity, we name the point of identification $x_0$ instead of $x_a$.

In the above assumption, the notation $\Delta_{0,i} m_j$ denotes $m_j(x_0) - m_j(x_i)$.

Having showed identification of our model assuming knowledge of $J$, we now consider the case where $J$ is unknown, and show it is identified, using the observable sequence of functions $(Q_k)_{k=1,...}$. As we see below, the number of mixture components $J$ is equal to the largest $j$ for which the function $Q_j$ not identically $0$ in $t$. Therefore one can sequentially compute the $\Delta_{ab}m_j$ using $Q_j$, for increasing $j$. Once there exists $j_0$ such that $Q_{j_0} =0$, then $J=j_0-1$.

}

Application to Identifiability of Auction Models with Unobserved Heterogeneity

It is of great interest to demonstrate that the preceding identification results potentially apply to nonparametric analysis of auction models with unobserved heterogeneity. As recognized in the recent literature, failing to properly taking account for unobserved heterogeneity in empirical auction models can lead to grossly misleading policy implications and counterfactual analyses. The reader is referred to \citeasnoun{haile2018unobserved} for various approaches to nonparametric identifiabilty in auction models when unobserved heterogeneity is present. Here we focus on application of the preceding mixture identification results to models with auction-specific unobserved heterogeneity. In particular, we focus on a symmetric affiliated auction model as considered in \citeasnoun{milgrom1982theory}. Suppose that valuations have the following multiplicative form, with $J$ unknown types of auctions

where $V^k$ is the valuation of bidder $k$, $1 \leq k \leq I$, who knows the number of bidders $I$, observed characteristics $x$, unobserved heterogeneity (i.e. unobserved type of auction) $j$, and a signal $S^k$. The function $\Gamma_j(x)$ depends on the two characteristics $x$ and $j$. The term $U_j^k$ can be interpreted as the “homogenized valuation" for bidder $k$, as used in \citeasnoun{haile2003nonparametric}. Let $B^k$ denote the bid of bidder $k$. The observables in this application is $(I,B^1,...,B^I,x)$. The rest remain unobserved.

We maintain that there are finite number of types in terms of auction heterogeneity. It is then possible to establish identification under quite weak assumptions. In the following result note that (i) valuations can be affiliated, and (ii) unobserved heterogeneity is treated flexibly, as not only it can affect valuations through the index function $\Gamma_j$ in an unrestricted way, the distribution of the homogenized valuation $U_j^k$ is allowed to depend on $j$ freely. Property (i) is important, as many preceding nonparametric identification results for auction with unobserved heterogeneity focus on the independent private value (IPV) model, as they tend to impose independence assumptions across valuations, with the exception of \citeasnoun{CHS}. For example, Property (i) implies that the result in this section applies to the common values model. Property (ii) about the flexible treatment of homogenized valuations is apparently new.

Assume

for every $j \in \{1,...,J\}$. Note that standard approaches to deal with unobserved heterogeneity do so through the index function $\Gamma_j$, and would not allow $(U^1,...,U^I)$ to depend on $j$ . Define $$ w(S,I,x,j) := \mathrm E\left[V^k | S^k = \max_{i \neq k, 1 \leq i \leq I}S^i = S, I, x, j\right] $$ which corresponds to the expected value of a bidder's valuation conditional on $I$, $x$, $j$, and the event that her equilibrium bid is pivotal. This is a quantity sometimes simply called “pivotal expected value". Let $w^k := w(S^k,I,x,j), 1 \leq k \leq I$ denote the pivotal expected value of the $k$-th bidder (whose signal is $S^k$) in an auction with characteristics $(x,j)$ and $I$ bidders. The goal here is to identify the joint distribution of $(w^1,...,w^I)$ in an auction with $(x,I,j)$, along with the distribution $(\lambda_1,...,\lambda_J)$ of the unobserved heterogeneity. Note that such knowledge is sufficient to address important questions often asked in practice: see, for example, footnote 9 of \citeasnoun{haile2018unobserved} for further discussions.

The above setting implies an expression of $w$ of the following form

where $\omega\left(S;I,j\right) = E[V^k|S^k = \max_{i \neq k}S^i = S, I,j]$. Like the homogenized valuation $\{\{U_j^k\}_{k=1}^I\}_{j=1}^J$, $\omega_j^k := \omega\left(S^k,I,j\right)$ is interpreted as a homogenized pivotal expected value of bidder $k$ in an auction of unobserved type $j$. It is well-known that the equilibrium bidding function preserves multiplicative separability in (ref), hence (ref), for each bidder $k$. Thus we obtain $$ B^k = \Gamma_j(x)R_j^k, $$ where $R_j^k$ is the homogenized valuation of bidder $k$ in type $j$ auction. Note that the unobserved auction type can affect equilibrium bids through two channels, that is, the index function $\Gamma_j$ and the homogenized bid $R_j^k$. Define $b^k = \log B^k$, $\gamma_j(x) := \log \Gamma_j(x)$ and $r_j^k := \log R_j^k$, then we have

Note that (ref) implies

conditional on $I$.

We now invoke Lemma (ref) to establish identification of this model. One of the main objects to be identified is the $I$-dimensional joint distribution of the pivotal expected values $w^1,...,w^I$ conditional on $(x,j,I)$, and our identification strategy works for each value of $I$. Thus in the rest of this section we treat $I$ as being fixed at a value, and suppress the index $I$ unless necessary. Let $c = (c_1,...,c_I)' \in {\mathbb R}^I$, and define $b(c) := \sum_{k=1}^{I}c_k b_{k}$, $C(c) := \sum_{k=1}^{n}c_k$ and $r_j(c) := \sum_{k=1}^{I}c_i r_j^k$. By (ref) and the finite mixture structure of the evaluation in (ref) we have $$ b(c) = C(c)\gamma_j(x) + r_j(c) \quad \text{ with probability } \lambda_j, 1 \leq j \leq J $$ where $r(c) {\bot\negthickspace\negthickspace\bot} x$ by (ref). Let $\left(b(c),\{C(c)\gamma_j(\cdot)\}_{j=1}^J,\{r_j(c)\}_{j=1}^J\right)$ play the role of $\left(z,\{m_j(\cdot)\}_{j=1}^J,\{\epsilon_j\}_{j=1}^J\right)$ in Lemma (ref), then $\left(C(c) \gamma_j(\cdot), \lambda_j\right)$ and the distribution of $r_j(c)$ are all identified for every $c \in {\mathbb R}^n$ and each $j \in \{1,...,J\}$. Moreover, we now know $\gamma_j(\cdot)$, $j \in \{1,...,J\}$ since $C(c)$ is known. Note that for each $j$, the marginal distribution of every linear combination $r_j(c)$ of the $I$-vector $(r_j^1,...,r_j^I)$ is identified as $c \in \mathbb{R}^I$ can be chosen arbitrarily. Then by Cram\'er-Wold the joint distribution of $(r_j^1,...,r_j^I)$ is obtained for each $j$. Apply this and the knowledge of $\gamma_j$ to equation (ref) to determine the joint distribution $(b^i,...,b^I)|x,j,I$. Using the first order condition for equilibrium bidding (see, e.g. \citeasnoun{haile2003nonparametric}, \citeasnoun{athey2007nonparametric} and Equation (2.4) in \citeasnoun{haile2018unobserved}) we can now back out the joint distribution of $(w^1,...,w^I)|x,j,I$ as desired. Note that the number of (unobserved) auction types $J$ is also identified by Proposition (ref).

Nonparametric estimation for $J=2$

This section develops a fully nonparametric estimation procedure based on our third identification result in Section (ref) where the number of mixture components is two. We first estimate the slopes of $m_1$ and $m_2$ nonparametrically. Define $\Delta = m_1(x_1) - m_1(x_0)$ and $\nabla = m_2(x_1) - m_2(x_0).$ Let us reintroduce notations. We write, for $j=1,2,$

where $F_i$ is the cumulative distribution function of $\epsilon_i,$ $h_n$ and $b_n$ are carefully chosen bandwidths for kernel density estimation. $\hat{M}^j(t)$ and $\hat{\phi}^j(s)$ are the Nadaraya-Watson regression estimators of respectively the conditional moment generating function and conditional characteristic function of $Z,$ when $X = x_j.$ $X$ being a vector, the kernel function $K$ can have a product form such as $K(X)=\Pi_{l=1}^k k(X^{(l)}).$

Our estimators are

where $(a_n)_n, (s_n)_n$ and $(t_n)_n$ are tuning parameters such that $a_n \to 0,$ $s_n \to \infty$ and $t_n \to \infty$. The notation Log$(\cdot)$ as before corresponds to the principal value of the logarithm of $\cdot$.

We enumerate here the assumptions on the kernel function needed to compute the rates of our estimators.

We need the following assumptions on the model parameters.

Proof 1 part 1

Proof 1 part 2

The only restriction imposed on the tuning parameter $a_n$ is that it converges to $0$.

For the sake of simplicity, we now write $\hat{\Delta} - \Delta = O_{\mathbb{P}} (\alpha_n)$ and $\hat{\nabla} - \nabla = O_{\mathbb{P}} (\beta_n )$. The rates $\alpha_n$ and $\beta_n$ depend on the distributions of the error terms, and we show here that the rates are polynomial in $n$ if these distributions are normal.

Indeed if $\epsilon_1|x \sim \mathcal{N}(0, \sigma_1^2)$ and $\epsilon_2|x \sim \mathcal{N}(0, \sigma_2^2),$ with $\delta = \sigma_1^2 - \sigma_2^2 >0 ,$ then Assumption (ref) is satisfied. For the ratio of the mgf, $\forall \epsilon >0 \mbox{, } e^{\epsilon t} \frac{M_2(t)}{M_1(t)} = e^{\epsilon t - \frac{\delta}{2} t^2} \underset{t \rightarrow \infty }{=} O(\mu(t)),$ with $\mu(t) = e^{-(\frac{\delta}{2}-\nu) t^2} \to 0,$ as $t \rightarrow \infty,$ for some $0 < \nu < \frac{\delta}{2}.$ And as for the ratio of the characteristic functions, $\frac{\phi_1(s)}{\phi_2(s)} = e^{- \frac{1}{2} \delta s^2} \underset{s \rightarrow \infty }{=} O(f(s)),$ with $f(s) = e^{- \frac{1}{2} \delta s^2} \xrightarrow [s \rightarrow \infty] {} 0.$ We take a fixed $a$ in the definition of $\hat{\nabla}$ here to simplify the computations, assuming $a$ is small enough. Applying the results from the estimation proofs, the convergence rates are

One can show that with the appropriate choice of the sequences $t_h, h_n, s_n,$ and $b_n$, the rates are polynomial in $n$. For example, it is the case if $k=1$, $h_n= n^{\frac{-1}{5} + \epsilon}$, $t_n=\frac{1}{\sigma_1} (\epsilon \log(n))^{\frac{1}{2}}$, $s_n=\frac{1}{\sigma_2} ( \beta \log(n))^{\frac{1}{2}}$ and $b_n = n^{\frac{-1}{5} + \beta}$ for $\epsilon, \beta < \frac{1}{5}.$

Proof 2

We now focus on the estimation of the remaining objects. We showed that if $\lambda \in (0,1),$ then $\lambda = \frac{\mathbb{E}(Z|X=x_1)- \mathbb{E}(Z|X=x_0) - \nabla}{\Delta - \nabla}.$ A natural estimator is therefore $$\hat{\lambda}=\frac{\hat{E}(Z|X=x_1)- \hat{E}(Z|X=x_0) - \hat{\nabla}}{\hat{\Delta} - \hat{\nabla}},$$ where $\hat{E}(Z|X=.)$ is the usual multivariate kernel regression estimator, $ \hat{E}(Z|X=x) = \frac{\sum_{p=1}^n Z_p K(\frac{X_p - x}{d_n})} {\sum_{p=1}^n K(\frac{X_p - x_j}{d_n})}$ where the kernel does not have to be the one used for the previous estimators but will be written $K$ for the sake of simplicity. Similarly the point estimation of the regression functions $m_1$ and $m_2$ is derived from Equation ((ref)). Writing $\hat{C}=\frac{1}{2} \left (\hat{E}(Z^2|X=x_0) - \hat{E}(Z^2|X=x_1) + \hat{\lambda} \hat{\Delta}^2 + (1 - \hat{\lambda}) \hat{\nabla}^2 \right),$ our estimators of $m_1(x_0)$ and $m_2(x_0)$ are

The convergence rate of these estimators can be computed easily. With the usual assumptions for kernel estimation and the appropriate choice of bandwidths $d_n=n^{\frac{-1}{k+4}},$ it is known that $\hat{E}(Z|X=x) - \mathbb{E}(Z|X=x) = O_{\mathbb{P}} (n^{-\frac{2}{k+4}})$ and $\hat{E}(Z^2|X=x) - \mathbb{E}(Z^2|X=x) = O_{\mathbb{P}} (n^{-\frac{2}{k+4}})$, see, e.g, \citeasnoun{hardle1994applied}. Writing $\epsilon_n = n^{-\frac{2}{k+4}} + \alpha_n + \beta_n$, one obtains $\hat{\lambda} = \lambda + O_{\mathbb{P}} (\epsilon_n).$ The estimators of $m_1(x_0)$ and $m_2(x_0)$, being linearizable functions of $\hat{\lambda} $, $\hat{\Delta}$ and $\hat{\nabla}$, their rates of convergence are similarly bounded. This is summarized in the next proposition.

Proof 3

To estimate the CDF of $\epsilon_1$ and $\epsilon_2$, we use Equation ((ref)) and propose the following estimator

In this formula $p(n) \in \mathbb{N}$ will be specified later, $\hat{g}(x)=\hat{m}_1(x) - \hat{m}_2(x),$ $\hat{\delta} = \hat{\Delta} - \hat{\nabla}$, and $\hat{F}(.|.)$ is the kernel regression estimator of the conditional cumulative distribution function, $$\hat{F}(z|x)= \frac{\sum_{j=1}^n \mathbbm{1} (Z_j \leq z) k(\frac{X_j - x}{c_n})} {\sum_{j=1}^n k(\frac{X_j - x}{c_n})}.$$ The kernel function and the bandwidth may differ from the choices for our previous kernel regression estimators, and will be here written as $k$ and $c_n$ respectively.

The assumptions from \citeasnoun{einmahl2005uniform} are conditions on the covering and measurability properties of the class of functions $\left\lbrace k(\frac{x-.}{c}) ;\ c>0, \, x \in \mathbb{R}^k \right\rbrace$.

In the case where $\epsilon_n$ is slower than $n^{-2 \: \frac{1 - 2a}{k+4}},$ for some $a,$ which happens when for instance the error terms are normally distributed, then $p_n$ is solution to $\epsilon_n p_n = t_0 e^{-t_0 p_n}.$

Conclusion

New nonparametric identification results for finite mixture models are developed. These open up the possibility of flexibly modeling economic behavior in the presence of unobserved heterogeneity.