Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
37,564 characters · 8 sections · 14 citation commands
Identification of Multivariate Measurement Error Models
\begin {abstract}
This paper develops new identification results for multidimensional continuous measurement-error models where all observed measurements are contaminated by potentially correlated errors and none provides an injective mapping of the latent distribution. Using third-order cross-moments, the paper constructs a three-way tensor whose unique decomposition, guaranteed by Kruskal’s theorem, identifies the factor loading matrices. Starting with a linear structure, the paper recovers the full distribution of latent factors by constructing suitable measurements and applying scalar or multivariate versions of Kotlarski’s identity. As a result, the joint distribution of the latent vector and measurement errors is fully identified without requiring injective measurements, showing that multivariate latent structure can be recovered in broader settings than previously believed. Under injectivity, the paper also provides user-friendly testable conditions for identification. Finally, this paper provides general identification results for nonlinear models using a newly-defined generalized Kruskal rank - signal rank - of intergral operators. These results have wide applicability in empirical work involving noisy or indirect measurements, including factor models, survey data with reporting errors, mismeasured regressors in econometrics, and multidimensional latent-trait models in psychology and marketing, potentially enabling more robust estimation and interpretation when clean measurements are unavailable.
Keywords: identification, measurement error, latent variable, factor model, multivariate, Kruskal rank, signal rank.
\end {abstract}
\setcounter{page}{0} \thispagestyle{empty}
\setstretch {1.15}
Multivariate continuous latent variables arise in numerous empirical settings in economics, psychology, marketing, epidemiology, and the social sciences. Examples include multidimensional skills, cognitive factors, latent preferences, health indices, productivity components, and risk attitudes. In practice, such latent constructs are rarely observed directly; instead, researchers rely on multiple imperfect measurements that contain potentially correlated forms of noise. The resulting measurement-error problem is especially severe when the latent variable is multidimensional: each measurement typically captures only a low-dimensional projection of the latent vector, and the noise may exhibit dependence or heterogeneity across measurement channels. As a result, classical approaches to continuous measurement error, which often rely on injectivity, offer limited guidance.
This paper develops an identification strategy tailored specifically to multivariate continuous latent variables measured with noise. The key innovation is to combine tools from multilinear algebra, specifically the uniqueness properties of so-called CP tensor decompositions, with the multivariate extension of Kotlarski's identity, a powerful deconvolution result based on characteristic functions. The starting point is the observation that third-order cross-moments of three separate measurements form a three-way moment tensor whose CP decomposition is governed by the latent factor loading matrices. By invoking Kruskal's theorem Kruskal1977, I show that these loadings are generically identifiable even when each measurement matrix is rank-deficient and---even more surprisingly---when the stack of all measurement matrices is non-injective. This greatly relaxes the standard requirements for identifying multivariate continuous latent variables, which typically demand injectivity of the measurement operator.
Once the factor loading matrices are recovered through tensor methods, the paper turns to the problem of identifying the entire joint distribution of the continuous latent vector. A crucial step of the identification is that the Kruskal rank condition leads to two conditionally independent noisy measurements of each coordinate of the latent vector, derived from suitable linear projections of the observed measurements. These constructed pairs satisfy the assumptions of Kotlarski's identity, enabling closed-form identification of each coordinate's marginal distribution. For the case of dependent continuous latent factors, a weaker version of the full rank condition allows us to produce two vector-valued measurements whose errors are independent of the latent vector, thereby permitting the use of the multivariate Kotlarski identity. This yields identification of the full joint distribution of the continuous latent vector without assuming independence across its components.
A major contribution of this paper lies in demonstrating that multivariate continuous latent variables can be identified nonparametrically from three non-injective measurements by combining tensor uniqueness with characteristic-function deconvolution. It can be considered as a small step towards the extension of the Kruskal's theorem to continuous settings. As allman2009identifiability showed, the Kruskal's theorem can identify distribution of discrete latent variables. Extending that to continuous variables would require the extension of the Kruskal's theorem to continuous setting, which, to the best of my knowledge, is still an open question. The theorem in hu_schennach can be considered as such an extension to the continuous case under injectivity because a full rank condition in the discrete case naturally extends to the injectivity condition in the continuous case. Furthermore, another important contribution of this paper is to provide a generalized Kruskal rank - signal rank - for integral operators, and show that identification can hold for general nonlinear models under a signal rank condition similar to the well-known Kruskal rank condition. The newly-defined rank is consistent with the existing Kruskal rank for matrices and the injectivity of integral operators, and can be used to describe how informative a measurement is in the continuous multivariate case. Based on the signal rank condition, I provide identification of nonlinear models under high-level assumptions. In summary, this paper provides a constructive identification solution to a linear multivariate measurement error model, and builds a foundation for a better solution to the identification of general nonlinear measurement error models.
This work is closely related to several strands of literature. Nonparametric identification of measurement-error models with continuous latent variables has been advanced by e.g., kotlarski1965pairs, and hu_schennach. Tensor decomposition methods have been widely used to handle discrete latent variables in psychometrics, e.g., CarrollChang1970,Harshman1970, signal processing, e.g., Sidiropoulos2000, and in econometrics and statistics, e.g., Hu2008discrete, allman2009identifiability s,BonhommeJochmansRobin2016. This paper contributes a novel bridge between these literatures by demonstrating how tensor uniqueness can be used to identify both the measurement system and the full distribution of a multivariate continuous latent variable without injectivity or with less injectivity restrictions.
The limitation of this paper lies in the high-level assumptions in the nonlinear cases. The linear specification of the multivariate measurement structure may be rich enough for most empirical applications, but not complicated enough for theoretical explorations. Relaxaion of the linearity and additivity requires new technical tools. For that purpose, this paper provides a promising starting point towards continuous or operator-valued generalizations of Kruskal's theorem. Such continuous tensor identifiability results would naturally complement the multivariate Kotlarski identity and could relax the need for discretized third-order moments. With the results and the new concept in this paper, we are a small step closer toward a unified theory of identification for high-dimensional continuous latent-variable models in the presence of complex, nonclassical measurement error.
This paper is organized as follows: Section 2 specifies the linear model; Section 3 shows the identification of the factor loadings using the Kruskal's theorem; Section 4 presents the identification of continuous distributions without injectivity, but also provides a user friendly identification results under injectivity; Section 5 uses a newly-defned signal rank to present the identification of nonlinear cases. Section 6 concludes. All the proofs are in the Appendix.
We are interested in a latent structure described by a vector of latent variables $$X^*=
\in \mathbb{R}^L$$ Instead of $X^*$, we observe in a sample of three multi-dimensional measurements, for $i=1,2,3$ $$X^i =
\in \mathbb{R}^{K_i}$$ The relationship between the latent vector $X^*$ and its measurements can be described by the integral operator $L_{X^i|X^*} : \mathcal{L}^2\left( \mathcal{R}^{L }\right) \rightarrow \mathcal{L}^2\left( \mathcal{R}^{K^i}\right)$
where $p_{X^*}$ stands for the probability density function of $X^*$. It is known that nonparametric identification of the distribution of the latent variable $X^*$ requires that the integral operator $L_{X^i|X^*}$ is injective. This paper presents the identification of the joint distribution of $X^*$ and $X=\left ((X^1)^T,(X^2)^T,(X^3)^T \right)^T$ without such injectivity or with fewer injectivity restrictions than in the existing literature.
We start with a linear specification of the relationship between the latent vector $X^*$ and the three measurements as follows:
where $M^i$ are factor loading matrices. In the case where the dimension of $X^i$ is smaller than that of $X^*$, integral operator $L_{X^i|X^*}$ can't be injective if $p(X^i|X^*)$ is continuous in all its arguments. In particular, when the factor loading matrix $M^i$ doesn't have a full column rank, the injectivity of $L_{X^i|X^*}$ fails. Furthermore, the injectivity may still fail even if we combine the three measurements. For example, we consider $L_{X|X^*} :\mathcal{L}^2\left( \mathcal{R}^{L }\right)\rightarrow \mathcal{L}^2\left( \mathcal{R}^{K^1+K^2+K^3}\right) $
A necessary condition for the injectivity of $L_{X|X^*}$ is that matrix $M$ has a full column rank, where $$ M=
.$$ However, it is possible that $M^1=M^2=M^3$ with $\mathrm{Rank} [M^i] <L$ so that $\mathrm{Rank} [M] <L$. That means the rank of the stacked matrix is the same as that of $M^i$ and the integral operator $L_{X|X^*}$ is still not injective. Such a special case is important for the situation where $X^i$ are repeated measurements, which may have the same factor loading matrix $M^i$. In such a case one can't generate an injective measurement by combining the measurements.
This paper provides a set of sufficient conditions under which the model is fully identified, even when there are no injective measurements, single or combined. We show identification in two steps. First, we provide sufficient conditions to identify the factor loadings. Then, we provide stronger assumptions to show identification of distribution of latent variables without injectivity. In addition, we also show what we can achieve with injectivity under some testable assumptions.
We first identify the factor loadings under assumptions as follows:
In order to identify the factor loadings, we only need the measurement error vectors to be conditional mean independent of each other and the common factors $X^*$. The condition that unconditional mean is equal to zero holds without loss of generality in this linear setting. Notice that the measurement error in different dimension in one multivariate measurement can be correlated with each other arbitrarily. Therefore, we can't split a multivariate measurement to generate more measurements satisfying the same conditions.
Assumption (ref).(ref) impose restrictions on the first three moments of the latent vector $X^*$. Assumption (ref).(ref) guarantees that for $i\neq j$ and $i\neq k$
which allows us to focus on nonzero third moments of $X^*$ for further identification of factor loadings.
A key observation is that Assumption (ref) implies
which presents a decomposition of a 3-way tensor on the left hand side. The Kruskal's Theorem Kruskal1977 implies that the elements on the right hand side, i.e., $M^1$, $M^2$, $M^3$, are unique up to permutation and scaling if the Kruskal ranks of $M^1$, $M^2$, $M^3$ satisfy
where $\kappa(M^i)$ defined as $$\kappa(M^i)=\max \big\{ k : \text{every set of } k \text{ columns of } M^i \text{ is linearly independent} \big\}.$$
The Kruskal rank of $M^i$ describes how much information the measurement $X^i$ has even when $M^i$ is not injective or doesn't have a full column rank. Kruskal’s Theorem implies that we can identify the factor loadings from the decomposition of the 3-way tensor. For example, identification is possible with $K_i=4$ and $L=5$ when $M^i$ has a full column rank $\kappa(M^i)=K^i=4$. That means three 4-dimensional measurements may recover a 5-dimensional latent vector. In summary, we have
Proof: See the Appendix.
In this section, we show identification of distributions of latent variables without injectivity under assumptions as follows:
This assumption suggests that the latent variable $X^*$ and the measurement errors $\varepsilon^1$, $\varepsilon^2$, and $\varepsilon^3$ are jointly independent and that the multi-dimensional latent vector contains jointly independent factors. Such a independence allows us to focus on these factors one by one. First, we find two scalar measurements of one factor. Then, we show that the Kruskal rank condition in Equation ((ref)) guarantees that the two measurements satisfy the conditions to use the Kotlarski's identity.
Given the definition of the Kruskal rank, this identification procedure applies to all the latent factors with any permutation of the columns or $X^*_l$ for $l=1,...,L$. The distributions of measurement errors can then be identified by deconvolution under that Assumption (ref).3, which implies $\phi_{\varepsilon^i} \neq 0$ due to $\phi_{X^i}=\phi_{\varepsilon^i}\phi_{M^i X^*} $. As shown in HuShiu2022, this condition is testable. Then, distributions of $X^*$, $\varepsilon^1$, $\varepsilon^2$ are nonparametrically identified with a closed-form solution as follows:
In summary, we have
Proof: See the Appendix.
The Kruskal rank condition may be abstract to practitioners. In the case where all the factor loading matrices $M^i$ have a full rank, we have
That means practitioners can easily detect whether the rank condition holds. In the meantime, we can also interpret the rank condition as how many latent factors the data can identify. We have
In this section, we present an illustrative example to provide more details on how the Kruskal rank condition leads to two classical measurements. Suppose each of the three measurements $X^i$ has $K_i=7$ dimensions with $L=8$ latent common factors $X^*$ with the Kruskal ranks equal to $\kappa_i=6$. Therefore, the Kruskal rank condition holds.
In this case, $X^1$ is a 7-dimentional vector
where $M^1_{7 \times 6}$ has a full column rank for any permutation of the columns in $M^1$. Therefore, we can find $Q^1$ so that
Given the same permutation of the columns in $M^1$, we find a full column rank matrix from the right side as follows:
and with a $Q^2$
Notice that $\kappa(M^2)=6$ implies that $$
\neq 0.$$ Otherwise, there exist 6 columns, i.e., the 1st one and the last 5, which are linearly dependent so that the Kruskal rank of $M^2$ can't be 6. Without loss of generality, we let $q^2_{1,1}\neq0$ We then consider the first element in each of the two transformed measurements.
The only common factor in these two scalar measurements is the first latent factor $X^*_1$. Given $\gamma$, we then identified the distribution of $X^*_1$ from the joint distribution of $(W^1_1,W^2_1/\gamma)$ by Kotlarski’s Identity.
In order to identify the distribution of other factors, we may simply swap the columns and then apply the same procedure. For example, for factor $X^*_3$, we just apply the same procedure to
In summary, we may identify 8 latent factors from three 7-dimensional measurements, while the Kruskal rank corresponding to each measurement is 6, i.e., without injectivity.
In many empirical applications, especially in the estimation of structural models, it is important to allow the unobserved variables in $X^*$ to be correlated. In fact, we may extend our results to this case with modified assumptions as follows:
Because the latent factors may be correlated, we are not able to identify the distribution of the factors one by one. Therefore, we impose full rank restrictions on the loading matrix as follows:
The Kruskal rank codition in Equation ((ref)) then becomes
These restrictions in Equation (ref) implies that we may generate two classical measurements of the whole latent vector $X^*$, to which the multivariate Kotlarski's identity applies. That means the distribution of the latent vector is identified with a closed form. Notice that these rank condition and Assumptions (ref).2 are testable from the data given the first step identification of the factor loadings.
Finally, We can then identify the distribution of all the latent random variables under additivity and independence. We have
where the denominsator is nonzero because $\phi_{X^1}=\phi_{\varepsilon^1}\phi_{M^1 X^*} \neq 0$ implies that $\phi_{X^*} \neq 0$ with a full rank $M^1$. In summary, we have
Proof: See the Appendix.
Here we assume that all the density functions are continuous. Therefore, it is not possible to map from a vector to a lower dimension vector without losing information. Technically, there is a one-to-one mapping from $\mathbb{R}^{K}$ to $\mathbb{R}$, but such a mapping can't be continuous for $K>1$, which makes it less useful for practitioners. Also for this reason, the theorem in hu_schennach applies to the current model under three injectivity restrictions, i.e., for $i=1,2,3$, $M^i$ has a full column rank equal to $L$, i.e., $\mathrm{Rank}(M^i)=L,$ in order to identify all the distributions. The advantage of their results is that they can handle general nonlinear measurement functions, such as $X^i=g(X^*,\varepsilon^i).$
Nevertheless, Our Theorem (ref) achieves identification of all the distributions without injectivity and Proposition (ref) identifies the model under weaker injectivity conditions. In particular, although it has stronger assumptions, Proposition (ref) is more friendly to practitioner because the full rank condition and the nonzero ch.f. condition are both testable and easy to interpret.
In this section, we show identification without imposing linearity by introducing a newly defined generalized Kruskal rank for integral operators. Our analysis focuses on the effectiveness of this new rank concept, while maintaining other high-level assumptions.
We start with an indicator/sparsity matrix similar to a Jacobian matrix. Let $\mathrm{I}(\cdot)$ be the indicator function, which equals 1 if the statement is true and 0 otherwise. for For vectors of random variables $W \in \mathbb{R}^K$ and $X^* \in \mathbb{R}^L$, we define
where $\mathrm{J}(p_{W|X^*})$ is a matrix of indicators of whether $X^*_j$ enters the distribution $p_{W_i|X^*}$ for each $W_i$ in vector $W$. Let $W_{1:k}$ be the sub-vector of the first $k$ entries in vector $W$, $I_{K\times K}$ be a $K\times K$ identity matrix, and $R_{K\times L}$ be a $K \times L$ generic matrix of unspecified remainders.
We introduce a generalized Kruskal rank for integral operators, called signal rank, as follows:
In the previous linear case, the signal rank is equal to the Kruskal rank of the factor loading matrix and a mapping $Q$ can be generated by the Gram–Schmidt process based on the loading matrix. Note that $\mathrm{\kappa^s}(L_{X|X^*}) \in \{0,1,2,...,L\}$. We show that a signal rank condition, similar to a Kruskal rank condition for matrices, is crucial for the identification of distributions of latent variables, while keeping other assumptions at the high level for simplicity.
We assume
This assumption leads to
Our goal is to identify every probability function within the integral on the right hand side. Given the original permutation of vector $X^*= (X^*_1,...,X^*_{L})^T$, we apply its corresponding mapping to obtain $W^1= Q_1(X^1)$. For $X^2$, we consider a permutation of $X^*$ as follows:
with $W^2= Q_2(X^2)$. We show that a sufficient condition for identification is that
where $\kappa^s_i$ is the signal rank of $L_{X^i|X^*}$. This condition guarantees that $X^*_1$ is the only comment factor in the first elements in $W^1$ and $W^2$. Therefore, we can show that idiosyncratic elements in $W^1_1$, and $W^2_1$ are integrated out as follows:
In order to identify the whole distribution with a relatively simple procedure, we assume
Kruskal’s theorem does not require Assumption (ref), that is, it does not rely on any of the matrices having full rank, yet it is supported by a more than ten-page proof involving highly sophisticated mathematics. The proof becomes far less complicated with one of the three matrices being full rank. That is why we adopt this assumption. It is possible that Assumption (ref) is not necessary for our results as well, as we show in the linear case before. Given that this section focuses on the effectiveness of the newly defined signal rank, we mimic this relatively simple case in the proof of the Kruskal's theorem.
Assumption (ref) implies that $L_{X^3|X^*}$ has a full signal rank with $W^3=Q_3(X^3)$ satisfying
and that for $i=1,2$
We can show that three joint distributions $p(W^i_1,W^j_1)$ in Equations ((ref)) and ((ref)) identifies $L_{W^3_1|X^*_1} D_{X^*_1} (L_{W^3_1|X^*_1})^T $, which contains only one unknown - $p(W^3_1,X^*_1)$. We put this as a high-level normalization assumption as follows:
Notice that the operator $L_{W^3_1|X^*_1} D_{X^*_1} (L_{W^3_1|X^*_1})^T $ can be considered as an operator based on the joint distribution of $(W^3_1,\tilde{W}^3_1)$ with
where $e^1$ and $\tilde{e}^1$ have the same distribution conditional on $X^*_1$. If $X^*_1$, $e^1$, and $\tilde{e}^1$ on the right hand side are jointly independent and have a nonvarnishing ch.f., the Kortlarski's identity implies that the distributions of $(X^*_1, e^1, \tilde{e}^1)$, equivalently $p(W^3_1,X^*_1)$, are all identified with a closed form. Note that these derivations hold for $L_{W^3_l|X^*_l}$ as well. That means that Assumption (ref) implies that $L_{W^3_l|X^*_l}$ for all $l$ are identified.
Finally, we show the whole distribution $p(X^1,X^2,X^3,X^*)$ is identified from
Because we have identified $L_{W^3_l \mid X^*_l}$ for all $l$ and they are invertible, we can identify \\ \( p(X^2, X^3, X^*) \). Given the mapping $Q^3$ is one-to-one and continuous, we also identify the distribution of $p(X^3 \mid X^*)$ from $p(W^3 \mid X^*)$. That means we have identified
We summarize the result as follows:
Proof: See the Appendix.
This paper establishes new identification results for multivariate measurement-error models in which all observed measurements contain correlated noise and none provides an injective mapping of the latent vector. By exploiting the structure of third-order cross-moments, I construct a three-way tensor whose decomposition is uniquely determined under Kruskal’s rank condition. This tensor decomposition identifies the factor loading matrices up to permutation and scaling. Building on a linear structure, the paper then identifies the full distribution of each latent component by constructing appropriate measurements and applying either scalar or multivariate versions of Kotlarski’s identity. As a result, the joint distribution of the latent vector and all measurement errors is determined nonparametrically without requiring injective measurements. With the injectivity, we also achieve the identification under testable and less restrictive conditions.
The results in this paper widen the scope of what can be identified in measurement-error models, demonstrating that multivariate latent structures can be recovered in settings that would otherwise appear underidentified using existing techniques. The identification in the linear case is constructive in the sense that it leads to a straightforward estimation procedure. This makes the approach useful for empirical work involving noisy regressors, survey misreporting, and multidimensional factor models in economics and the social sciences.
Furthermore, this paper provides a generalized Kruskal rank - signal rank - for integral operators, and show that identification holds for general nonlinear models under a signal rank condition similar to the well-known Kruskal rank condition. The newly-defined signal rank is consistent with the existing Kruskal rank for matrices and the injectivity of integral operators, and can be used to describe how informative a measurement is in the multivariate case. Although our identification of nonlinear models is not as constructive as that of linear models, the results here can be seen as a small step toward developing a continuous analogue of Kruskal’s classical theorem. As shown in allman2009identifiability, Kruskal’s theorem delivers strong identification results for models with discrete latent variables. Extending these ideas to continuous latent variables would require a version of Kruskal’s theorem that works in a continuous setting, which, to the best of my knowledge, is still an open question. The Hu–Schennach theorem provides a partial analogue, but only under an injectivity condition. This paper provides constructive identification for linear measurement error models and defines signal rank of integral operators for the multivariate case, which moves the literature closer towards a solution of the important open question.