Inference for Rank-Rank Regressions

Inference for Rank-Rank Regressions

\and Daniel Wilhelm. The author gratefully acknowledges financial support from the European Research Council (Starting Grant No. 852332)}} \thispagestyle{empty}

Introduction

Regressions involving ranks are widely used in empirical work in economics. A notable example is a rank-rank regression for measuring the persistence in socioeconomic status across generations. Dahl:2008tg and Chetty:2014tr have been influential in promoting this approach and a vast and fast-growing empirical literature is using such and related regressions for the study of intergenerational mobility in different socioeconomic outcomes, countries, regions, and time periods.\footnote{Some examples are Olivetti:2015tt, Black:2019oi, Abramitzky:2021ii, Fagereng:2021wb, Nybom:2024aa, and Ward:2023uu. Recent surveys by Deutscher:2023oo and Mogstad:2023uu provide numerous further examples, references, and some discussions of reasons for choosing the rank-rank specification.} The resulting measures are crucial inputs to broader political and public debates about inequality, about the importance of the family and the neighborhood into which children are born, and about how to create opportunities for children to rise out of poverty (Mogstad:2023uu). Beyond intergenerational mobility, regressions involving ranks are used in a range of other areas such as behavioral economics (e.g., Huffman:2022lk), development (e.g., Badge:2016yt), education (e.g., Murphy:2020ii), health (e.g., Gronqvist:2020oi), industrial organization (e.g., Cage:2019oi), labor (e.g., Faia:2023oi), migration (e.g., Ward:2022jh), and urban economics (e.g., Lee:2017uu).

In its simplest form, a rank-rank regression in the intergenerational mobility literature consists of performing two steps: first, rank a child's and their parent's socioeconomic status (say, income) in their respective distributions and, second, run a regression of the child's rank on a constant and the parent's rank. The slope coefficient in this regression is then interpreted as a relative measure of intergenerational mobility. A small value of the slope indicates low dependence of the child's position in their income distribution on the parent's position in their income distribution, and thus high mobility. Applied work often interprets the slope coefficient as the rank correlation of the two incomes (e.g., Chetty:2014tr or Deutscher:2023oo, which facilitates the interpretation of its magnitude as a parameter taking values between minus one and one.

In this paper, we develop an asymptotic theory for the OLS estimator of such rank-rank regressions. First, we demonstrate that commonly used inference methods for rank-rank regressions are invalid. Specifically, we show that neither the homoskedastic nor the Eicker-White robust variance estimators consistently estimate the asymptotic variance of the OLS estimator. In fact, their probability limits may be too large or too small, depending on the shape of the copula of the two variables to be ranked. In consequence, inference based on these commonly used variance estimators may be conservative or fail to satisfy coverage criteria. The intuitive reason for this is that these variance estimators ignore the estimation error in the ranks. In addition, we show that, in the presence of pointmasses in the underlying distribution, the regression estimand, the OLS estimator and its asymptotic distribution may be highly sensitive to the way in which ties in the ranks are handled.

Motivated by these findings, we derive the asymptotic distribution of the OLS estimator in a general rank-rank regression with covariates and without assumptions on whether the underlying distribution is continuous or not. In the special case in which the underlying distribution is continuous and there are no covariates, the OLS estimator is equal to Spearman's rank correlation, and our limiting distribution coincides with that derived by Hoeffding:1948uu. In general, however, the OLS estimator is not equal to Spearman's rank correlation and Hoeffding:1948uu's results do not apply. In particular, in the presence of pointmasses, our result shows how the regression estimand, the OLS estimator, and the asymptotic variance all depend on the definition of the ranks, i.e. how ties in the ranks are handled. Inference on the regression coefficients can be based either on the plugin estimator of the asymptotic variance\footnote{A software implementation is provided in the R package csranks, available on CRAN.} or on the nonparametric bootstrap.

Finally, in two empirical applications, we illustrate the importance of employing valid inference methods in rank-rank regressions and the sensitivity of results to how ties in the ranks are handled.

Our paper contributes to the general literature on nonparametric rank statistics. Since the literature is large, we provide here only some key references, referring an interested reader, for example, to a recent review C23. As mentioned above, Hoeffding:1948uu derived the asymptotic distribution for Spearman's rank correlation in the case of continuous distributions. N07 defined rank correlation measures for noncontinuous distributions, using a specific way of handling ties, with the extension satisfying several intuitive axioms that any measure of concordance between random variables should satisfy. MQ10 extended the results of N07 to cover the case of more than two variables. GNR13 studied the problem of estimating rank correlations in the multivariate case with discontinuities in the distribution. OL16 derived the asymptotic normality result and the asymptotic variance formula for Spearman's rank correlation for variables with finite support. We note here that these extensions are, although relevant, quite different from our work. In particular, we study the OLS estimator in general regression specifications involving ranks, which coincide with Spearman's rank correlation only in the special case in which the marginal distributions are continuous and there are no covariates. Otherwise, our estimands and estimators differ from those in the references listed here.

Our work is also related to Klein:2020oi, Mogstad:2024aa, and Mogstad:2024dd but the key difference is that they focused on inference on ranks of population parameters whereas we consider inference on features of the distribution of ranked observations.

Survey on the Use of Regressions Involving Ranks

Empirical researchers often transform variables into ranks before using them in a regression and then employ standard methods for inference that do not account for the estimation error in the ranks. To document this practice, we used Google Scholar to search for articles (and their appendices) published between January 2013 and February 2024 containing the words “rank” and “regression”. We restricted the search to the journals American Economic Review (excluding comments, P&P), Journal of Political Economy (excluding JPE Micro and JPE Macro), Quarterly Journal of Economics, and Review of Economic Studies. We dropped all theoretical articles without an empirical application and those that used the word “rank” in a different context, leading to a sample of 62 articles.

Many of the articles contain a large number of different regression specifications involving ranks, but we only record which specifications (“rank-rank”, “level-rank”, “rank-level”), which types of estimators (“OLS”, “TSLS”, “nonparametric”, “other”), and which types of standard errors (“homoskedastic/unknown”, “robust”, “clustered”, “bootstrap”, “other”, “none”) occur in a paper. “Rank-rank” refers to a regression in which both the outcome and at least one regressor have been transformed into ranks before running the regression. Similarly, “level-rank” (“rank-level”) refers to a regression in which at least one regressor (only the outcome) but not the outcome (none of the regressors) has been transformed into ranks. We obtain a total of 153 regressions, i.e. (paper $\times$ regression specification $\times$ estimator type $\times$ standard error type) combinations, in the sample.

Table (ref) categorizes the 153 regressions by specification (in the different columns), by type of estimator (Panel A), and by type of standard error (Panel B). First, consider the 40 regressions involving rank-rank specifications. Half of them (20) were estimated by OLS. For many regressions (11), it was not specified which method for the computation of standard errors was used. In most of these, we suspected the use of homoskedastic variance estimators and thus grouped them together with the one paper (2 regressions) that explicitly indicated “homoskedastic” standard errors. Many regression results were presented without standard errors (21). Not a single paper employed robust standard errors.

Level-rank specifications occurred in 83 of the regressions, most of them estimated by OLS (33) or nonparametric methods (34). For these regressions, clustering or not reporting standard errors was common. Rank-level specifications occurred somewhat less frequently in our sample (30).

To gain some insight into the topics of the articles we plotted the number of times each top-level JEL-code appears on the articles that contain regressions involving ranks, but do not report the results here to save space. Labor and Demographic Economics (JEL codes starting with “J”) are the most frequent, but health, education, public economics, and microeconomics are also listed as classifications by a substantial share of the articles. Overall, the results indicate that regressions involving ranks are used in a variety of subfields of economics.

Motivation

To motivate the subsequent theoretical developments, in this section we first show that commonly used inference methods for rank-rank regressions are not valid, even in the simplest case when the underlying variables are continuously distributed and there are no covariates. We then explain how the presence of pointmasses or covariates affects the estimand and the statistical properties of the OLS estimator.

Rank-Rank Regressions

For concreteness, consider a child's income $Y$ and their parent's income $X$. Suppose we are interested in measuring intergenerational mobility by running a regression of the child's income rank on a constant and the parent's income rank. In this section, we assume there are no further covariates. The slope coefficient from this regression reflects the persistence of the two generations' positions in their respective income distributions. A small value of the slope indicates low persistence and thus high mobility.

Let $F$ be the joint distribution of the two random variables $X$ and $Y$, and let $F_X$ and $F_Y$ be the corresponding marginals. Suppose we observe an i.i.d. sample $\{(X_i,Y_i)\}_{i=1}^n$ from $F$. The rank-rank regression involves ranks of $X_i$ and $Y_i$. To deal with potential ties, we consider a general definition of the rank: for a fixed, user-specified $\omega\in[0,1]$, let $$R_X(x) := \omega F_X(x) + (1-\omega)F_X^-(x),\quad x\in\mathbb R, $$ where $F_X^-(x):=P(X<x)$. We then define the rank of $X_i$ as $R_i^X:=\widehat{R}_X(X_i)$, where

is an estimator of $R_X(x)$ and $\widehat F_X^-(x) = n^{-1}\sum_{i=1}^n \mathds{1}\{X_i < x\}$ is an estimator of $F_X^-(x)$. This definition of the rank is such that a large value of $X_i$ is assigned a large rank. If $F_X$ is continuous, then the probability of a tie among $X_1,\ldots,X_n$ is zero and the rank $R_i^X$ is the same for all values of $\omega$. If $F_X$ is not continuous, then different choices of $\omega$ lead to definitions of the rank that handle ties differently. For instance, if $\omega=1$ then $R_i^X=\widehat{F}_X(X_i)$ and tied individuals are assigned the largest possible rank. Similarly, $\omega=0$ leads to a definition of the rank that assigns to tied individuals the smallest possible rank. If $\omega=1/2$, then $R_i^X$ is the mid-rank as defined in Hoeffding:1948uu, which assigns to tied individuals the average of the smallest and largest possible ranks. Table (ref) illustrates the different definitions of ranks in an example with ties. The quantities $F_Y$, $F_Y^-$, $R_Y$, and their estimators are defined analogously for $Y$. We assume that $\omega$ is the same value in $R_X$ and $R_Y$, so that ranks are defined consistently for both variables $X$ and $Y$. Throughout the paper, this value of $\omega$ is fixed and chosen by the researcher, so it does not appear as argument or index anywhere.

In a rank-rank regression, we first compute the parent's income rank $R_i^X$ and the child's income rank $R_i^Y$. Then, we run a regression of $R_i^Y$ on a constant and $R_i^X$. The OLS estimator of the slope parameter can then be written as the sample covariance of $R_i^Y$ and $R_i^X$, $S_{YX}:=\frac{1}{n}\sum_{i=1}^n(R_i^Y-\bar{R}^Y)(R_i^X-\bar{R}^X)$ where $\bar{R}^X$ and $\bar{R}^Y$ are sample averages of $R_i^X$ and $R_i^Y$, divided by the sample variance of $R_i^X$, $S_X^2 := \frac{1}{n} \sum_{i=1}^n (R_i^X-\bar{R}^X)^2$:

Researchers then perform inference by computing standard errors and confidence intervals using the usual variance formulas for the OLS estimator. In particular, these could be the homoskedastic or the Eicker–White robust variance formulas: $$ \widehat{\sigma}^2_{hom} := \frac{1}{nS_X^2}\sum_{i=1}^n \widehat{\varepsilon}_i^2 \quad\text{and}\quad \widehat{\sigma}^2_{EW} := \frac{1}{nS_X^4}\sum_{i=1}^n \widehat{\varepsilon}_i^2(R_i^X-\bar{R}^X)^2, $$ where $\widehat{\varepsilon}_i := R^Y_i - \bar{R}^Y -\widehat{\rho}(R^X_i-\bar{R}^X)$, $i=1,\dots,n$, is the residual from the regression. The estimator $\widehat\sigma_{hom}^2$ is suitable when regression errors are believed to be homoskedastic whereas $\widehat\sigma_{EW}^2$ is robust to heteroskedasticity. However, as we now demonstrate, these formulas yield inconsistent estimators of the asymptotic variance of $\widehat\rho$.

Inconsistency of OLS Variance Formulas

To simplify the discussion, suppose that $F_X$ and $F_Y$ are both continuous, so that with probability one there are no ties in the sample. In this case, the OLS estimator is equal to Spearman's rank correlation, which is defined as $\widehat{\rho}_S := \frac{S_{YX}}{S_XS_Y}$ with $S_Y^2$ denoting the sample variance of $R_i^Y$:

with probability one.\footnote{To see this, note that by continuity of the distributions, with probability one there are no ties in the data and $R_1^Y,\dots,R_n^Y$ is just a reordering of $R_1^X,\dots,R_n^X$. Therefore, $S_X=S_Y$ with probability one.} Thus, the OLS estimator of the rank-rank regression and Spearman's rank correlation are numerically identical in finite samples. As a result, they must have the same asymptotic properties, which we now discuss.

First, it is well-known that the probability limit of $\widehat{\rho}_S$ is

with $C$ denoting the copula of the pair $(X,Y)$, i.e. $C(x,y) := P(F_X(X)\leq x,F_Y(Y)\leq y)$ for all $x,y\in[0,1]$. Second, Hoeffding:1948uu showed that $$\sqrt{n}(\widehat{\rho}_S-\rho) \to_D N(0,\sigma^2), $$ where

and

On the other hand, the probability limits of the variance estimators $\widehat{\sigma}^2_{hom}$ and $\widehat{\sigma}^2_{EW}$ are given by the following lemma.

When $X$ and $Y$ are independent, then $\rho=0$ and both probability limits are equal to the correct (asymptotic) variance, $\sigma^2_{hom}=\sigma_{EW}^2 = \sigma^2=1$. In general, however, the three variances are different. In fact, $\sigma_{hom}^2$ and $\sigma_{EW}^2$ can each be larger or smaller than the correct variance $\sigma^2$. To see this note first that all three variances depend on the joint distribution $F$ only through its copula $C$. Therefore, whether and by how much the variances differ from each other depends only on the shape of the copula of child and parent incomes. As an illustration, we thus compare the three variances in three simple parametric families of copulas: the Gaussian, the Student-t with one degree of freedom, and a quadratic copula.\footnote{The Gaussian and Student-t families are each indexed by a correlation parameter $\theta\in[-1,1]$. The quadratic copula with parameter $\theta\in[0,1]$ is defined as the copula of $(Y,X)$ where $X\sim U[-1/2,1/2]$, $Y = 1/2 + \theta X + (1-\theta)X^2 + \varepsilon$, and $\varepsilon \sim N(0,10^{-6})$.}

Figure (ref) plots the three variances within each copula family as we vary the copula parameter in $[0,1]$.\footnote{For each parametric family, the copula parameter refers to the parameter $\theta$ defined in Footnote (ref).} The dashed line shows the value of the copula parameter value corresponding to the rank correlation between child and parent incomes in Germany ($0.384$), computed from the SOEP data used in Section (ref). For Germany, the quadratic copula produces hom and EW variances that are substantially smaller than the correct variance. The Student-t copula also yields a hom variance well below the correct variance, but the corresponding EW variance is essentially equal to the correct variance. The Gaussian copula generates hom and EW variances that are close to the correct variance, though both are slightly larger than the correct variance uniformly over the parameter value.

The variances $\sigma^2$, $\sigma_{hom}^2$, and $\sigma_{EW}^2$ are all bounded, and so mutual differences between them are bounded as well. This is not true for ratios, however. The following lemma shows that if we do not restrict the distributions $F$, the ratios $\sigma_{hom}^2/\sigma^2$ and $\sigma_{EW}^2/\sigma^2$ can both be arbitrarily large. To emphasize the dependence of the variances on the joint distribution $F$ of $(X,Y)$, we use the notation $\sigma_{hom}^2(F)$, $\sigma_{EW}^2(F)$, and $\sigma^2(F)$.

An implication of this lemma is that the two variances $\sigma_{hom}^2$ and $\sigma_{EW}^2$ can be larger than the correct variance by arbitrarily large factors. As the proof of the lemma reveals, the divergence occurs under sequences of copulas approaching perfect dependence so that both variances tend to zero, but the correct variance converges to zero at a faster rate than the variances $\sigma_{hom}^2$ and $\sigma_{EW}^2$.

Given the result in Lemma (ref), the next interesting question is how small the variances $\sigma_{hom}^2$ and $\sigma_{EW}^2$ can be relative to the true variance $\sigma^2$. The following lemma provides a partial answer to this question in the case of $\sigma_{hom}^2$.

By this lemma, the ratio $\sigma_{hom}^2/\sigma^2$ is bounded away from zero, so that $\sigma_{hom}^2$ can be smaller than $\sigma^2$ only by a factor that is bounded from below by the constant $c$.\footnote{We do not know whether a version of Lemma (ref) holds for the EW variance $\sigma_{EW}^2$ but we would be surprised if that were not the case.} However, it is important to emphasize that this constant may be rather small. In particular, this constant is strictly smaller than one, as there do exist distributions $F$ such that $\sigma_{hom}^2(F)/\sigma^2(F)<1$. For example, we have already seen in Figure (ref) that for simple well-known classes of copulas, like the Student-t copula, the hom variance may be substantially smaller than the correct variance. In the empirical applications of Section (ref), we also find that the hom variance may be substantially smaller than the correct variance.

We conclude this subsection with a heuristic explanation for why the hom and EW variance estimators fail to consistently estimate the asymptotic variance of the OLS estimator. The OLS estimator can be written as $$\widehat\rho = \frac{S_{YX}}{S_X^2} = \frac{1}{n S_X^2} \sum_{i=1}^n (R_i^Y-\bar{R}^Y)(R_i^X-\bar{R}^X). $$ If $R_i^X$ and $R_i^Y$ were replaced by the population ranks $R_X(X_i)$ and $R_Y(Y_i)$, then, after scaling and recentering, the above quantity would be asymptotically equivalent to a sample average of i.i.d. random variables with asymptotic variance equal to $\sigma_{EW}^2$. However, since the ranks $R_i^X$ and $R_i^Y$ are estimators of the populations ranks, the above argument ignores the fact that these variables are sample averages themselves. Therefore, the OLS estimator is, in fact, a triple sum over the sample divided by the sample variance $S_X^2$ and asymptotically equivalent to a U-statistic of order three. Employing U-statistics theory from, e.g. Serfling:2002re, one can show that the OLS estimator is asymptotically normal, but the variance is $\sigma^2$, not $\sigma_{EW}^2$. Intuitively, the estimation error in the ranks, i.e. the estimation error in the empirical cdfs, cannot be ignored even in the limit as the sample size tends to infinity because it is of the same order of magnitude as the estimation error in the infeasible OLS estimator in which the estimated ranks are replaced by the population ranks.

In conclusion, this subsection has shown that commonly used estimators for the asymptotic variance of the OLS estimator in a rank-rank regression are not consistent. In particular, the resulting homoskedastic and Eicker-White standard errors may be too small or too large depending on the shape of the underlying copula. In consequence, confidence intervals may be too short or too wide, possibly leading to under-coverage or conservative inference.

Noncontinuous Marginal Distributions

The well-established asymptotic theory for Spearman's rank correlation described in the previous subsection requires both marginal distributions, $F_X$ and $F_Y$, to be continuous. In empirical applications of rank-rank regressions, however, pointmasses are common. For instance, incomes may be top-coded and there may be pointmasses at zero or negative incomes (e.g. as in Chetty:2018iu).\footnote{In fact, one of the commonly cited (Deutscher:2023oo, Mogstad:2023uu) advantages of measuring mobility by the rank-rank slope is that it can be estimated even in the presence of zero incomes, while other popular measures like elasticities can only be estimated after removing observations with zero incomes.} Alternatively, ranks may be computed from discrete measures other than income, e.g., occupational status as in Ward:2023uu, human capital as in Croix:2022aa, or years of education as in Asher:2024iu. In such cases, pointmasses in the marginal distributions create ties in the ranks. The presence of ties changes inference in the rank-rank regression in at least three important ways, which we now discuss.

\paragraph*{Interpretation of the estimand.} The probability limit of the OLS estimator is equal to

i.e. it is equal to the rank correlation times the ratio of standard deviations of the ranks. For continuous distributions, the ratio of standard deviations is equal to one. However, in the presence of pointmasses, this ratio can take any value in $(0,\infty)$ and, thus, the slope coefficient of the rank-rank regression may lie outside of the $[-1,1]$ interval and cannot be interpreted as a correlation.

\paragraph*{Sensitivity of the estimand to the definition of ranks.} In the presence of pointmasses, the population ranks $R_X(X)$ and $R_Y(Y)$ depend on $\omega$, i.e. on how ranks are assigned to tied observations. Therefore, the OLS estimand (ref) also depends on $\omega$. In other words, different ways of handling ties imply different estimands.

\paragraph*{Sensitivity of the OLS estimator's statistical properties to the definition of ranks.} In the presence of pointmasses, not only the population ranks, but also the sample ranks $R_i^X$ and $R_i^Y$ depend on $\omega$. Therefore, the OLS estimator and its statistical properties also depend on $\omega$ and, thus, on how ties are handled. The previous paragraph already argues that the estimator's probability limit depends on $\omega$, but in Section (ref) we formally show that its asymptotic variance also depends on $\omega$. In consequence, standard errors for the OLS estimator vary depending on how ties in the ranks are handled. Moreover, in the empirical application in Section (ref), the OLS estimator and the corresponding standard errors exhibit high sensitivity to the way in which ties are handled.

Covariates

As seen in (ref), the rank-rank slope $\rho$ is not necessarily equal to the rank correlation. This case occurs when the standard deviations of $R_Y(Y)$ and $R_X(X)$ differ due to the presence of pointmasses.

Another reason for why $\rho$ may not be interpretable as a correlation is the presence of covariates, say $W$, in the rank-rank regression. With covariates the rank-rank slope satisfies $$\rho = Corr(\widetilde{R}_Y(Y),\widetilde{R}_X(X)) \sqrt{\frac{Var(\widetilde{R}_Y(Y))}{Var(\widetilde{R}_X(X))}},$$ where $\widetilde{R}_Y(Y)$ and $\widetilde{R}_X(X)$ denote the residuals from partialling out $W$ from $R_Y(Y)$ and $R_X(X)$. Therefore, the rank-rank slope is equal to the correlation of $\widetilde{R}_Y(Y)$ and $\widetilde{R}_X(X)$ only if the standard deviations of the two residuals happen to be identical. In general, however, the ratio of standard deviations can take any value in $(0,\infty)$ and, thus, the rank-rank slope may lie outside of $[-1,1]$.

A General Asymptotic Theory

Asymptotic Normality Result

In this section, we develop a general, unifying asymptotic theory for coefficients in a rank-rank regression that allows for any definition of the rank, for continuous or noncontinuous distributions, and for the presence of covariates.

We consider the following regression model:

where $W$ is a $d$-dimensional vector of covariates, $\varepsilon$ is noise, and $\rho$, $\beta$ are regression coefficients to be estimated. In applications, $W$ typically includes a constant, but our theoretical results below do not require that.

Letting $\{(X_i,W_i,Y_i)\}_{i=1}^n$ be an i.i.d. sample from the distribution of the triplet $(X,W,Y)$, we study the properties of the following OLS estimator of the vector of parameters $(\rho,\beta')'$:

The population ranks $R_X(X)$, $R_Y(Y)$ and the estimated ranks $R_i^X$, $R_i^Y$ are defined as in Section (ref) with the same, but arbitrary value of $\omega\in[0,1]$.

To derive the asymptotic normality of $\widehat\rho$, we introduce the projection of $R_X(X)$ onto the covariates:

where $\nu$ is a random variable representing the projection residual, and $\gamma$ is a vector of parameters. Consider the following regularity conditions:

These are standard regularity conditions underlying typical regression analyses. Assumption (ref) requires that the rank $R_X(X)$ can not be represented as a linear combination of covariates. Importantly, our regularity conditions do not require $X$ and $Y$ to be continuously distributed.

Under these conditions, we have the following asymptotic normality result.

The proof of this result can be found in Appendix (ref). In fact, Theorem (ref) follows from a more general result, which we state in Appendix (ref) (Theorem (ref)). The more general result shows joint asymptotic normality for $\widehat\rho$ and $\widehat\beta$, $$\sqrt{n}

= \frac{1}{\sqrt{n}} \sum_{i=1}^n \psi_i + o_P(1) \to_D N(0,\Sigma),\qquad \Sigma := E[\psi_i\psi_i'], $$ and provides an explicit formula for $\psi_i$. The joint asymptotic distribution for $\widehat\rho$ and $\widehat\beta$ is useful for empirical work in the intergenerational mobility literature. Suppose there are no covariates so $W$ only contains a constant. Besides the rank-rank slope, another popular measure of intergenerational mobility (\cite{Deutscher:2023oo}) is the expected income rank of a child given that their parent's income rank is equal to some value $p$: $\theta_p := \beta + \rho\, p$. With the asymptotic joint distribution above, it is straightforward to compute standard errors and confidence intervals for $\theta_p$ based on the plugin estimator $\widehat{\theta}_p := \widehat{\beta} + \widehat{\rho}\, p$.

Theorem (ref) has the following interpretation. If we knew the population ranks $R_Y(Y_i)$ and $R_X(X_i)$, we would simply estimate $\rho$ and $\beta$ by an OLS regression of $R_Y(Y_i)$ on $R_X(X_i)$ and $W$. The asymptotic variance of such an estimator for $\rho$ would be given by $E[h_1(X,W,Y)^2]/\sigma_{\nu}^4$, which is the familiar OLS variance for heteroskedastic regression errors. Without knowing the population ranks, however, we have to plugin their estimators, and the extra terms in the asymptotic variance, represented by $h_2(X,Y)$ and $h_3(X)$, provide the adjustments necessary to take into account the noise coming from estimated ranks. Importantly, the estimation error in the ranks is of the same order of magnitude as that in the infeasible OLS estimator in which the estimated ranks are replaced by the population ranks. Therefore, the estimation error in the ranks does not become negligible in large samples, not even in the limit as the sample size tends to infinity.

Consistent Estimation of the Asymptotic Variance

In this subsection, we propose an estimator of the asymptotic variance $\sigma^2$ appearing in the asymptotic normality result in Theorem (ref) and show that it is consistent. In particular, we consider the following plug-in estimator: $$\widehat{\sigma}^2 := \frac{1}{n \widehat{\sigma}_{\nu}^4} \sum_{i=1}^n(H_{1i} + H_{2i} + H_{3i})^2, $$ where $\widehat{\sigma}_{\nu}^2 := n^{-1}\sum_{i=1}^n \widehat{\nu}_i^2$ is an empirical analog of $\sigma_{\nu}^2 = E[\nu^2]$, $\widehat{\nu}_i := R_i^X-W_i'\widehat{\gamma}$, and

for all $i=1,\dots,n$. The following lemma shows that this simple plug-in estimator is consistent without any additional assumptions.

Theorem (ref) and Lemma (ref) give the correct way to perform inference in rank-rank regressions. For example, a $(1-\alpha)\times 100\%$ asymptotic confidence interval for $\rho$ can be constructed using the standard formula $$ \left(\widehat\rho - \frac{z_{\alpha/2}\widehat\sigma}{\sqrt n}, \widehat\rho + \frac{z_{\alpha/2}\widehat\sigma}{\sqrt n}\right), $$ where $z_{\alpha/2}$ is the number such that $P(N(0,1) > z_{\alpha/2}) = \alpha/2$. Standard hypothesis testing can be performed analogously.

Other Regressions Involving Ranks

Motivated by the empirical practice documented in Section (ref) we now extend the asymptotic normality result in Theorem (ref) to other regressions involving ranks. We consider (i) a rank-rank regression with subpopulations, where ranks are computed in the national distributions rather than in the subpopulation-specific distributions, (ii) a regression of the level of $Y$ on the rank of $X$, and (iii) a regression of the rank of $Y$ on the level of $X$. For brevity of the paper, we keep the discussions of each extension relatively short.

Rank-Rank Regressions With Subpopulations

In this subsection, we consider a population (e.g., the U.S.) that is divided into $n_G$ subpopulations (e.g., commuting zones). We are interested in running rank-rank regressions separately within each subpopulation. The ranks, however, are computed in the distribution of the entire population (e.g., the U.S.). Chetty:2014tr has been influential in promoting this kind of regression for the analysis of mobility across regions, where the scale of the mobility measure is fixed by the national distribution. The survey by Deutscher:2023oo provides more examples of empirical work running such regressions, for instance Corak:2020io and Acciari:2022uu. In Section (ref), we apply the methods from this section to study income mobility across states in Germany.

Consider the model

where $G$ is an observed random variable taking values in $\{1,\ldots,n_G\}$ to indicate the subpopulation to which an individual belongs. The quadruple $(G,X,W,Y)$ has distribution $F$, and we continue to denote marginal distributions of $X$ and $Y$ by $F_X$ and $F_Y$. The quantities $F_X^-$, $F_Y^-$, $R_X(x)$, and $R_Y(y)$ are also as previously defined, so that $R_X(X)$, for instance, is the rank of $X$ in the entire population, not the rank within a subpopulation. So, in model (ref), the coefficients $\rho_g$ and $\beta_g$ are subpopulation-specific, but the ranks $R_Y(Y)$ and $R_X(X)$ are not. In consequence, $\rho_g$ cannot be interpreted as the rank correlation within the subpopulation $g$. Instead, in the intergenerational mobility literature, the rank-rank slope is interpreted as a relative measure of mobility in a region $g$, where its scale is fixed by the national population. Unlike the rank-rank slope in the model without subpopulations, (ref), the slopes $\rho_g$ do not only depend on the copula of $Y$ and $X$ in a subpopulation, but also on the marginal distributions of $Y$ and $X$ in the subpopulation. To see this note that adding a fixed amount to every child and parent income in a subpopulation does not change the ranking of children and parents within the subpopulation, but it may change the ranking of these individuals in their national income distributions. In conclusion, the $\rho_g$ may then also change.

We now introduce a first-stage projection equation similar to the one in (ref), except that the coefficients are subpopulation-specific:

Let $\{(G_i,X_i,W_i,Y_i)\}_{i=1}^n$ be an i.i.d. sample from the distribution of $(G,X,W,Y)$. Ranks are computed using all observations, i.e. $R_i^X := \widehat R_X(X_i)$ with $\widehat R_X(x)$ as in (ref) and $\widehat{F}_X$ ($\widehat{F}^-_X$) the (left-limit of the) empirical cdf of $X_1,\ldots,X_n$. The computation of the rank $R_i^Y$ is analogous.

First, notice that an OLS regression of $R_i^Y$ on all regressors, i.e. $(\mathds{1}\{G_i=g\}R_i^X)_{g=1}^{n_G}$ and $(\mathds{1}\{G_i=g\}W_i^X)_{g=1}^{n_G}$, produces estimates $(\widehat\rho_g,\widehat\beta_g)_{g=1}^{n_G}$ of $(\rho_g,\beta_g)_{g=1}^{n_G}$ that can be written as:

Therefore, $(\widehat\rho_g,\widehat\beta_g)$ can be computed by an OLS regression of $R_i^Y$ on $R_i^X$ and $W_i$ using only observations $i$ from subpopulation $g$. Similarly, $\widehat{\gamma}_g$ can be computed by an OLS regression of $R_i^X$ on $W_i$ using only observations $i$ from subpopulation $g$. Note however, as explained above, that the ranks $R_i^X$ and $R_i^Y$ are computed using observations of $X$ and $Y$ from all subpopulations and thus the OLS estimators for different subpopulations are not independent.

The following are Assumptions (ref)--(ref) adapted to the model with subpopulations:

As in the previous section, note that our assumptions do not require the marginal distributions $F_X$ and $F_Y$ to be continuous. In addition, we introduce an assumption about the number and size of the subpopulations:

Observe that if the number of subpopulations were to increase together with the sample size $n$, with the number of units within each subpopulation being of the same order, the extra noise coming from estimated ranks would be asymptotically negligible, and the standard OLS variance formula would be applicable. The new result below applies to the case with a fixed number of subpopulations so that the estimation error in the ranks is not negligible even in large samples. This scenario seems reasonable, for instance, in our empirical application in which a subpopulation is a German state.

Under these four assumptions, we have the following extension of Theorem (ref).

In Appendix (ref), we prove a joint asymptotic normality result for all regression coefficients and provide explicit formulas for the asymptotic variance. In particular, letting $\widehat\rho:=(\widehat\rho_1,\ldots,\widehat\rho_{n_G})'$, $\rho:=(\rho_1,\ldots,\rho_{n_G})'$, $\widehat\beta:=(\widehat\beta_1',\ldots,\widehat\beta_{n_G}')'$, and $\beta:=(\beta_1',\ldots,\beta_{n_G}')'$, the appendix shows that

and provides an explicit formula for $\psi_i$. From this result, one can then easily calculate the asymptotic distribution of linear combinations of parameters. For instance, similarly as in the rank-rank regression without subpopulations, a popular measure of intergenerational mobility (Deutscher:2023oo) is the expected rank of a child with parents at a given income rank $p$,

The asymptotic distribution in (ref) allows us to construct a confidence interval for $\theta_{g,p}$ for a specific commuting zone $g$ or simultaneous confidence sets across all commuting zones. Here, an estimator of the asymptotic variance can be obtained using the plugin method and its consistency can be proved using the same arguments as those in Lemma (ref).

Level-Rank Regressions

In this subsection, we consider a regression model with the level of $Y$ as the dependent variable and a rank as a regressor:

Such a regression has been used, for instance, by Murphy:2020ii who regress student outcomes like test scores on students' ranks in their classrooms. Other examples are Chetty:2014tr and Abramitzky:2021ii who regress a child's outcome like college attendance or teenage pregnancy on their parent's income rank.

For the above regression, the OLS estimator takes the form

The following theorem derives asymptotic normality for $\widehat\rho$.

In Appendix (ref), we provide a joint asymptotic normality result for all regression coefficients and derive the expression for $\psi_i$. A consistent estimator of the asymptotic variance can be obtained by the plugin method, analogously to our discussion in Section (ref).

Rank-Level Regressions

In this subsection, we consider a regression model in which the outcome variable has been transformed into a rank, but the regressors are included in levels:

where, for simplicity of notation, we let the vector $W$ absorb the regressor $X$. Examples of such a regressions appear in Ghosh:2023oi who regress income rank on a binary indicator for whether a couple is in an isonymous marriage and Gronqvist:2020oi who regress a student's grade point average rank on an indicator of childhood lead exposure.

The OLS estimator takes the following form:

We then have the following result.

An explicit formula for the matrix $\Sigma$ can be found in the beginning of the proof of this theorem and, like above, a consistent estimator of $\Sigma$ can be obtained by the plugin method, analogously to our discussion in Section (ref).

Empirical Applications

In two empirical applications, we illustrate the importance of employing valid inference methods in rank-rank regressions and the sensitivity of results to how ties in the ranks are handled.

Intergenerational Income Mobility in Germany

In this section, we study regional differences in intergenerational income mobility in Germany. Following Chetty:2014tr, we measure mobility by rank-rank slopes and expected child ranks for the different states in Germany. As is common in the literature we then compare mobility in the different states to estimates from other countries. Such comparisons are important inputs to debates about potential causes for some regions or countries to have higher mobility than others. Through comparisons to high-mobility regions policy-makers in less mobile regions may seek to learn about potential policy levers for improving mobility.

\paragraph*{Data.} We use data from the German SOEP:2022aa containing income on linked child-parent pairs. The sample is a subsample of that used in Dodin:2024op, restricted to children for whom we observe the state in which they were born and restricted to states for which we observe at least 20 linked child-parent pairs. The resulting sample size is 664. The incomes of the child and the parent are computed as gross family income as in Dodin:2024op.

For the comparison with the U.S., we also use the publicly available estimates of expected child ranks and rank-rank slopes for each commuting zone (CZ) from Chetty:2014tr, provided in the replication package Chetty:2022oi.

\paragraph*{Econometric Specification.} We estimate the rank-rank slopes in (ref) with a subpopulation $g$ referring to the German state in which the child was born. $Y$ is the child's income, $X$ the parent's income, and $W$ only contains a constant. For each state $g$, we estimate the rank-rank slope $\rho_g$ and the expected rank of a child with parents at the income rank $p=0.25$, i.e., $\theta_{g,0.25}$ defined in (ref). Unlike Dodin:2024op, we do not use sample weights. Ranks are defined as in (ref) with $\omega=1$. Confidence intervals are based on the plugin estimator of the asymptotic variance in (ref) (“correct”), based on the homoskedastic (“hom”), or on the Eicker-White (“EW”) variance estimators for the regression in (ref).

\paragraph*{Results.} The estimated rank-rank slope for the whole country, i.e. using all observations in our sample, is $0.384$ with a standard error of $0.035$. This estimate indicates relatively low mobility compared to other countries like Australia ($0.215$, Deutscher:2023oo), Denmark ($0.203$, Helso:2021oo), Italy ($0.220$, Acciari:2022uu), or the U.S. ($0.341$, Chetty:2014tr).

To study the heterogeneity of mobility across states, Figure (ref) shows estimates of the rank-rank slopes ($\rho_g$) and expected child ranks ($\theta_{g,0.25}$) with 95% confidence intervals for each state. For most states, the rank-rank slope is imprecisely estimated as indicated by wide confidence intervals. The confidence intervals for expected child ranks, on the other hand, are more informative.

For some states, e.g., Baden-Wuerttemberg, the confidence intervals based on different estimators of the asymptotic variances are similar. However, there are other states, e.g. Mecklenburg-West Pomerania and Rhineland-Palatinate, for which the correct and the hom variance estimates substantially differ. For instance, Mecklenburg-West Pomerania's hom standard error for the rank-rank slope is almost twice as large as the correct one and Rhineland-Palatinate's hom standard error for the expected child rank is 27% smaller. In this dataset, the EW standard errors turned out to be close to the correct ones for all states and both parameters.

To show that the differences in confidence intervals across methods might matter, we compare the mobility estimates for the German states to those for CZs in the U.S.. To this end, for each German state and for each of the mobility parameters (rank-rank slope and expected child rank), we count how many U.S. CZs have mobility estimates that are larger (smaller) than the upper (lower) bound of the confidence interval of the German state.\footnote{Since the mobility estimates for the U.S. are constructed from the full population of administrative records, we ignore estimation uncertainty in these.} For instance, in the ranking of all 741 CZs, the correct confidence interval for Mecklenburg-West Pomerania's value of $\rho_g$ indicates that its mobility could be placed anywhere from rank 730 to 741, i.e. among the 12 CZs with the lowest mobility.\footnote{The ranking of CZs is in decreasing order of mobility, i.e. the CZ with rank equal to one has the highest mobility. Note that high mobility is indicated by a small (large) value of the rank-rank slope (expected child rank).} In contrast, the hom interval is so wide that it would place Mecklenburg-West Pomerania's mobility among the CZs ranked 130th or worse. Therefore, the correct confidence interval is substantially more informative, allowing us to rule out equal mobility with 729 of the 741 CZs, while the hom confidence interval rules out equal mobility only with 129 of the CZs.

For instance, the correct confidence interval implies that CZs like Boston, Salt Lake City, San Antonio, Minneapolis, and Washington DC have mobility significantly higher (smaller rank-rank slope) than Mecklenburg-West Pomerania, but the hom confidence interval includes all of these CZs' mobility values.

For some states, the differences in confidence intervals appear small, but the differences occur in regions of mobility values that are attained by many CZs. Therefore, even such apparently small differences can be meaningful. For instance, Saxony's correct confidence interval for the expected child rank excludes the 583 CZs with highest mobility, while the hom confidence interval excludes only 516 of the highest-mobility CZs. So, even though the hom interval is only slightly wider, it includes 67 more CZs than the correct one.

In summary, the correct confidence interval for mobility parameters in the rank-rank regression may be substantially narrower or wider than those based on other commonly used variance estimators. It therefore may lead to different conclusions in cross-country comparisons of mobility.

Intergenerational Education Mobility in India

Asher:2024iu compute rank-based measures of mobility from data on children's and parents' years of education in India. Because education is observed on discrete points of support, they partially identify rank-rank slopes for a latent, continuously distributed measure of education. We complement their analysis by directly estimating rank-rank slopes from the observed, discrete education data, highlighting the sensitivity of the rank-rank slope to the way in which ties are handled, and by providing valid confidence intervals.

\paragraph*{Data.} We use the 2012 India Human Development Survey (IHDS) dataset provided in the replication package Asher:2024aa and focus on the relationship between fathers' and sons' years of education.

\paragraph*{Econometric Specification.} For each birth cohort of the children, we estimate the rank-rank slopes in (ref) with $W$ containing only a constant. Unlike Asher:2024iu we do not use sample weights. Ranks are defined as in (ref) with $\omega\in\{0,0.5,1\}$. Confidence intervals for the rank-rank slope are based on the plugin estimator of the asymptotic variance from Section (ref). Confidence intervals for the rank correlation are computed using the bootstrap.

\paragraph*{Results.} Fathers' and sons' education is observed on seven support points from 0 to 14. Because of this discreteness, we compare rank-rank slope estimates with different ways of handling ties in the ranks. For each birth cohort and each definition of the rank, Figure (ref) reports the three terms in (ref): the estimated rank-rank slope $\widehat\rho$, Spearman's rank correlation $\widehat\rho_S$, and the ratio of estimated standard deviations $S_Y/S_X$.

Consider the graph on the left, which shows estimates and confidence intervals for the rank-rank slope. The estimates are very sensitive to the way in which ties are handled, particularly in the earlier birth cohorts. For instance, using the largest rank ($\omega=1$) leads to a rank-rank slope close to one with a relatively wide confidence interval. On the other hand, using the smallest rank ($\omega=0$) leads to a rank-rank slope of $0.42$. In the latest birth cohort (1985), the largest and smallest ranks lead to rank-rank slopes of $0.65$ and $0.42$, still a substantial difference. For the 1950 birth cohort, according to the estimates based on the smallest ranks, mobility in India is higher than in the U.S. (i.e., the rank-rank slope is smaller), while the estimates based on the largest ranks imply that India has extremely low mobility with a rank-rank slope about twice as large as that of the U.S.. Also, based on the largest ranks, mobility in India has increased (rank-rank slopes have decreased) across birth cohorts. According to the estimates based on the smallest ranks, mobility has remained constant at a high level (small rank-rank slope). It is worth emphasizing that, for a given birth cohort, the difference in estimates is due only to the way in which ties in the ranks are handled, i.e. which value of $\omega$ is used. Otherwise, the estimator and the data used are identical.

The graph in the middle shows estimates and confidence intervals for the rank correlation for different ways of handling ties. Interestingly, the estimates and their confidence intervals are almost insensitive to the way in which ties are handled. The estimates are close to $0.5$ for all birth cohorts and all values of $\omega$, and the confidence intervals are all very narrow. To explain why the estimates of the rank-rank slopes and of the rank correlation substantially differ, recall that the rank-rank slope $\widehat{\rho}$ is equal to the rank correlation $\widehat{\rho}_S$ multiplied by the ratio of standard deviations of the ranks; this is the first equality in (ref), which holds regardless of whether the marginal distributions of $X$ and $Y$ are continuous or not. If they are both continuous, then the second equality in (ref) also holds because the ratio of standard deviations is equal to one. On the other hand, when the marginal distributions are discrete, the ratio can take any value in $(0,\infty)$. While the rank correlation is almost insensitive to how ties are handled and barely varies from birth cohort to birth cohort, the ratio of standard deviations is very sensitive to the way in which ties are handled and varies substantially across birth cohorts. The ratios of standard deviations is displayed in the right-most plot in Figure (ref). When the smallest ranks ($\omega=0$) are employed, then the ratio of standard deviations is close to $0.8$ for all birth cohorts, while for the largest ranks ($\omega=1$), the ratio starts out at a value close to $1.9$ for the 1950 birth cohort and then decreases towards about $1.25$ for the 1985 birth cohort. Overall, the patterns in the ratios of standard deviations mimic those found in the rank-rank slopes, while the rank correlation remains close to $0.5$ for all birth cohorts and all definitions of the ranks. Therefore, the patterns seen in the rank-rank slopes across birth cohorts and their sensitivity to the way in which ties are handled originate from the variation in the ratios of standard deviations rather than in Spearman's rank correlation.

In summary, in this dataset, it is possible to reach robust conclusions about the value of the rank correlation, namely it being close to $0.5$ for all birth cohorts. On the other hand, the rank-rank slope estimates are so sensitive to the definition of the ranks that they do not provide robust conclusions about whether the trend across birth cohorts is increasing or decreasing, nor whether the rank-rank slope in India is larger or smaller than that of the U.S.. In particular, the trends observed in rank-rank slopes are not due to trends in the rank correlation, but rather due to trends in the marginal distribution of education.