Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
97,748 characters · 21 sections · 46 citation commands
Generalised Covariances and Correlations
Keywords: dependence measure; statistical functional; identification function; quantile correlation; copula; tail dependence
Measuring the dependence of two random variables $X$ and $Y$ has been a long-standing task in statistics with relevance for almost any empirical field of science. The two key approaches to this task are regression analysis, considering $Y$ conditional on $X$, and mutual dependence measures. The most popular measures of dependence are covariance, Pearson correlation and the rank correlations coefficients Spearman's $\rho$ and Kendall's $\tau$. Overviews of the vast literature on dependence measures are given in Mari2001, Balakrishnan2009, and Tjostheim2022. Directed dependence measures are usually normalised to range between $-1$ and $1$, and they indicate the direction of dependence by their sign and the strength of dependence by the proximity of their absolute value to 1. Crucial for the usefulness and interpretability of dependence measures are certain properties, called R\'enyi's axioms Renyi1959 and often modified subsequently Schweizer1981, Embrechts2002, Balakrishnan2009. In particular, a dependence measure should indicate the extreme forms of independence, perfect positive and negative dependence by attaining the values 0, 1 and $-1$, respectively. Pearson correlation suffers from some well-known shortcomings Embrechts2002, most importantly, attainability issues: For given marginal distributions of $X$ and $Y$ there are in general no joint distributions with these marginals achieving a Pearson correlation of $1$ and $-1$, respectively. This seriously impacts the interpretability of Pearson correlation.
Let us now consider the definitions of covariance $\mathrm{Cov}(X,Y)$ and Pearson correlation $r(X,Y)$ more closely:
where $\mathrm{Var}(X) = \mathrm{Cov}(X,X)$ denotes the variance of $X$. Covariance measures the average co-movements of $X$ and $Y$ around their respective means, $\mu(X)$ and $\mu(Y)$. Pearson correlation is a normalised version of covariance, relying on the Cauchy--Schwarz inequality. We aim at measuring the dependence of $X$ and $Y$ around general statistical functionals $T_1(X)$ and $T_2(Y)$ such as quantiles, expectiles or thresholds. Thus, we strive at providing a more complete picture of the dependence structure. This widening of the perspective is akin to the methodological advancement in other key branches of statistics. In univariate statistics, statistical functionals constitute valuable summary measures complementing the mean. In regression analysis, the shift to modelling functionals of the conditional distribution other than the mean was initiated by the advent of quantile and expectile regression koenker1978, newey1987.
In Section (ref), we introduce generalised covariance. Replacing the mean by another functional necessitates a different, in general non-linear, measurement of deviation from the functional of interest to preserve the property that independence implies nullity. We replace the classical error or deviation from the mean, $X-\mu(X)$, by generalised errors, typically constructed via identification functions. Then, our new generalised covariances are defined as the expectation of the product of these generalised errors.
Next, we normalise the generalised covariance accordingly to arrive at our generalised correlation (Section (ref)). We show that the classical Cauchy--Schwarz normalisation employed in Pearson correlation should not be used here as it leads to serious attainability issues. Instead, we propose an alternative and natural normalisation via what we call the Fr\'echet--Hoeffding bounds, which are sharp by construction. This new normalisation can also straightforwardly be used for classical covariance, which leads to what we call mean correlation, an attainable version of Pearson correlation. The generalised correlations have favourable properties, allow for measuring new forms of dependence and thus for gaining additional insights about the dependence structure between $X$ and $Y$. In particular, quantile correlation and the closely related threshold correlation arise, which allow for measuring dependence locally around a pair of quantiles of $X$ and $Y$ or around any point in the codomain of $(X,Y)$. Quantile correlation can be viewed as the correlation analogue to quantile regression.
Considering all the information on local dependence jointly, that is, using the whole families of quantile or of threshold covariances and correlations, respectively, leads to what we call distributional covariances and correlations (Section (ref)). They are function-valued objects revealing the entire dependence structure, with one-to-one connections to the copula (for the former) and the joint distribution of $X$ and $Y$ (for the latter). Since they are normalised, attaining values in $[-1,1]$, their interpretation is a lot easier than interpreting copulas or joint distribution functions. Interestingly, a constant zero (one\,/\,minus one) of these distributional correlations implies independence (perfect positive\,/\,perfect negative dependence) of $X$ and $Y$. The distributional correlations can be interpreted as generalised correlations of the identity functional, constituting the correlation counterpart of distributional regression chernozhukov2013, kneib2021.
In Section (ref) we introduce tail correlations, which arise as limits of quantile correlations, and elaborate on their connection to the widely-used coefficients of tail dependence Coles1999, Joe2014. Since they are derived from correlations, they attain values between $-1$ and $1$. Our new measures coincide with the coefficients of tail dependence if they are positive, but nicely distinguish between different strengths of negative tail dependence and tail independence, whereas the latter are 0 in these cases. Thus, constituting one of the rare occasions for a Pareto improvement in statistical methodology, they should arguably supersede the traditional coefficients of tail dependence.
Section (ref) goes back to the classical task of dependence measures, summarising overall dependence in a single number, and elaborates on the idea of integrating over distributional correlations to construct such summary covariances and correlations. Strikingly, this recovers classical and Spearman covariance as well as mean and Spearman correlation as canonical special cases.
We elaborate on the finite sample counterparts of generalised covariances and correlations and and show that they constitute consistent estimators (Section (ref)). In Section (ref), we illustrate the usage of our newly introduced dependence measures on demographic data stemming from the Panel Study of Income Dynamics. The Appendix contains proofs and additional details on the relation of generalised errors and identification functions and on the empirical applications. We provide an R package accompanying this paper at \url{https://github.com/MarcPohle/GCor}.
Let $(\Omega, \mathfrak{F}, \mathbb P)$ be a non-atomic probability space. Denote by $L^0(\mathbb{R}^d)$, $d=1,2$, the space of all $\mathbb{R}^d$-valued random variables. For $p\in[1,\infty)$, let $L^p(\mathbb{R}) = \{X\in L^0(\mathbb{R}) \mid \mathbb{E}[|X|^p]<\infty\}$ and $L^p(\mathbb{R}^2) = \{(X,Y) \in L^0(\mathbb{R}^2) \mid X, Y \in L^p(\mathbb{R})\}$. We consider statistical functionals $T$ as law-determined maps from some collection of random variables $\mathcal L\subseteq L^0(\mathbb{R})$ to a set $\mathsf{A}\subseteq \mathbb{R}$, meaning that for any $X,X'\in\mathcal{L}$ it holds that $T(X) = T(X')$ whenever $F_X=F_{X'}$, where $F_X,F_{X'}$ are the distribution functions of $X,X'$, respectively.
Generalised covariances and generalised correlations are law-determined maps from a class $\mathcal{D} \subseteq L^0(\mathbb{R}^2)$ of bivariate random vectors $(X,Y)$ to $\mathbb{R}$ and to $[-1,1]$, respectively. Recall the classical covariance and Pearson correlation on $\mathcal{D} = L^2(\mathbb{R}^2)$ from (ref). The rationale behind the definition of covariance is to measure average co-movements of $X$ and $Y$ around their respective means, $\mu(X)$ and $\mu(Y)$. The very idea behind generalised covariances is to measure average co-movements around functionals $T_1$ and $T_2$ other than the mean, e.g., around certain quantiles of $X$ and $Y$. A naive ansatz is to merely replace $\mu$ by $T$ in the definition of the covariance. However, this is not a suitable way to measure co-movements around arbitrary functionals. For example the fundamental property that independence of $X$ and $Y$ implies nullity of the generalised covariance would be violated. Covariance is constructed via deviations from the means of $X$ and $Y$, or errors, $X-\mu(X)$ and $Y-\mu(Y)$. We need to find a suitable way to measure deviations of a random variable $X$ from an arbitrary functional $T(X)$, which leads to the notion of generalised errors for $T$, capturing the most important properties of the prototypical error $X-\mu(X)$: having mean zero, being positive (negative) if $X$ realises above (below) $T(X)$, and being (weakly) larger the further $X$ realises away from $T(X)$.
Further examples of generalised errors besides the prototypical $e_\mu(X) = X - \mu(X)$ are discussed in Subsection (ref). A natural way to construct a generalised error map is via so-called identification functions.
In the field of forecast evaluation, identification functions are a central tool to assess forecast calibration NoldeZiegel2017,DimiPattonSchmidt2019. In econometrics, they are often known as moment functions, and they give rise to Z-estimation or the (generalised) method of moments estimation Huber1967, Hansen1982, NeweyMcFadden1994. An example for a strict $L^1(\mathbb{R})$-identification function for the mean, which induces the error for the mean $e_\mu(X)$ from above, is $v_\mu(t,x) = x-t$ (see again Subsection (ref) for further examples). The following proposition provides a recipe how to build generalised errors from identification functions, which will be the construction principle for all generalised errors in this paper but the ones for thresholds (Example (ref)) and quantiles in the non-continuous case (Example (ref)), where there is still a very close connection to identification functions. Assumption (ref) is spelled out in the Appendix.
Usually, an estimator or forecast for the functional $T$ is plugged in as the first argument of the identification function. Just plugging in the true functional itself yields a generalised error.
A classical sufficient condition for the integrability of the product $e_{T_1}(X) e_{T_2}(Y)$ is that the factors are square integrable, exploiting the Cauchy--Schwarz inequality. An alternative and weaker condition is provided in Proposition (ref).
Since the generalised errors are centred by definition, $\mathbb{E}\big[e_{T_1}(X)\big] = \mathbb{E}\big[e_{T_2}(Y)\big]=0$, independence implies nullity.
This nicely illustrates the rationale of generalised covariances at $T_1$ and $T_2$, measuring average co-movements of $X$ and $Y$ around their respective reference functionals. An increasing likelihood of joint positive deviations of $X$ and $Y$ from $T_1(X)$ and $T_2(Y)$ leads to an increase in $\mathrm{Cov}_{T_1, T_2}(X,Y)$. On the other hand, an increasing likelihood of countermovements decreases this covariance. Moreover, thanks to the errors being increasing, the value of the covariance is also sensitive to the magnitude of the deviations of $X$ and $Y$ from their reference functionals. Finally, if there is no systematic mutual influence between $X$ and $Y$, i.e., they are independent, the covariance vanishes.
Obviously, $\mathrm{Cov}_{T_1, T_2}(X,Y)$ depends on the choice of the generalised errors, $e_{T_1}$, $e_{T_2}$. For the leading situation when the generalised error is induced by an identification function (see Proposition (ref)), we characterise this dependence in Section (ref) of the Appendix (Remark (ref)) and remark that generalised correlations are actually independent of the choice of the identification function, subject to regularity conditions (Proposition (ref)). For the examples discussed in the following subsection we utilise the canonical identification functions as suggested by GneitingResin2021.
Some of the examples of generalised covariance we discuss here have appeared in the literature. We discuss relations to the literature in Subsection (ref) when introducing the respective generalised correlations.
Just as the mean, the $\tau$-expectile is translation equivariant in the sense that $\mu_\tau(X + c) = \mu_\tau(X)+c$ for all $X\in L^1(\mathbb{R})$ and $c\in\mathbb{R}$, and it is positively homogeneous, i.e., $\mu_\tau(\lambda X) = \lambda \mu_\tau(X)$ for all $X\in L^1(\mathbb{R})$ and $\lambda>0$. The canonical expectile identification function (ref) shares similar properties: It is positively homogeneous and translation invariant. Hence, the expectile covariance is also translation invariant and positively homogeneous in both arguments.
The quantile enjoys even more invariance properties than the expectile. It is equivariant under all strictly increasing transformations: $q_\alpha\big(g(X)\big) = g\big(q_\alpha(X)\big)$ for any $X\in L^0(\mathbb{R})$ and for any strictly increasing $g\colon\mathbb{R}\to\mathbb{R}$. The quantile error (ref) inherits this invariance, $e_{q_{\alpha}} (g(X)) = e_{q_{\alpha}} (X)$. Hence, the induced quantile covariance is also invariant with respect to strictly increasing transformations in both arguments.
To turn a covariance into a correlation, it is essential to ensure normalisation of the correlation meaning that it only attains values in $[-1,1]$. Normalisation enhances the interpretability of a correlation. It can be achieved by bounding the covariance in absolute values with a constant $K_{X,Y}$, which may depend on the marginal distributions of $X$ and $Y$
Then, trivially $\mathrm{Cov}_{T_1,T_2}(X,Y)/ K_{X,Y}\in[-1,1]$. Pearson correlation $r(X,Y)$ (ref) builds on this idea and exploits the Cauchy--Schwarz inequality to bound the covariance. That is, for $X,Y\in L^2(\mathbb{R})$,
For the generalised covariance, we could hence simply exploit the identity (ref) and obtain
provided that $e_{T_1}(X), e_{T_2}(Y) \in L^2(\mathbb{R})$. The implied quantity is just the Pearson correlation of the generalised errors
However, it is well-known that the Cauchy--Schwarz inequality (ref) is not sharp in general. Consequently, the Pearson correlation of the generalised errors (ref) does not always attain all values in $[-1,1]$ for given marginal distributions of $X$ and $Y$. In particular, its minimum and maximum may be far away from $-1$ and 1, respectively, and their absolute values may differ strongly, depending on the specific marginal distributions of $X$ and $Y$, see Embrechts2002. This compromises its interpretability in that it is not really able to indicate the strength of dependence (by the closeness of its absolute value to 1). In fact, a value close to 0 may arise even if the dependence is quite strong for certain marginal distributions.
We revisit the conditions for attainability of Pearson correlation more closely now. For $X,Y\in L^2(\mathbb{R})$, $r(X,Y)$ is $1$ ($-1$) if and only if $X$ and $Y$ have perfect positive (negative) linear dependence. That is, if and only if for some $a, a'\in\mathbb{R}$, $b, b'>0$ it holds almost surely that $Y = a + bX$ ($Y = a' - b'X$). Therefore, for given non-degenerate marginals $F_X$ and $F_Y$, there exists a joint distribution $F_{X,Y}$ with Pearson correlation $1$ ($-1$) if and only if $Y$ and $X$ ($-X$) are of the same type, meaning that $Y \stackrel{\mathrm{d}}{=} a + bX$ ($Y \stackrel{\mathrm{d}}{=} a' - b'X$) for some $a, a'\in\mathbb{R}$, $b, b'>0$. This observation leads to the following insight.
The restriction to symmetric distributions of the same type is substantial beyond the confinements of a Gaussian world. In the context of generalised covariances, which can be written as covariances of generalised errors (see (ref)), it is definitely too restrictive. Here, the distributions of the generalised errors $e_{T_1}(X)$ and $e_{T_2}(Y)$ are generally neither symmetric nor of the same type.
The following proposition provides a sharp version of the inequality (ref), exploiting previous results by hoeffding1940, Frechet1957, and Embrechts2002. Recall that $(X,Y)$ are called comonotonic if $(X,Y)\stackrel{\mathrm{d}}{=} \big(\nu_1(Z), \nu_2(Z)\big)$ for some random variable $Z$ and two increasing functions $\nu_1$, $\nu_2$. Similarly, $(X,Y)$ are countermonotonic if $(X,Y)\stackrel{\mathrm{d}}{=} \big(\nu_1(Z), \nu_2(Z)\big)$ for some random variable $Z$ with $\nu_1$ increasing and $\nu_2$ decreasing. In other words, comonotonicity (countermonotonicity) corresponds to the situation of perfect positive (negative) dependence between $X$ and $Y$.
Combining generalised covariance from Definition (ref) with the Fr\'echet--Hoeffding normalisation from Proposition (ref) leads to generalised correlation.
We summarise the most important properties of generalised correlation.
Properties (i), (ii) and part (a) of (iii) are fundamental properties that every correlation-type dependence measure should fulfil. They ensure interpretability in that the measure takes the right values in the extreme cases of independence and perfect positive and negative dependence. Part (b) of (iii) is not always desirable, e.g., when it comes to local dependence measures, see Remark (ref).
We would like to highlight the novelty of normalising covariances with Fr\'echet--Hoeffding bounds, which are sharp by construction. We are aware of similar constructions only in the context of a dependence measure for binary random variables cole1949 and to combat the non-attainability of Spearman's $\rho$ and Kendall's $\tau$ in the discrete case VandenhendeLambert2003, genest2007.
We are going to review the examples of generalised covariances from Subsection (ref) and see how they translate into generalised correlations. To that end, we will mainly focus on the normalisation terms, that is, the denominators in (ref). Throughout this section, let again $(X,Y)$, $(X,Y')$, $(X,Y'')$ be pairs with the same marginal distribution, where $(X,Y')$ is countermonotonic and $(X,Y'')$ is comonotonic.
It follows from Proposition (ref) that expectile correlation and hence in particular mean correlation is invariant under linear transformations.
Quantile correlation is invariant with respect to strictly increasing transformations, which follows from Proposition (ref). Thus, quantile correlation belongs to the family of rank correlations.
Quantile correlation does not only generalise median correlation. When moving from the center to the tails, that is, considering the limit of $\mathrm{QCor}_{\alpha,\alpha}$ for $\alpha \to 0$ or $\alpha \to 1$, a quantity closely related to the well-known coefficient of tail dependence shows up which is discussed in Section (ref).
There are some measures in the literature related to quantile, quantile-mean and threshold covariances and correlations. Linton2007 employ a closely related quantity in their quantilogram, namely our quantile covariance (see Example (ref)), but normalised with the Cauchy--Schwarz normalisation, see also Han2016. Similarly, what Li2015 call quantile correlation amounts to our quantile-mean covariance (see Example (ref)), but again normalised with the Cauchy--Schwarz normalisation. Further, so-called indicator covariances from spatial statistics Dubrule2017 are closely related to our threshold covariances (see Example (ref)).
As discussed in Remark (ref), threshold and quantile covariance can be seen as local measures of dependence around a certain point. Accordingly, we call threshold and quantile correlation local correlations. Surprisingly, such local dependence measures have hardly been touched upon in the literature. Noteworthy exceptions are the local dependence function of Holland1987, see also Jones1996, and the local Gaussian correlation of Tjostheim2013.
The generalised covariances and correlations presented so far measure the dependence of two random variables $X$, $Y$ around the two functionals $T_1(X)$, $T_2(Y)$. As such, they focus on a certain aspect of the dependence structure of $X$ and $Y$. This section proposes measures that uncover the entire dependence structure of $X$ and $Y$. The idea behind those measures is to consider all the local information contained in the whole families of local covariances (correlations) jointly (see Remark (ref)), leading to dependence measures that are functions in two arguments -- the thresholds $(a,b) \in \mathbb{R}^2$ or quantile levels $(\alpha,\beta) \in (0,1)^2$.
An alternative way to arrive at the same measures is to approach them from the angle of generalised covariances and correlations by not considering point-valued functionals $T_1(X)$ and $T_2(Y)$ as in Sections (ref) and (ref), but the entire CDF or the quantile function themselves. An $L^0(\mathbb{R})$-identification function for the identity or CDF-functional is the function-valued map $v_{\mathrm{CDF}}(F,x) = \big(F(a) - \mathds{1}\{x\le a\}\big)_{a\in\mathbb{R}}$, leading to the function-valued generalised error $e_{\mathrm{CDF}}(X) = \big(F(a) - \mathds{1}\{X\le a\}\big)_{a\in\mathbb{R}}$. Alternatively, one may consider the quantile function (QF) functional, mapping a CDF, $F$, to its generalised inverse, $F^{-1}$. On the class of random variables with a continuous CDF, denoted by $L_{\text{con}}$, we have the $L_{\text{con}}$-identification function $v_{\mathrm{QF}}(F^{-1},x) = \big(\alpha - \mathds{1} \{x \le F^{-1}(\alpha)\}\big)_{\alpha\in(0,1)}$. The modification discussed in Example (ref) for the general case leads to the generalised error $e_{\mathrm{QF}}(X) := \big( F_X( q_\alpha (X) ) - \mathds{1}\{X \le q_\alpha(X)\} \big)_{\alpha\in(0,1)}$. We can construct generalised covariances (and correlations) from $e_{\mathrm{CDF}}$ and $e_{\mathrm{QF}}$ via an outer product ansatz, generalising (ref). This reasoning explains why we call the resulting function-valued dependence measures CDF and quantile function covariance (correlation), and subsume both under the name distributional covariances (correlations).
Just as in univariate statistics when characterising the distribution of a single random variable, we may either take the perspective of the quantile function or of the CDF when characterising dependence between two random variables, as there are two corresponding families of distributional dependence measures. Indeed, the distributional covariances and correlations characterise the dependence structure between $X$ and $Y$ fully: Given the marginals $F_X$ and $F_Y$, it is possible to recover the joint CDF $F_{X,Y}$ or the copula $C_{X,Y}$, which contain the information on the full dependence structure, from $\mathrm{CDFCor}$ or $\mathrm{QFCor}$ (as well as $\mathrm{CDFCov}$ or $\mathrm{QFCov}$). This is clear from the representations of $\mathrm{TCov}$ in terms of $F_{X,Y}$ and the marginals in (ref) and $\mathrm{QCov}$ in terms of the $C_{X,Y}$ and the marginals in (ref) as well as the normalisations, which only depend on the marginals, see Examples (ref) and (ref). If $F_X$ and $F_Y$ are continuous, $\mathrm{QFCor}$ and $\mathrm{QFCov}$ are independent of the marginals, and by (ref) and again the normalisations from Example (ref), there is even a one-to-one mapping between $\mathrm{QFCov}$ and $C_{X,Y}$ (as well as $\mathrm{QFCor}$ and $C_{X,Y}$). While of course the joint CDF and copula contain the information on the dependence structure as well, we emphasise that particularly the normalised quantities, that is, the distributional correlations, have the great advantage that they are really dependence measures. Thus, they uncover the full dependence structure, and regions of stronger and weaker dependence can be identified relatively easily, see Subsection (ref) and Section (ref) for theoretical and empirical examples, respectively.
Distributional correlations are able to characterise the limiting cases of independence and perfect positive and negative dependence properly. In particular, nullity of any of the distributional covariances or correlations implies independence (Proposition (ref)). Further, unity (negative unity) of distributional correlations implies perfect positive (negative) dependence (Corollary (ref)). Additionally, $\mathrm{QFCor}$ is invariant to strictly increasing transformations.
Figures (ref) and (ref) present plots of quantile function correlations for classical examples of bivariate copulas, respectively. Figure (ref) depicts the quantile function correlation of a bivariate Cauchy copula with a Spearman correlation of 0 alongside a scatter plot of 1000 random draws from this copula. While joint CDFs or copulas and their respective densities are often difficult to interpret graphically, such a scatter plot has usually been considered the best, yet informal, tool to depict and understand the dependence structure. Quantile function correlation provides a formal tool that uncovers the full dependence structure: Even though $X$ and $Y$ have a Spearman correlation of 0, there is quite a strong dependence in the tails. In the first and third quadrants the dependence is positive, meaning that larger values (exceedances of certain quantiles) of $X$ are associated with larger values (exceedances of certain quantiles) of $Y$, while in the second and fourth quadrants it is negative, meaning that larger values (exceedances of certain quantiles) of $X$ are associated with smaller values (falling short of certain quantiles) of $Y$. Actually, Spearman's $\rho$ is equal to a properly normalised Lebesgue integral over $\mathrm{QFCov}$ (see Example (ref)) and thus the positive and negative dependence here cancels out when summarising the full dependence structure represented by $\mathrm{QFCor}$ in a single number.
Figure (ref) contains quantile function correlations stemming from four different copulas, all having a Spearman correlation of 0.5: a Gaussian, a Cauchy, a Clayton, and a Gumbel copula. Despite their Spearman's $\rho$ being the same, they exhibit very different dependence structures. For example, the Gaussian copula has a weak dependence in the lower left and upper right corner, while the Cauchy copula shows a strong dependence in both corners, and the Clayton and Gumbel copula exhibit strong dependence in one corner, but not the other. In the upper left and lower right corners, for all but the Cauchy copula there is strong positive dependence, which just means, e.g., for the lower right corner that exceedances of a high quantile of $X$ are strongly positively associated with exceedances of a small quantile of $Y$. This means that very large values of $X$ and very small values of $Y$ virtually never occur jointly for those copulas. In contrast, for the Cauchy copula, despite the overall positive dependence between $X$ and $Y$ as indicated by Spearman's $\rho$, the dependence in the lower right and upper left corners even becomes negative, reflecting the tail behaviour of the Cauchy distribution also seen in Figure (ref). Bivariate t-distributions with more degrees of freedom exhibit a similar behaviour as the Cauchy examples in both figures. Plots of the closely related CDF correlations (see Remark (ref)) usually look like distorted versions of quantile function correlations, where the distortion originates from the influence of the marginals. We discuss examples of CDF correlations in Section (ref).
There is a vast literature on the question what it means or what it should mean that two random variables $X$ and $Y$ are globally positively (negatively) dependent. We refer to Mari2001 and Balakrishnan2009 for overviews of this strand of literature. The distributional covariances and correlations suggest a natural definition.
The following proposition shows that we can define global positive and negative dependence also in terms of quantile function correlation.
For example, the first three copulas from Figure (ref) are globally positively dependent, while the Cauchy copulas in Figures (ref) and (ref) represent cases of mixed dependence, with regions of local positive as well as local negative dependence.
Each generalised covariance and correlation directly gives rise to a specific concept of positive (negative) dependence between two random variables $(X,Y)$ as well. The following proposition shows that such a dependence is implied by global dependence.
Proposition (ref) can also be viewed as a complement to Theorem (ref) in that it contains a further desirable property of generalised correlations: They not only take the correct values of $-1$\,/\,0\,/\,1 in the extreme cases, but also the correct sign under positive and negative dependence.
It is often of interest to analyse co-movement in the tails, i.e., if for example very large values in $X$ and $Y$ tend to occur together, if there is no dependence in the tails, or if large values of $X$ render large values of $Y$ rather more unlikely and vice versa. Such questions are subsumed under the term of tail, extremal or asymptotic dependence in the literature, see, e.g., Joe2014 or Coles1999.
Natural measures for lower and upper tail dependence are the respective limits of quantile correlation or the limits of quantile function correlation when moving to the upper right and lower left corner, respectively.
On top of the upper and lower tail correlation, one might also consider $\lim_{\alpha \to 0} \mathrm{QCor}_{\alpha,1-\alpha}(X,Y)$ and $\lim_{\alpha \to 0} \mathrm{QCor}_{1-\alpha,\alpha}(X,Y)$, provided that these limits exist. The discussion is similar to what follows and is therefore omitted.
The by far most prominent measure of tail dependence is the coefficient of tail dependence, see Joe1993, Coles1999.\footnote{Fiebig2017 contains a literature review on the use and naming of the coefficient in different fields.} To facilitate the following discussion of the relation between coefficients of tail dependence and tail correlations we assume continuity of the marginals $F_X$ and $F_Y$ throughout, stated by $X, Y\in L_{\mathrm{con}}$. The coefficients of lower and upper tail dependence are defined as \[ \lambda_l(X,Y) := \lim_{\alpha \to0} \mathbb{P} ( Y \leq q_{\alpha} (Y) \mid X \leq q_{\alpha} (X)) = \lim_{\alpha \to0} \frac{C_{X,Y}(\alpha,\alpha)}{\alpha}, \quad X, Y\in L_{\text{con}}, \] and \[ \lambda_u(X,Y) := \lim_{\alpha \to1} \mathbb{P} ( Y > q_{\alpha} (Y) \mid X > q_{\alpha} (X)) = \lim_{\alpha \to1} \frac{\overline{C}_{X,Y}(\alpha,\alpha)}{ 1 - \alpha}, \quad X, Y\in L_{\text{con}}, \] where $\overline{C}_{X,Y}$ denotes the survival function of the copula $C_{X,Y}$ and where we assume that the limits exist. The following lemma clarifies the relation between the coefficients of tail dependence and the tail correlations.
In the literature, the cases $\lambda_l=0$ and $\lambda_u=0$ are usually called tail or asymptotic independence and $\lambda_l>0$ and $\lambda_u>0$ tail dependence mcneil2015. The lower and upper tail correlations provide a more nuanced picture of the tail behaviour. While under positive tail dependence as introduced in Definition (ref), the coefficients of tail dependence and the tail correlations coincide, the latter measures are able to classify the situation of $\lambda_l=0$ and $\lambda_u=0$ into actual tail independence (the tail correlations are 0) on the one hand and negative tail correlation on the other hand -- also indicating the strength of negative dependence. In fact, a countermonotonic pair $(X,Y)$ yields a lower (upper) tail correlation of $-1$, while the coefficients of tail dependence are still 0, deeming the pair asymptotically independent.
Hence, we make the case for replacing the coefficients of tail dependence with the tail correlations: Since no information is lost, but strictly more information is gained, this is one of the rare cases in statistical methodology where a Pareto improvement is possible and should therefore be implemented.
The distributional covariances and correlations from Section (ref) reveal the full dependence structure between $X$ and $Y$. Nevertheless, it is often required or useful to summarise the dependence structure in a single number. In fact, this is what most classical dependence measures aim for. A natural way to construct such summary measures from distributional covariances is to compute weighted averages.
We tacitly assume that the integrals in (ref) and (ref) exist and are finite. Since both $\mathrm{QCov}$ and $\mathrm{TCov}$ are bounded, a sufficient condition is that $\nu$ and $\kappa$ are finite. The summary covariances inherit the properties of the distributional covariances. They are 0 under independence and nonnegative (nonpositive) under global positive (negative) dependence. Further, the quantile function summary covariance is invariant under strictly increasing transformations.
Interestingly, two of the most popular dependence measures arise as canonical special cases of summary covariances.
We discuss further examples, which focus on specific regions of interest, when dealing with the respective correlations below.
Summary correlations arise when summary covariances are appropriately normalised, again utilising co- and countermonotonic random couplings with identical marginals as $X$ and $Y$.
The normalisation terms in (ref) and (ref) can be computed from (ref) and (ref) and the Fr\'echet--Hoeffding bounds presented in Examples (ref) and (ref). The summary correlations inherit the appealing properties of the distributional correlations.
By Proposition (ref), $\mathrm{SCor}_{\mathrm{QF},\kappa}$ is invariant under strictly increasing transformations of $X$ and $Y$ as well.
In practice, Pearson and Spearman correlation are most often interpreted as summaries of the full dependence structure, expressed in a single number. However, by definition covariance and Pearson correlation measure dependence around the means (which was the starting point of this paper), while Spearman's $\rho$ does the same on the rank scale, see (ref) and (ref). Our formal approach to summary correlations, where they arise as canonical special cases, provides a powerful justification for this practical use. Of course, the dependence structure cannot be fully described by a single number, for example for all the bivariate copulas in Figure (ref), Spearman's $\rho$ has the same value, $\rho=0.5$, but they have very different dependence structures as the distributional correlations uncover. The two closely related families (threshold and quantile family) of local, distributional and canonical summary correlations provide dependence measures for different purposes and should be chosen according to the statistical problem at hand: measuring dependence locally, characterising the full dependence structure or condensing it in a single number.
The empirical analogues of generalised covariances and correlations are fairly straightforward, exploiting plug-in estimators and the method of moments. They constitute natural and consistent estimators for the respective quantities on the population level. Suppose we have a random sample $\{(X_i,Y_i),\ i=1,\ldots, n\}$ from $F_{X,Y}$ and we are interested in estimating the generalised covariance, $\mathrm{Cov}_{T_1,T_2}(X,Y)$, or generalised correlation, $\mathrm{Cor}_{T_1,T_2}(X,Y)$, at $T_1$, $T_2$. Further, suppose that the corresponding generalised errors are induced by increasing and non-constant $\mathcal{L}_1$- and $\mathcal{L}_2$-identification functions $v_{1}$ and $v_{2}$, invoking Proposition (ref). To start with, suppose that $\mathcal{L}_1$ and $\mathcal{L}_2$ contain all random variables with any empirical distribution. Then, we can simply apply the definition of the generalised covariance and correlation to the empirical distribution and use this as an estimator for the population quantity. To obtain the sample analogue $\hat t_1^n := \widehat T_1^n(X)$ of an identifiable functional $T_1(X)$, we use the solution in $t$ of
Similarly, we write $\hat t_2^n := \widehat T_2^n(Y)$. Then, the generalised covariance on the sample level is
To obtain the normalisation for the generalised correlation, we take the empirical marginal distributions from the random sample $\{(X_i,Y_i),\ i=1,\ldots, n\}$, but couple the observations with the corresponding co- and countermonotonicity copulas, utilising the increasing order statistics $X_{(1)}\le \cdots \le X_{(n)}$ and $Y_{(1)}\le \cdots \le Y_{(n)}$. Hence, we obtain for the comonotonic coupling $(X,Y'')$ and the countermonotonic coupling $(X,Y')$
Finally, we set
For threshold and quantile correlation the estimation simplifies as the normalisation does not involve the co- and countermonotonic coupling. For threshold correlation, $\mathrm{TCor}_{a,b}(X,Y)$, we only need to estimate $F_{X,Y} (a,b)$, $F_X(a)$ and $F_Y(b)$ via
and replace the theoretical quantities by their empirical counterparts in $\mathrm{TCov}_{a,b}(X,Y)$ from (ref) and the normalisation terms in Example (ref).
To estimate quantile correlation, $\mathrm{QCor}_{\alpha,\beta}(X,Y)$, we first obtain the sample $\alpha$- and $\beta$-quantiles, or more formally, for the canonical identification functions $v_{q_\alpha}$ and $v_{q_\beta}$ in (ref) we set \[ \hat q^n_\alpha := \inf \{t \mid \frac{1}{n}\sum_{i=1}^n v_{q_\alpha}(t,X_i)\le0\}, \qquad \hat q^n_\beta := \inf \{t \mid \frac{1}{n}\sum_{i=1}^n v_{q_\beta}(t,Y_i)\le0\}. \] Then, we obtain estimates for $C_{X,Y}\big(F_X(q_\alpha(X)), F_Y(q_\beta(Y))\big)$, $F_X(q_\alpha(X))$ and $F_Y(q_\beta(Y))$ by replacing the thresholds $a$, $b$ in (ref) with $q^n_\alpha$ and $q^n_\beta$ and replacing the theoretical quantities in (ref) and the normalisation terms in Example (ref) with those estimators.
To construct estimators for distributional correlations, we use the respective estimators for the local correlations just described on a grid (of size 10000 in the following applications). We close this section by establishing the consistency of the described estimators, subject to typical regularity conditions.
Threshold covariance and threshold correlation do not satisfy the conditions of Proposition (ref), but the strong consistency follows directly from the strong law of large numbers and the continuous mapping theorem. For quantile covariance and quantile correlation, we establish strong consistency if the marginal distributions are continuous at the respective quantiles, see Proposition (ref).
To illustrate the use of local, distributional and summary correlations in practice, we extract data on mixed-sex couples from the 2019 wave of the Panel Study of Income Dynamics.\footnote{Citation: Panel Study of Income Dynamics, public use dataset. Produced and distributed by the Survey Research Center, Institute for Social Research, University of Michigan, Ann Arbor, MI (2023).} After data cleaning we have a sample of 4417 couples living in the same households, of whom 85 % are married. As all of the variables we analyse are discrete (either due to being inherently discrete or, e.g., heights being recorded in full inches), we use bubble plots for our scatter plots to avoid overplotting. Further, note that CDF and quantile function correlation only change their value at jumps of the CDF (see Remark (ref)). In all graphical representations we focus on the ranges between the 2.5%- and the 97.5%-quantiles.
We first consider the heights of the couples in cm. Figure (ref) depicts a scatter plot and the corresponding CDF correlation. Heights of men and women are globally positively dependent as CDF correlation is positive everywhere, indicating a preference for assortative mating stulp2013. Mean correlation equals $\mathrm{MCor}=0.215$ (Spearman's $\rho=0.189$), indicating a weak positive relation on average. The local dependence structure, however, varies strongly: In the lower right corner, the dependence is quite strong, while elsewhere (with the exception of the upper left corner) it is weak. This reflects the male-taller norm in Western societies stulp2013: For women and men fairly close to the average height difference (indicated by the dotted line), CDF correlation is close to 0, indicating that mating behaviour in this region is hardly influenced by the partner's (in relation to the own) height. As the height of the man approaches the height of the woman (the diagonal is represented by the solid line), CDF correlation rises abruptly, suggesting that the mating behaviour is strongly influenced by height in this region: heights of partners are strongly positively associated there.\footnote{Actually, the norm is rather that the woman should be at least a few centimetres smaller than the man.} When computing the regional summary correlation $\mathrm{SCor}_{\mathrm{CDF},\lambda_B}$ from Example (ref) below and above the diagonal we consequently get 0.601 and 0.201, respectively. In the far upper left corner the dependence gets quite strong as well, reflecting the male-not-too-tall norm stulp2013. Quantile function correlation yields qualitatively the same picture, see Figure (ref) in the Appendix and the discussion below.
Figure (ref) depicts the bubble plot, the CDF and quantile function correlation for the body mass indices (BMI, mass in kg divided by the square of height in cm) of the couples. Again the two variables are (almost) globally positively dependent and the canonical summary correlations from examples (ref) and (ref) take the values $\mathrm{MCor}=0.304$ and $\rho=0.287$. In the lower left corner below a BMI of 25, which corresponds to the threshold between having a normal weight and being overweight according to the World Health Organization (WHO),\footnote{\url{https://www.who.int/europe/news-room/fact-sheets/item/a-healthy-lifestyle---who-recommendations}, accessed: 7th May 2023} $\mathrm{CDFCor}$ and $\mathrm{QCor}$ are close to 0, whereas elsewhere they are larger, in particular if one of the partners exceeds the threshold of 30, from which on a person is classified as obese according to the WHO. Thus, if both partners are in a normal weight range, there seems to be no association between BMIs, but once one of the partners is obsese, the assocation gets strongly positive.\footnote{Here, the dependence is of course not only generated by mating behaviour, but also by mutual influence in lifestyle.} This example nicely illustrates how CDF and quantile correlation may differ and how they complement each other: While $\mathrm{CDFCor}$ is often nicely interpretable as it operates on the observation scale itself, sometimes a lot of space in the plot is occupied by regions where few observations lie (which is why we chose to present only the central regions of the axes with 95% of the probability mass in the first place). For example here, due to the marginal distributions of the BMIs being right-skewed, regions with higher values occupy comparatively more space in the plot. $\mathrm{QFCor}$, on the other hand, always has a solid interpretation in terms of quantile levels and naturally assigns space in the plot according to probability mass.
Next, we consider an example exhibiting regions of positive and of negative dependence. Figure (ref) contains the scatter plot as well as the CDF correlation of the number of strength trainings per week and the BMI of the men in our sample. On average, frequency of strength training is negatively associated with BMI, $\mathrm{MCor}=-0.112$ and $\rho=-0.088$. Locally, the negative dependence starts to arise with about the overweight threshold of 25 and gets stronger the higher BMI gets, being particularly strong for people above the obesity threshold of 30. For men in the upper normal range of BMI, $\mathrm{CDFCor}$ is close to 0, whereas for men in the lower normal range, the association is positive. Thus, for people with a low BMI strength training is associated with a gain in body weight -- probably due to increased muscle mass, while for overweight people it is associated with a loss in body weight -- probably due to fat loss outweighing increased muscle mass.
In Part (ref) of the Appendix we provide the above-mentioned additional two figures and compare mean correlation from Example (ref) and Pearson correlation from (ref) for the three examples above and a fourth one.
We present new concepts and measures of dependence. On the one hand, we consider dependence from the perspective of statistical functionals and put forward generalised covariances and correlations as corresponding dependence measures. On the other hand, with our local and distributional correlations we introduce local dependence (including tail dependence) measures as well as function-valued measures uncovering the full dependence structure. Summary correlations average over distributional correlations and close the loop to classical measures of dependence like covariance, Pearson correlation and Spearman's $\rho$. We analyse the properties of the new measures and present first applications.
These measures open many opportunities for future research. First of all, they will be useful in a wide variety of applications, possibly providing deeper insights about dependence structures than classical measures. Our measures are concerned with dependence between two random variables. Thus, they naturally extend to settings where pairwise dependence plays a role, for example correlation matrices, temporal or spatial dependence. Extending our measures to examine dependence for a vector of variables jointly is naturally more difficult, as negative dependence turns into a subtle concept for more than two variables Mari2001. Further aspects of statistical inference, beyond those related to estimation and discussed in the paper, are relegated to future research. Our measures are fundamentally different from recent popular measures of functional dependence szekely2007, szekely2009, chatterjee2021, reshef2011, which are not measures of directional dependence, but only of strength of dependence, thus mapping to $[0,1]$. Analysing possible connections of those measures to distributional or summary correlations is certainly interesting. Finally, exploring the links between generalised correlations and generalised regression approaches might be fruitful.
We are grateful to Patrick Cheridito, Timo Dimitriadis, Tilmann Gneiting, Bettina Gr\"un, Alexander Jordan, Johanna Ne\v{s}lehov\'a, Melanie Schienle and Jan-Lukas Wermuth for valuable discussions about the topic. We further thank seminar participants at Heidelberg University, Heidelberg Institute for Theoretical Studies, Vienna University of Economics and Business, University of Sussex and ETH Z\"urich and conference participants at DAGStat 2022 and the Bernoulli Young Researcher Event 2022 for helpful comments. Marc-Oliver Pohle is grateful for support by the Klaus Tschira Foundation, Germany. The collection of data used in this study was partly supported by the National Institutes of Health under grant number R01 HD069609 and R01 AG040213, and the National Science Foundation under award numbers SES 1157698 and 1623684.