Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
86,043 characters · 29 sections · 90 citation commands
Robust Estimation of Realized Correlation: New Insight about Intraday Fluctuations in Market Betas
Keywords:{ Correlation, Pearson, Kendall, Subsampling, Robustness, Consistency, Epps effect, High-frequency data, Microstructure, Jump}
{}
The correlation is a measure of association between two variables that plays a central role in many empirical methods. The correlation is most commonly estimated by the sample correlation, which is known as Pearson's $r$. Other classical correlation estimators include the Quadrant estimator, the Kendall's tau, Spearman's rank correlation, and the Gaussian rank correlation estimator, see Kruskal:1958 for the relationships between these measures and an historical account of their developments. The choice of estimator involves a tradeoffs between robustness and efficiency. This tradeoff is influenced by many features of the underlying distribution, including heteroskedasticity that is particularly important for many economic applications.
In this paper, we propose a new robust correlation estimator that is well-suited for heteroskedastic time-series, such as high-frequency financial data. Time-varying volatility and market microstructure noise are innate features of high-frequency financial data, and both features undermine the reliability of standard correlation estimators. We compare the sensitivity of correlation estimators to departures from homoskedasticity and show that the Quadrant estimator is the only estimator that is robust to heteroskedasticity, among the classical estimators. The other estimators are inconsistent, except in very special cases. Unfortunately, the Quadrant estimator is rather inefficient. We recover much efficiency by combining the Quadrant estimator with subsampling and this makes it possible to improve efficiency while retaining consistency. We derive theoretical properties of the new estimator and study it using simulation designs that mimic empirical high-frequency financial data. We show that the realized correlation can be very biased as documented in our empirical analysis. We apply the new estimator to high-frequency data for 22 assets and an exchange-traded fund that tracks the S&P 500 index. The empirical results suggest that the new estimator is more accurate than other estimators, with the improvements likely resulting from better bias properties. We combine intraday correlation estimates with estimates of relative volatility to form an estimate of intraday market beta, as analyzed in AndersenThyrsgaardTodorov:2021. We find substantial variation in market betas within the trading day, with some stocks having increasing betas over the trading hours, while others tend to have decreasing betas. Our empirical results corroborate the finding in AndersenThyrsgaardTodorov:2021, even though we use different estimation methods and a different (narrower) estimation window. Our estimation approach enables us to decompose the time variation in betas into time variation in correlation and time-variation in relative volatility. Interestingly, we find that the variation in betas is mainly driven by time-variation in correlations. Relative to the market, all assets in our analysis have increasing correlations and decreasing relative volatilities over the trading hour. The declines in relative volatilities are very similar across assets. The relative volatility during the last hour of active trading is typically between 50%-75% of relative volatility during the first hour of trading. There is far more variation across assets in terms of their correlations with the market. For many assets their market correlation is 2-5 times larger during the last hour than during the first hour. These assets have nearly linearly increasing market betas during the trading hours. Another set of assets, which are characterized by high market correlations, have their correlations increase by much less than 100% during the day. These assets have, on average, decreasing market betas during the trading hours. Thus, we document that intraday variation in both correlation and relative volatility contribute to the variation in market betas, but the variation across assets is primarily driven their time-variation in correlations with the market.
Time-varying volatility in high-frequency financial data is well documented, see e.g. AndersenBollserslev:1998b. Similarly, it is well documented that market microstructure noise can harm realized measures of volatility, see Zhou:1996\nocite{Zhou:1998}, zhang-mykland-aitsahalia:05, BandiRussell:2006, and HansenLunde:JBES2006. Market microstructure noise is defined as the difference between the observed prices and true prices. The latter are characterized by having certain martingale properties, whereas the former typically entails some degree of predictability. Market microstructure noise arises from many intricate aspects of high-frequency data. For instance, noise can arise as artifacts of imputation methods and recording and rounding errors. These issues are all important for correlation estimation, see e.g. Reno2003, PrecupIori:2007, and MunnixSchaeferGuhr:2011, and TothKertesz:2007.\nocite{TothKertesz:2009} The lack of synchronicity in observation times induces a type of noise that is particularly important for covariance and correlation estimation. This will often manifest as the Epps effect, where the sample correlation decreases as the sampling frequency increases, see Epps:1979. HayashiYoshida:2005 proposed an estimator that adjusts for asynchronicity and VoevLunde:2007 and GriffinOomen:2011 proposed related estimators that are robust to additional forms of noise. Jumps in prices and adversely effect empirical measures, including realized variances, covariances, and correlations. However, these effects can be alleviated by truncation methods, see Mancini:2009 and RaymaekersRousseeuw:2021.
A standard remedy for market microstructure noise in high-frequency data is sparse sampling. AndersenBollerslev:1998a estimated realized variances using $5$-minute intraday returns and this sampling frequency appears to offer a reasonably good compromise between bias and variance in many applications, see e.g. HansenLunde:JBES2006, BandiRussell:2008, and LiuPattonSheppard:2015. Realized measures that utilize more information include the subsampled realized variance by zhang-mykland-aitsahalia:05, the realized kernel estimator by BNHLS:2008, and the pre-averaging estimators by JacodLiMyklandPodolskijVetterPreaverage. These three approaches, subsampling, realized kernels, and pre-averaging, lead to the same class of estimators, aside from minor differences caused by end-effects, see BNHLS-SubRK:2011.
Realized correlations are often computed from multivariate estimators, such as those proposed in MalliavinMancino:2002, BNS:2004, ChristensenKinnebrockPodolskij:2010, AitSahaliaFanXiu:2010, BNHLS:2011, and ChristensenPodolskijVetter:2013, among others. If volatility varies over the period for which estimators are computed, then the resulting estimator will be inconsistent, aside from special cases, as we detail in Section 3.
This paper is organized as follows. Section 2 reviews the benchmark correlation estimators and introduces the subsampled Quadrant estimator. In Section 3, we present the properties of the estimators, including efficiency, consistency, and robustness. Section 4 reports the results of a series of simulation studies based on the Levy and Heston model adding prevailing microstructure issues and jumps. The empirical illustrations are presented in Section 5. We extend the correlation estimation of bivariate variables to the higher dimensional correlation matrices in section 6. Section 7 concludes.
We begin by reviewing classical correlation measures, starting with the Pearson correlation.
For two random variables, $X$ and $Y$, with finite variances, the correlation coefficient is defined by \[ \rho=\frac{\sigma_{XY}}{\sqrt{\sigma_{X}^{2}\sigma_{Y}^{2}}},\qquad\text{where}\quad\sigma_{XY}=\mathrm{cov}(X,Y)=\mathbb{E}[(X-\mu_{X})(Y-\mu_{Y})], \] $\mu_{X}=\mathbb{E}(X)$, $\mu_{Y}=\mathbb{E}(Y)$, $\sigma_{X}^{2}=\mathrm{var}(X)$, and $\sigma_{Y}^{2}=\mathrm{var}(Y)$.
Nowadays, the “correlation” is commonly understood to mean $\rho=\sigma_{XY}/\sqrt{\sigma_{X}^{2}\sigma_{Y}^{2}}$, but $\rho$ is just one of several classical population measures of the correlation. Another measure is defined from sign-concordances, \[ \tau=\mathbb{E}[\mathrm{sgn}\{(X-\xi_{X})(Y-\xi_{Y})\}], \] where $\mathrm{sgn}(x)$ denotes the sign of $x$, and $\xi_{X}$ and $\xi_{Y}$ are the medians of $X$ and $Y$, respectively. The parameter $\tau$ is given from the quadrant probabilities of the recentered variables, $\tilde{X}=X-\xi_{X}$ and $\tilde{Y}=Y-\xi_{Y}$, since $\tau=\Pr[Z>0]-\Pr[Z<0]$, where $Z=\tilde{X}\tilde{Y}$, and $\tau$ is the population quantity that is estimated by the quadrant estimators we use below. For spherical and continuously distributed variables, we have $q\equiv\Pr[Z>0]=1-\Pr[Z<0]=(\tau+1)/2$, such that $\tau=2q-1$. A closely related population measure is Kendall's tau, which is given by \[ \tau_{K}=\mathbb{E}[\mathrm{sgn}\{(X_{1}-X_{2})(Y_{1}-Y_{2})\}], \] where $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ are independent and distributed as $(X,Y)$. For a continuous bivariate distribution with cdf, $F$, it can be shown that $\tau_{K}=\mathbb{E}[4\{F(X,Y)-\tfrac{1}{4}\}]$ whereas $\tau=4[F(\xi_{x},\xi_{y})-\tfrac{1}{4}]$. The two quantities, $\tau$ and $\tau_{K}$, are identical for elliptical distributions.
Other classical correlation measures include the Gaussian rank correlation and Spearman's rank correlation, where the latter estimates $\eta=12\{\mathbb{E}[F(X)G(Y)]-\tfrac{1}{4}\},$ where $F$ and $G$ are the cumulative distribution functions for $X$ and $Y$, respectively. We do not include these estimators in our comparison, because they are not competitive for various reasons discussed later in the paper.
The population measures, $\rho$, $\tau$, $\tau_{K}$, and $\eta$ are closely related and all have values ranging between $-1$ and $1$. The exact relation between these quantities depends on the bivariate distribution of $(X,Y)$. For elliptical distributions we have $\tau(\rho)=\tfrac{2}{\pi}\arcsin\rho$, such that the inverse mapping is:
This link function was derived in Greiner:1909, albeit the identity is implicit from results in Sheppard:1899, who first related quadrant probabilities to the correlation. Greiner derived the result under the assumption that $(X,Y)$ are normally distributed, but ((ref)) is valid for a broader class of distributions that includes all symmetric elliptical distributions for which the correlation is well defined, such as the multivariate $t$-distribution with degrees of freedom greater than two, see Proposition (ref). The link function is also unaffected by skewness as defined by the moments and cumulants of odd order, see Kendall:1949. The link function in ((ref)) makes it possible to translate an estimate of $\tau$ into an estimate of $\rho$. In general, the link function between $\tau$ and $\rho$ depends on actual bivariate distribution and we have shown three examples of the link function in Figure (ref).
The blue line represents Greiner's link function, ((ref)), while the green dashed and the red dotted lines represent link functions for two non-elliptical distributions. The green dashed link function, labelled “Uniform” is based on the following bivariate distribution \[ X=U_{1}-U_{2}\quad Y=\rho X+\sqrt{1-\rho^{2}}(U_{3}-U_{4}),\qquad\text{\ensuremath{\rho\in[-1,1]}} \] where $U_{1}$,..., $U_{4}$ are independent and uniformly distributed on $[0,1]$, and the red dotted link function, labelled “Exponential” is based on \[ X=aZ_{0}-(1-|a|)Z_{1}\quad Y=|a|Z_{0}+(1-|a|)Z_{2},\qquad a\in[-1,1], \] where $Z_{0},Z_{1},Z_{2}$ are independent and standard exponentially distributed. The correlation for this distribution is $\rho(a)=a|a|/[a^{2}+(1-|a|)^{2}]$. Examples of these distributions are shown in the right panels using scatterplots with 1,000 observations. The upper right panel is the bivariate normal distribution with correlation $-0.5$. The middle right panel is that based on four uniformly distributed random variables with $\rho=-0.5$, and the lower right panel is based on the exponential random variables with $a=-0.5$, which happens to translate to the same correlation, $\rho(-0.5)=-0.5$. Thus Greiner's link function is a good approximation to the two other link functions in Figure (ref), and may offers a good approximation to a broader class of distributions than the elliptical distributions. However, it is also possible to construct a bivariate distribution whose link function differs to a greater degree from that in ((ref)).\footnote{For instance, pathological examples can be created by assigning small probabilities to extreme events. carefully shifting probability mass near zero to shift the binary distribution over signs, which will have negligible without having much impact on $\rho$. }
Next, we introduce classical correlations estimators. To simplify the exposition we use $(x_{i},y_{i})$, $i=1,\ldots,n$, to denote recentered variables, such that their sample means (or sample medians) are zero.\footnote{The sample mean is subtracted before applying the Pearson estimator and the sample median is subtracted if the Quadrant or Kendall estimators are used. }
The Pearson correlation estimator is the well-know sample correlation, which takes the form
\[ P=\frac{\sum_{i=1}^{n}x_{i}y_{i}}{\sqrt{\sum_{i=1}^{n}x_{i}^{2}\sum_{i=1}^{n}y_{i}^{2}}}. \] This estimator is asymptotically efficient if the data are iid Gaussian. A drawback of the Pearson estimator is that it is sensitive to outliers, as we discuss below. More robust estimators of $\rho$ can be constructed from estimators of $\tau$, such as the quadrant estimator
\[ \hat{\tau}_{Q}=\frac{1}{n}\sum_{i=1}^{n}\mathrm{sgn}(x_{i}y_{i}), \] and Kendall's tau coefficient \[ \hat{\tau}_{K}=\frac{2}{n(n-1)}\sum_{i<j}\mathrm{sgn}([x_{i}-x_{j}][y_{i}-y_{j}]). \] Quadrant-based estimation of the correlation was introduced in Sheppard:1899, with the relation between $\rho$ and $\tau$ spelled out in Greiner:1909. The asymptotic properties of the quadrant estimator were derived in Blomqvist:1950. Esscher:1924 introduced the $\hat{\tau}_{K}$ estimator and cited Greiner:1909 for the link function. This estimator was rediscovered in Kendall:1938 and is commonly known as Kendall's tau coefficient and Kendall rank correlation coefficient. Note that $\hat{\tau}_{K}$ is the quadrant estimator applied to $\{(X_{i}-X_{j},Y_{i}-Y_{j})\}_{i<j}$, and it is easy to verify that $\rho=\mathrm{corr}(X_{1},Y_{1})=\mathrm{corr}(X_{1}-X_{2},Y_{1}-Y_{2})$, if $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ are independent and identically distribution.
In this paper, we employ Greiner's link function to map the estimators of $\tau$ to estimators of $\rho$. The estimators of $\rho$ are therefore defined by \[ Q=\sin(\tfrac{\pi}{2}\hat{\tau}_{Q})\qquad\text{and}\qquad K=\sin(\tfrac{\pi}{2}\hat{\tau}_{K}), \] respectively. A convenient feature of these two estimators, is that they bypass the need for estimating the variances of $X$ and $Y$. In fact, $Q$ and $K$ do not rely on $X$ and $Y$ having finite moments. For non-elliptical distributions, ((ref)) may not be the appropriate link function, and this type of misspecification can therefore induce a bias in these estimators. Fortunately, Greiner's link function does appear to offer a good approximation beyond the class of elliptical distributions, as illustrated in Figure (ref). In our empirical application we use sparsely sampled financial returns, which is an application where a Gaussian assumption has some theoretical justification.
Our new estimator is motivated by the empirical situation one encounters with high-frequency financial data, where market microstructure noise, jumps, and time-varying volatility pose challenges to the validity of correlation estimators. While Pearson is the ideal estimator when the variables are distributed as a bivariate Gaussian distribution, it is inconsistent under more realistic and commonly accepted assumptions for intraday returns. The $K$ estimator is more robust, but also inconsistent under time-varying volatility, while $Q$ is is very inefficient. This motivates the estimator introduced below.
Let $X(t)$ and $Y(t)$ denote the the observed logarithmically transformed price processes over some period, such as a trading day. We denote the intraday returns over a time-interval with length $\delta$ by \[ \Delta_{\delta}X_{t}=X(t)-X(t-\delta), \] and similarly for $\Delta_{\delta}Y_{t}$. In the context of high frequency data it is common to sample sparsely to mitigate the effects of market microstructure noise, and a popular choice is to set $\delta$ equal to five minutes. If we normalize the interval of time to be $[0,1]$ and set $\delta=\frac{1}{n}$, then the correlation estimators given above, may be applied to $(x_{i},y_{i})=(\Delta_{\frac{1}{n}}X_{\frac{i}{n}},\Delta_{\frac{1}{n}}Y_{\frac{i}{n}})$ for $i=1,\ldots,n$.
Let $N$ denote the number of intraday returns at the highest possible sampling frequency and suppose, for simplicity, that $N$ is divisible by $n$, such that $S=N/n\in\mathbb{N}$. Then we can create $S$ distinct grids by shifting the initial observation time to be $t_{s}=s/(Sn)$ for $s=0,\ldots,S-1$. Each grid will have sparely and non-overlapping returns, and combined we have $N-S+1$ pairs of sparsely sampled returns, $(\Delta_{\delta}X_{\frac{j}{N}},\Delta_{\delta}Y_{\frac{j}{N}})$, for $j=S,\ldots,N$.\footnote{For instance, a 6.5 hour long trading day has $n=78$ intraday returns when partitioned into $5$-minute intervals. By shifting the starting time we obtain partitions with distinct 5-minute returns, each having just 77 returns. By shifting the starting time in one-minute increments we obtain $S=5$ different partitions and a total of $386$ 5-minute returns.}
We are now ready to introduce the subsampled variant of the Quadrant estimator, defined by
\[ Q_{S}=\sin(\tfrac{\pi}{2}\hat{\tau}_{S})\qquad\text{with}\qquad\hat{\tau}_{S}=\frac{1}{N-S+1}\sum_{j=S}^{N}\mathrm{sgn}(\Delta_{\frac{S}{N}}X_{\frac{j}{N}}\Delta_{\frac{S}{N}}Y_{\frac{j}{N}}). \] The estimator does not require $N$ to be divisible by $S$, but if $N$ is divisible by $S$, then $\hat{\tau}_{S}$ can be expressed as a simple average of $S$ $\tau$-estimators based on different grids. This construction is similar to many robust estimators of the long-run variance. PolitisRomanoWolf99 noted that the subsampled sample variance is identical to the moving-blocks estimator and the jackknife variance estimator, and it is almost identical to the Bartlett estimator, \nocite{Bartlett:1950}Bartlett:1946, which is often referred to as the Newey-West estimator in the econometrics literature.\footnote{PolitisRomanoWolf99: “{[}...{]} the variance estimator $\hat{\sigma}_{\text{\textsc{sub}}}^{2}$ is actually asymptotically equivalent to the Bartlett kernel estimator{[}...{]}” and PolitisRomanoWolf99: “In addition, $\hat{\sigma}_{\text{\textsc{sub}}}^{2}$ is identical to the moving blocks bootstrap and/or jackknife variance estimator of the variance of the sample mean proposed by Kunsch:89 and LiuSingh:92 {[}...{]}”.} In the context of volatility estimation with high-frequency data, the subsampling idea was first used in Zhou:1996. The theoretical foundation for subsampled realized variances was established in zhang-mykland-aitsahalia:05 and Zhang:2006, and the close connection between subsampled estimators and kernel estimators is detailed in BNHLS-SubRK:2011.
The subsampled quadrant correlation estimator has several appealing properties. First, it inherits the robustness of the quadrant estimator while being more precise than $Q$. The robustness is characterized by the influence function, which is discussed below. Second, $Q_{S}$ is consistent under to time-varying volatilities. This is important because time-varying volatility is common in economic time series, especially in high-frequency financial data. Third, another computationally attractive feature of the new $Q_{S}$ estimator, is that it relies on binary variables. This makes it easier to scale this estimator to large data sets.
One challenge with ultra-high-frequency data is that price increments can be zero over short time intervals, resulting in $\Delta_{\delta}X\Delta_{\delta}Y=0$. This issue may be caused by stale prices and rounding to a grid defined by the minimum tick size. This issue abates quickly with sparse sampling, and zeros are infrequent in our empirical analysis once we sample a frequencies below one minute. Most of our emprical results are based on $\delta=3$-minutes. Still, we will explore the properties of the estimators at at higher sampling frequencies to gain insight about them and market microstructure noise. For this reason we need to account for zero returns, and we do so by redefining the estimator, \[ \hat{\tau}_{S}=\frac{1}{N_{1}-S+1}\sum_{j=S}^{N}\mathrm{sgn}(\Delta_{\delta}X_{\frac{j}{N}}\Delta_{\delta}Y_{\frac{j}{N}}),\qquad\text{with}\quad N_{1}=\sum_{j}1_{\{\Delta_{\delta}X_{\frac{j}{N}}\Delta_{\delta}Y_{\frac{j}{N}}\neq0\}}, \] such that we only count non-zero product-pairs.
In this section, we establish several properties of the estimators, and we highlight some of the key advantages that are unique to $Q_{S}$. We first consider the simple case with iid and normally distributed variables. This is the situation that arises when the price process are given from Brownian motions with constant volatility and the observed prices are measured without error. We then proceed with more realistic models with time-varying volatility and discuss robustness by means of the influence function of the estimators. The impact of general types of market microstructure noise will be analyzed in Section 4.
We begin with the simplest possible situation, where logarithmic price processes follow Brownian motions with constant volatilities and constant correlation.
In this situation, intraday returns, $(\Delta_{\delta}X_{i\delta},\Delta_{\delta}Y_{i\delta})$, $i=1,\dots,n$ are iid and normally distributed. This is the ideal situation for the Pearson estimator, because $P$ is the maximum likelihood estimator of $\rho$ for the sample with the $n$ pairs of observations. We should therefore expect $P$ to compare favorable to $Q$ and $K$. It is less obvious how $P$ will compare with $Q_{S}$, because the latter utilizes the shifted grids of sparsely sampled returns, $(\Delta_{\delta}X_{j\delta/S},\Delta_{\delta}Y_{j\delta/S}),j=S,\dots,N$, and is thus computed from a larger data set. The asymptotic distribution of the new estimator is given next.
The corresponding asymptotic distributions for the estimators, $Q$, $K$, and $P$, are well known, see e.g. CrouxDehon:2010. For the sake of comparison, these are included below.
We can compare the asymptotic variance of $Q_{S}$ to those of the other estimators. The asymptotic variances depend on $\rho$ and those for $P$, $K$, and $Q_{S}$ are shown in Figure (ref). With $S=1$ we obviously have $V_{Q_{S}}=V_{Q}$.\footnote{With $S=1$ we have $\sum_{s=-S}^{S}\arcsin^{2}(\tfrac{S-|s|}{S})-\arcsin^{2}(\tfrac{S-|s|}{S}\rho)=\arcsin^{2}(1)-\arcsin^{2}(\rho)=\tfrac{\pi^{2}}{4}-\arcsin^{2}\rho$ such that $V_{Q_{S}}=V_{Q}$ as expected.} For a sufficiently large $S$, the subsampled Quadrant estimator is more accurate than the Kendall estimator. For small values of $\rho$, $Q_{S}$ is more accurate than $K$ when $S\geq5$, whereas a larger value of $S$ in required for larger values of $\rho$. The $Q_{S}$ estimator is similar to $P$ for large values of $S$, with $Q_{S}$ having the edge for small values of $\rho$, whereas $P$ has the edge for large values of $\rho$.
Realized measures are commonly computed from sparsely sampled returns, such as 5-minute returns, to minimize the impact of market microstructure noise. There will typically be a large number of observations within each 5-minute interval, and this makes it possible to use a relatively large value for $S$.
Time-varying volatility is an intrinsic feature of financial time-series. For instance, volatility if found to vary substantially in high-frequency financial data, even within a trading day. Next, we relax the assumption that volatility is constant and evaluate the effect this has on the correlation estimators. The asymptotic properties of correlation estimators stated above need not apply in this context, because they were derived under constant volatility.
We can illustrate the issues that arise from time-varying volatility with a simple bivariate Brownian semimartingale.
The assumption can be generalized in many ways, such as having a random drift term,\footnote{Formally, we can let the logarithmic price process be defined on the filtered probability space $(\Omega,\mathcal{F},(\mathcal{F})_{t\in[0,1]},\mathbb{P})$ with a locally bounded predictable drift function, $a(u)$, where $a$, $\sigma$, and $W$ are adapted to a $\mathcal{F}_{t}$.} but the simple setup presented here suffices to show that traditional correlation estimators are biased in the presence of time-varying volatility, and establish that quadrant-based estimators are robust to time-varying volatility. We will, initially, take the correlation coefficient to be constant over time. The case with time varying correlation is discussed below in Section 3.3.
The important result from Theorem (ref) is that $Q_{S}$ emerges as the only consistent estimator when volatility is time-varying. Both $P$ and $K$ are generally inconsistent, except in the following special case where the two volatility processes are perfectly collinear.
The results for $P$ and $K$ in this special case are easy to verify, because perfectly collinearity implies $\lambda=1$ and that
\[ h(u,v)=\frac{c\sigma_{x}^{2}(u)+c\sigma_{x}^{2}(v)}{\sqrt{\sigma_{x}^{2}(u)+\sigma_{x}^{2}(v)}\sqrt{c^{2}(\sigma_{x}^{2}(u)+\sigma_{x}^{2}(v))}}=1, \] for all $u,v$.
We illustrate the inconsistency with a simple example. Consider the functions, $g(u)=\tfrac{1}{5}+\tfrac{4}{5}u$ and $h(u)=\frac{1}{2}(\tfrac{6}{5}+\cos(2\pi u))$, which we will use to construct volatility paths with varying degrees of collinearity. The upper left panel in Figure (ref) represents a case with low collinearity, where $\sigma_{x}(u)=g(u)$ and $\sigma_{y}(u)=h(u)$, and the upper right panel corresponds to a case with high collinearity, where $\sigma_{x}(u)=\frac{3}{10}g(u)+\frac{6}{10}h(u)$ and $\sigma_{y}(u)=h(u)$. The lower panels show the resulting bias of the correlation coefficients, $P$, $K$, and $Q$, as a function of the true correlation coefficient, $\rho$. With low collinearity, the sample correlation, $P$, has a large bias unless $\rho$ is near zero, and the bias in $K$ is about half that in $P$. These estimators, $P$ and $K$, are also biased in the example with high collinearity, but the bias is substantially smaller. The bias of these estimators are also pronounced in standard simulation design with the Heston model, as we document in Section (ref).
An important implication of the results in this subsection is that conventional estimates of correlations between assets are systematically influenced by the degree of collinearity in their volatilities.
The correlation may be time varying, as is the case for volatility. To accommodate this situation we could modify Assumption (ref) and let $\rho(u)$ be a CADLAG process. In this situation, the integrated correlation, $\rho_{\bullet}=\int_{0}^{1}\rho(u)\mathrm{d}u$, is a natural object of interest. Unfortunately, none of the correlation estimators are consistent for $\rho_{\bullet}$. For instance, $Q_{S}$ will estimate $\tilde{\rho}_{\bullet}=\sin[\int_{0}^{1}\mathrm{asin}\{\rho(u)\}\mathrm{d}u]$, and since $\mathrm{asin}(t)$ is strictly convex for $t>0$ and concave for $t<0$, it follows that $\tilde{\rho}_{\bullet}\geq\rho_{\bullet}$ if the $\rho(u)\geq0$ and $\tilde{\rho}_{\bullet}\leq\rho_{\bullet}$ if the $\rho(u)\leq0$.
One way to partially account for time-varying correlations is to apply the correlation estimators over relatively short intervals of time and aggregate these local estimates to an estimate of $\int_{0}^{1}\rho(u)\mathrm{d}u$. This approach was used in jump-robust estimation of the integrated covariance in BoudtCornelissenCroux:2012jump. In our empirical analysis we will also use local estimates of $\rho$ to assess time-variation in $\rho(u)$.
Interestingly, it is not advisable to combine the robust correlation estimator with volatility estimators for the purpose of estimating the integrated covariance, $\mathrm{IC}=\int\sigma_{xy}(u)\mathrm{d}u$, which simplifies to $\rho\int\sigma_{x}(u)\sigma_{y}(u)\mathrm{d}u$ when $\rho(u)=\rho$ for all $u$. Now, if we multiply $Q_{S}$ by consistent estimates of $\sqrt{\int\sigma_{x}^{2}(u)\mathrm{d}u}$ and $\sqrt{\int\sigma_{y}^{2}(u)\mathrm{d}u}$, we will be estimating $\frac{1}{\lambda}\mathrm{IC}$, instead of $\mathrm{IC}$. Using $K$ is not advisable either, because it leads to another incorrect limit. For this problem, localized estimators of spot volatility and spot correlation can be use, as proposed in BoudtCornelissenCroux:2012jump. In order to be robust to jumps, they combine the MedRV estimator by AndersenDobrevSchaumburg2012 and the Gaussian rank correlation. This is further explored in VanderVeredas:2016 who employ additional robust correlation estimators, as a component to estimate $\mathrm{IC}$. They also combine non-localized estimates of $\sqrt{\int\sigma_{x}^{2}(u)\mathrm{d}u}$ and $\sqrt{\int\sigma_{y}^{2}(u)\mathrm{d}u}$ with a range of correlation estimators. Some of these combinations will be inconsistent for the reason stated earlier. This may explain that VanderVeredas:2016 find the bivariate realized kernel estimator by BNHLS:2011 to be the most accurate estimator of $\mathrm{IC}$ in the absence of jumps.
The influence function can be used to measure an estimator's sensitivity to data contamination. It measures the sensitivity of a statistical functional, $R$, to data contamination in a baseline distribution, $F$, and is defined by \[ \mathrm{IF}((x_{0},y_{0}),R,F)=\lim_{\eta\searrow0}\frac{R((1-\eta)F+\eta\mathbf{\Delta}_{(x_{0},y_{0})})-R(F)}{\eta}, \] where $\mathbf{\Delta}_{(x_{0},y_{0})}$ is the Dirac measure at $(x_{0},y_{0})$. We have $R_{P}(F)=R_{Q}(F)=R_{K}(F)=\rho$ for $F=\Phi_{\rho}$, which denotes the standard bivariate normal distribution with correlation equal to $\rho$. From DevlinGnanadesikanKettenring:1975 and CrouxDehon:2010 we have their influence functions.
The important message from the influence functions is that $Q$ and $K$ have bounded influence functions whereas $P$ has an unbounded influence function. This difference motivate their labeling as robust and non-robust estimators, respectively. The unbounded influence function of $P$ makes it sensitive to outliers. It is intuitive that $Q$ and $K$ are less sensitive to outliers, since they are computed from signed variable alone. This limits the harm an outlier can cause to merely flipping the sign. The analogous results for $Q_{S}$ are qualitative very similarly, and are presented in the Supplementary Material. One way to alleviate the sensitivity that $P$ has to outliers is to use truncation estimators, which is commonly used for estimating realized variances, see e.g. Mancini:2009.
The influence function for the Spearman estimator is also bounded but can be shown to have a larger bound than $Q$ and $K$, whereas the Gaussian rank estimator has an unbounded influence function, see Rousseeuw:1984, BoudtCornelissenCroux:2012gauss, and RaymaekersRousseeuw:2021 for details and additional results on influence functions.
We compare the estimators in simulation studies that are designed to emulate the situation we encounter in our empirical analyses with high-frequency data. We generate the two logarithmic price processes, $X_{t}^{\ast}$ and $Y_{t}^{\ast}$, using the Heston model:
where $W_{x,t}$ and $B_{x,t}$ are standard Brownian motions with $\mathrm{cov}(\mathrm{d}W_{x,t},\mathrm{d}B_{xt})=\varrho_{x}dt$, and $Y_{t}^{\ast}$ is generated similarly with $\mathrm{cov}(\mathrm{d}W_{y,t},\mathrm{d}W_{y,t})=\rho\mathrm{d}t$. The model is calibrated using the simulation design in Table (ref), which was previously used in AitSahaliaFanXiu:2010. The initial values for volatility $\sigma_{x,0}^{2}$ and $\sigma_{y,0}^{2}$ are drawn from Gamma distributions, $\Gamma(2\kappa_{x}\bar{\sigma}_{x}^{2}/s_{x}^{2},s_{x}^{2}/2\kappa_{x})$ and $\Gamma(2\kappa_{y}\bar{\sigma}_{y}^{2}/s_{y}^{2},s_{y}^{2}/2\kappa_{y})$, and the price processes are initialized with $X_{0}^{\ast}=\log(100)$ and $Y_{0}^{\ast}=\log(40)$. The simulated model is a discretized version with $23,400$ increments, which translates to 1 second observations over a 6.5 hours period -- the length of a typical trading day.
We present results for two values of the true correlation, $\rho=1/4$ and $\rho=2/3$, which are typical levels of the correlation in our empirical analysis. In the Supplementary Material we present the corresponding results for $\rho=1/2$ and $\rho=3/4$.
We first consider the case where prices are observed without measurement error. This defines the limit to which we can apply subsampling. For instance, for sparsely sampled 1-minute returns we can set $S=60$. In the absence of noise, there is no need to sample sparsely, but we gain valuable insight about the the estimators by studying their properties at lower sampling frequencies.
The Heston model generates prices process with time-varying volatilities. For this reason, we should not expect $P$ and $K$ to be consistent. While $Q$ and $Q_{S}$ are consistent, they may have a bias in finite samples, because sampling error in $\hat{\tau}$ and the non-linear transformation, $\rho=\sin(\frac{\pi}{2}\tau)$, will induce a finite-sample bias in $Q$ and $Q_{S}$.
The average values of the estimators are shown in the upper panels of Figure (ref) for the case were $\rho=0.25$ and $\rho=0.6\overline{6}$. As expected, $P$ and $K$ are biased as expected, since volatility is time-varying in the Heston model, which $P$ being substantially more biased than $K$. At the highest sampling frequency, $P$ and $K$ become very accurate estimates of incorrect quantities, as defined in Theorem (ref). The $Q_{S}$ estimator is largely unbiased when returns are sampled more frequently that every minute. At slower sampling frequencies a bias begin to emerge in $Q_{S}$, which is a consequence of Jensen's inequality. The variance of $\hat{\tau}$ increases with the sampling frequency and the concavity of $\tau\mapsto\sin(\frac{\pi}{2}\tau)$ for $\tau>0$ explains the downwards bias that becomes evident at slow sampling frequencies. However the bias of $Q_{S}$ is substantially smaller than those of $K$ and $P$.
The corresponding root mean squared errors (RMSEs) are shown in the two lower panels. The new estimator has the smallest RMSE, which is driven by its ability to reduce the bias.
Next, we amend the simulation to mimic features commonly seen in empirical data. We do so, by adding different forms of market microstructure noise. Noise will influence estimators in different ways. Noise that only alters the sign of a small fraction of returns will have minute impact on the robust estimators, but could have a large impact on $P$. Rare outliers provide an example of this scenario, and can be inferred directly from the influence functions for the different estimators.
Independent noise in the price processes can induce the Epps effect. The independent noise reduces the correlation in returns and this downwards bias is increasing in the sampling frequency. We simulate independent noise as follows: \[ X_{t}=X_{t}^{\ast}+\epsilon_{xt} \] where $\epsilon_{xt}\sim iidN(0,\omega_{x}^{2})$ and similar for $Y_{t}$ with $\epsilon_{xt}$ independent of $\epsilon_{yt}$. Following similar simulation designs in this literature, see e.g. BandiRussell:2006 and BNHLS:2008, we set $\omega_{x}^{2}=\xi^{2}\sqrt{T^{-1}\sum_{i=1}^{T}\sigma_{x,i/T}^{4}}$ with $\xi^{2}=0.001$, such that variance of the noise is proportional to square root of the integrated quarticity.
In practice, high-frequency financial prices are restricted to a grid defined by their tick-size. This induces a particular type of market microstructure noise, as analyzed in DelattreJacod:1997, Horel2007, Rosenbaum:2009, ManciniGobbi:2012, Hansen:MC2015, LiMykland:2015, HansenHorelLundeArchakov, and LiZhangLi:2018. We will study this phenomenon by letting observed prices be given by \[ X_{t}=\alpha\lfloor X_{t}^{\ast}/\alpha\rfloor\qquad\text{and}\qquad Y_{t}=\alpha\lfloor Y_{t}^{\ast}/\alpha\rfloor, \] where $\alpha$ defines the coarseness of the grid.\footnote{In reality it is nominal prices, $\exp X_{t}$ and $\exp Y_{t}$ that are confined to a grid, but it makes no practical difference over trading day.} In our simulations we let the coarseness be proportional to the level of volatility, $\alpha=c\sigma$ in order to control the average number of price changes within a given period of time. The true price processes are, as before, define by ((ref)).
Next we add noise to the grid of observed prices. Specifically we now observe \[ X_{t}=\alpha\lfloor X_{t}^{\ast}/\alpha\rfloor+\epsilon_{xt},\qquadwith\quad\epsilon_{xt}=
\] and similarly for $Y_{t}$ with $\epsilon_{xt}$ and $\epsilon_{yt}$ independent.
We introduce stale pricing using
This will generate “flat pricing” and the expected duration between price updates will be $(1-q)^{-1}/N$.
Jumps are prevalent in high-frequency prices, and we could generate such with
\[ X_{t}=X_{t}^{\ast}+\sum_{s\leq t}J_{s}^{x}\qquad Y_{t}=Y_{t}^{\ast}+\sum_{s\leq t}J_{s}^{y}, \] where $J_{t}^{x}$ and $J_{t}^{y}$ denote jump processes. The impact that jumps have on the estimators is characterized by their influence functions. The robust estimators, $Q_{S}$ and $K$, are essentially unaffected by jumps, whereas $P$ is highly sensitive. Independent jumps will cause $P$ to be biased towards zero, whereas a co-jump (a simultaneous jump in both series) will bias $P$ towards $-1$ or $1$, depending on the sign of $J_{s}^{x}J_{s}^{y}$. Co-jumps in the same direction will cause $P$ to be biased towards one, whereas co-jumps in the opposite direction will cause $P$ to be biased towards $-1$. Jumps can be alleviated by truncation methods, see Mancini:2001 and AndersenDobrevSchaumburg2012.\nocite{Mancini:2009} Simulation results with jumps are presented in the Supplementary Material.
The bias that different types of noise induce on the correlation estimators are show in Figure (ref). The true correlation is $\rho=1/4$ in the left panels and $\rho=2/3$ in the right panels. Results for additional levels of correlation and types of noise are presented in the supplementary material. Panel (a) in Figure (ref) presents the results when the efficient prices are contaminated with independent Gaussian noise with a variance that is about $10^{-3}$ times the square root of integrated quarticity of the two series. Independent noise is one (of several ways) to bring about the Epps effect. The independent noise reduces the correlation between returns, which induces a downwards bias that increases with sampling frequency, to an extend that all estimators essentially becomes noisy estimates of zero when computed with 1-second intraday returns. Independent noise is a good stating point for studying estimators, but there is overwhelming empirical evidence that contradicts the independent noise assumption in high frequencies data, see HansenLunde:JBES2006, which is also the case in our empirical analysis.
The correlation signature plots in our empirical analysis resemble those in Panel (b) of Figure (ref), where the noise is defined by a rounding error ($\alpha=10^{-4}$), to resemble the tick size in prices. Interestingly, the rounding error causes $Q_{S}$ to be upwards bias a higher sampling frequencies. This is also true for $K$, but to a much lesser extend, whereas $P$ is largely unaffected, but maintains the downwards bias caused by time-varying volatilities. In Figure (ref) (c) we consider the same level of rounding error ($\alpha=10^{-4}$) and add additional noise by shifting the price up or down by one tick size with equal probability, $p/2$ with $p=0.75$. This induces a downwards bias, which is most pronounced a high sampling frequencies. Finally, in Figure (ref) (d) we add additional staleness to prices on the grid, as defined by ((ref)), where one price series remains stale with probability $q_{x}=0.5$ and the other series remains stale with probability $q_{y}=0.8$. The combined impact of rounding and staleness is a sizable downwards bias.
The corresponding root mean squares errors (RMSEs) are reported in Figure (ref). The new correlation estimator, $Q_{S}$, tends to have the smallest RMSE, which is also true for the additional simulation experiments presented in the Supplementary Material.
We apply the correlation estimators to high-frequency data for about 100 assets. We begin by analyzing and comparing their daily correlation estimates. For instance, we use correlation signature plots to study market microstructure noise, and explore how sensitive the estimators are to the choice of sampling frequency, as defined by $\delta$. Then we turn to estimation of intraday correlations, which we find to vary substantially over the hours with active trading. Correlations between stocks and the market are, on average, increasing for all assets in our sample. We obtain estimates of intraday betas, by combining the correlation estimates with estimates of relative volatility. We then proceed to related intraday variation in correlations and betas with asset characteristics, such as low frequency based market beta, market capitalization, and book-to-market valuations. This part of the analysis is done with an expanded set of assets detailed below.
Our sample period covers the period from January 1, 2015 to December 31, 2021 and includes 1,763 trading days. We use NYSE and NASDAQ transaction prices from the TAQ database that were accessed through the Wharton Research Data Services (WRDS) system. The data were cleaned following the guidelines in BNHLS:2011, and prices (when unavailable) were interpolated by the previous-tick methods. We will analyzed 22 stocks and SPY, an exchange traded fund that tracks the S&P 500 index, in great details. We label this data set “Small Universe”. The 22 stocks were selected to be the two largest stocks (by market capitalization) within each of the eleven GICS\footnote{Global Industry Classification Standard.} sectors. A larger set of asset of assets, “Large Universe” is used to identify asset-characteristics associated with different patterns in intraday market betas. The Large universe includes the assets in the S&P 100 index, as of {[}date{]}, we excluded two of these assets from the Large Universe. PYPL (PayPal) was excluded because it only started trading in 2015 after being spun off eBay, and RTX (formerly Raytheon Tech) was excluded because it merged with United Technologies, which was completed in April 2020.
Table (ref) presents the summary statistics for the Small Universe with 22 assets. The exchange traded fund, SPY, is the most frequently traded asset, followed by AAPL and FB. On average, these securities have just over 2 seconds between transaction prices. The price range is an interesting statistic, because the tick-size is more likely to induce rounding errors and price staleness for assets trading at low prices. This appears to be relevant for AMD that traded for less than \$3 in all of 2015 and below \$10 during most of the first three years in our sample period. This likely explains the many zero increments. More than 19% of all 3-minute returns are zero in this sample period.
The assets in the Large Universe are listed and organized by sectors in Table (ref).
We apply the correlation estimators to daily high frequency data using calendar-time sampling with frequencies ranging from 1 second to 15 minutes. The resulting correlation signature plots are shown in Figure (ref) for a subset of the assets. These are the two most actively traded securities, SPY and AAPL, the stock with most zero returns, AMD, and the two stocks from the Material sector, LYB and NEM, whose liquidity and percentage of zero returns is more typical for assets in the Small Universe. Signature plots were introduced in andersen-bollerslev-diebold-labys:00a who plotted the average realized variance against the sampling frequency used to compute the underlying intraday returns. Signature plots help identify bias in the estimators, which tend to be most pronounced at high sampling frequencies. If the estimator is unbiased over a range of sampling frequencies, then the signature plot will be roughly flat over that those sampling frequencies.
The signature plots in Figure (ref) are signature plot for correlations, which can be used to visualize biases, such as the Epps effect. Here we observe that many of the plots have patterns that resemble the effect for rounding to a grid, because $Q_{S}$ often has an upwards bias at high sampling frequencies, while $P$ has a downwards biased. Additional signature plots are presented in the Supplementary Material, see Figure (ref). We adopt 3-minutes as a common sampling frequency for all estimators. This is in part motivated by the signature plots tend to be flat for $\delta\geq$3 minutes, and in part because it makes our results more comparable to those in ATT.
Ideally, one would determine an empirical way to select an optimal sampling frequency, because the optimal sampling frequency likely varies over time and across assets. We leave this for future research.
Table (ref) presents summary statistics for correlations between stocks in the same sector. For each of the three estimators, we compute the average, median, and interquartile range across the 1,763 daily estimates. For all pairs, these quantities are similar for the three estimators. The interquartile range is across days in the sample that predominately is driven by time-variation in the daily correlation. So, a similar width for the interquartile range should not be interpreted as the estimators having similar precision. In the next subsection, we present results that strongly indicate that $Q_{S}$ is more precise than $K$ an $P$. While the measurements are similar for the three estimators, we always have $Q_{S}>K>P$. This ordering is in line with our theoretical results, that time-varying volatility induces a bias in $P$ than in $K$, and that $P$ is more biased than $K$.
Next, we estimate daily correlations between each of the 22 stocks and SPY. The average, median, and interquartile range (over the 1,763) estimates are shown in Figure (ref). Once again we see that the quantities are similar for the three estimators, as was the case in Table (ref), and once again do we have $Q_{S}>K>P$ uniformly across all assets and across all measurements.
Estimating betas from high frequency data is an active research area, see e.g. AndersenBollerslevDieboldWu:2005\nocite{ABDW:2006}, TodorovBollerslev:2010, DovononGoncalvesMeddahi:2013, HansenLundeVoev:2014, and ReiBTodorovTauchen:2015. This literature as also documented substantial time variation in the betas over time. Recently, AndersenThyrsgaardTodorov:2021 (ATT) documented systematic time-variation in betas within the trading day. Specifically, they estimated betas for rolling windows (spanning two hours) using 3-minute intraday returns. Their local estimate of beta is simply a local estimator of the covariance between asset and market returns divided by a local estimates of the quadratic variation of market returns. The local (time-of-the day) estimates are averaged over the 2,243 days in their sample period (2010-2018). Interestingly, ATT found a great deal of variation in the betas within the day. Some stocks have increasing betas over the trading day while other other assets had decreasing betas over the day.
We can use correlation estimators to cast new light on the patterns in intraday market betas, by decomposing the market beta into correlation multiplied by relative volatility, \[ \beta_{i,t}=\rho_{i,t}\times\lambda_{i,t},\qquad\lambda_{i,t}=\frac{\sigma_{i,t}}{\sigma_{0,t}}, \] where $\rho_{i,t}$ is the correlation between the $i$-th asset and the market and $\sigma_{i,t}$ and $\sigma_{0,t}$ are the volatilities for the $i$-th asset and the market, respectively. We will estimated local market betas using local correlation estimators combined with estimators of relative volatility. Specifically, we compute $P$, $K$, and $Q_{S}$ and $\lambda$ using a rolling window with 60 minutes of high frequency data. We will the investigate how much of the intraday variation in betas is explained by into intraday variation in correlations and how much can be ascribed to intraday variation in relative volatility.
We estimate $\rho_{i,t}$ with each of the correlation estimators, $P,$ $K$, and $Q_{S}$, using a rolling window that spans 60 minutes. Similarly, we estimate the relative volatility, $\lambda_{i,t}$, with subsampled range-based estimators with truncations, as defined by \[ \hat{\lambda}_{i,t}=\frac{\sum_{j\in I_{t}}\left\llbracket \Delta_{\frac{S}{N}}Y_{\frac{j}{N}}\right\rrbracket _{\nu_{y}}}{\sum_{j\in I_{t}}\left\llbracket \Delta_{\frac{S}{N}}X_{\frac{j}{N}}\right\rrbracket _{\nu_{x}}},\qquad\left\llbracket x\right\rrbracket _{\nu}=
\qquad I_{t}=[\lfloor tN\rfloor-W+S,\lfloor tN\rfloor], \] where $\nu_{x}$ and $\nu_{y}$ are adaptive thresholds for jump truncation. These are defined by $\nu_{x}=4\sqrt{\mathrm{BV}_{x}}/n^{0.49}$ and $\nu_{y}=4\sqrt{\mathrm{BV}_{y}}/n^{0.49}$, where $\mathrm{BV}_{x}=\frac{\pi}{2}\sum_{j=2}^{n}|\Delta_{\delta}X_{j\delta}||\Delta_{\delta}X_{(j-1)\delta}|$ is the jump robust bipower variation estimator of daily integrated volatility, see barndorff-shephard:2004BiPower, and $\mathrm{BV}_{y}$ is defined analogously. We have explored estimation of relative volatility using the bipower variation measures, with the same thresholds. These estimated we virtually identical to those of $\hat{\lambda}_{i,t}$.
In our implementation, we have $N=23,400$, $W=3,600$, and $S=180$. This results in 3,421 overlapping 3-minute returns within each hour we use to compute the subsampled quantities.
Intraday estimates of correlations, relative volatilities, and betas are show for four assets in Figure (ref). All quantities are estimated using a rolling window that spans 60 minutes return. The time-stamp used along the x-axis refers to the end of the 60 minute period. The estimates are averaged over the 1,763 trading days in the sample. The left panels report the intraday correlation estimates for $P$, $K$, and $Q_{S}$. A horizontal dashed line indicate the average correlations over the trading hours. The middle panels report the relative volatility as defined by $\hat{\lambda}_{i,t}$ above, which is multiplied by the three correlation estimates to obtain estimates of market betas. These three intraday market betas are shown in the right panels along with the regression based estimate, based on the same methodology as ATT. The corresponding results for all assets in the Small Universe is presented in the Supplementary Material, Figure (ref).
We note that the time variation in the estimates of $Q_{S}$ tends to be smoother than those of $K$ and $P$. This strongly suggests that $Q_{S}$ is a more accurate than $K$ and $P$. We also note that $Q_{S}$ tends to be slightly larger than $K$ and $P$, which may be related to them having a larger variance causing another source of bias in $K$ and $P$. These smoother lines for $Q_{S}$ and slightly larger values carries over to the intraday estimates of market betas. The lines for the regression based estimates of intraday betas are also less smooth that those based on $Q_{S}$.
We find correlations to be generally increasing over the day, while relative volatilities are decreasing. Whether their product, the market beta, is increasing or decreasing will depend on which of the terms changes the most. Unlike correlations and relative volatility, the paths for intraday market betas take many different shapes. Some assets have clearly increasing market beta over the day (e.g. NEM), others have decreasing market betas (e.g. AMD), and a third group of assets have market betas that goes both up and down (e.g. AAPL), or stay relatively flat for a large part of the day (e.g. LYB). It is interesting to compare the market betas for LYB and NEM, which are both Materials sector stocks, with trading intensity below the average for stocks in the Small Universe. Despite these commonalities the intraday beta patterns for LYB and NEM are very different. The reason can be found in their intraday correlations. For LYB the correlation only increases by about 50% over the trading hours (from about 0.37 to 0.55), whereas the correlation for NEM increases by nearly 500% (from about 0.05 to 0.25). A great variety of shapes for time-varying betas are shown in Figure (ref) in the Supplementary Material.
All estimated correlations are positive, we can therefore factorize the logarithm of intraday market betas, as \[ \log\beta_{i,t}=\log\rho_{i,t}+\log\lambda_{i,t}. \] We use this decomposition to investigate who much of the intraday variation in market betas can be ascribed to changes in correlations and changes in relative volatilities. For this purpose, we expand this part of our analysis to include all assets in the Large Universe.
In Figure (ref) we have plotted changes in intraday market betas against intraday changes in intraday correlations and against changes in relative volatility, for all assets in the Large Universe. We use color codes to indicate the GICS industry sectors for each of the assets. The increments (changes) are defined by the logarithmic difference between the estimate from the first hour of trading and the analogous quantity from the last hour of trading.
When changes in intraday betas are plotted against intraday changes in correlations, it reveals a strong linear relationship between the two, see left panel of Figure (ref). In contrast there is only a weak relationship between changes in market betas and changes in relatively volatilities. In fact, as can be seen from the range of the x-axis in the right panel of Figure (ref), there is far less cross-sectional variation in the changes in relative volatility. Overall, we can see that most of the cross sectional variation in market betas can be ascribed to variation in intraday correlations.
Interestingly, there is a great deal of clustering by sectors in Figure (ref). In terms of changes in intraday correlations, there is a large degree of sector-specific separation, whereas in terms of changes in relative volatilities there are notable variation within all sectors, as evident by asset dispersion along the x-axis.
In the Supplementary material, Figure (ref), we have explored the intraday variation in greater details. For instance, we plot changes in intraday market correlations, relative volatility, and market betas against Fama-French type variables. We do not detect a strong association with key characteristics such as market capitalization and book-to-market ratios.
In Figure (ref), we present scatterplots of the intraday market betas against conventional market betas, which are computed from daily returns. The left panel has the market beta for the first hour of trading and the right panel has the market beta for the last trading hour plotted against the low frequency market beta. Not surprisingly, do we find a strong relationship between the low-frequency market betas and the intraday market betas. The scatterplots in Figure (ref) corroborates findings in ATT, who found market betas to be less disperse at the end of the day, than the beginning of the day.
The correlation coefficient is a fundamental measure of linear dependence with broad applications across various fields of empirical analysis. For instance, in modern finance, it has a central role in risk management, portfolio selection, and the pricing of derivatives.
In this paper, we have introduced a novel robust correlation estimator that is particularly well-suited for high-frequency financial data analysis. We have shown that the sample correlation, $P$, and Kendall's tau, $K$, are inconsistent under time-varying volatility, while the quadrant estimator is robust to time-varying volatilities. The subsampled quadrant estimator, $Q_{S}$, inherits the consistency of the quadrant and is far more efficient, because it leverages additional high-frequency data. The theoretical properties we established for the estimators are supported by simulation-based evidence and an extensive empirical analysis spanning seven years of high-frequency return data for about 100 securities.
The empirical analysis also offers valuable insights into the time-varying nature of market betas within a trading day. Market betas can be expressed as the product of the correlation (with market returns) and relative volatility. We have documented that the time-variation in market betas within the day is mainly driven by time-variation in intra-day correlations.
While the estimator, $Q_{S}$, is particularly well-suited for high-frequency financial data analysis, it may also be useful for other time-series with time-varying volatility, or time series that are prone to outliers and noise. The $Q_{S}$ estimator might also be useful for nonparametric estimation of the leverage leverage effect, as analyzed in KalninaXiu:2017. There are several ways the $Q_{S}$ estimator could be extended and possibly improved. For instance, there might be more efficient ways to handle zero returns, such as distinguishing between cases were both returns are zero and cases were just one of the returns is zero. A multivariate version of the $Q_{S}$ estimator would be interesting to explore. Constructing a correlation matrix from univariate correlation estimates, need not result in a positive definite matrix. So, a subsequent matrix projection to the set of positive definite correlation matrices might be needed, such as those proposed by Higham:2002 and QiSun:2006.