The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
86,041 characters
Robust Estimation of Realized Correlation: New Insight about Intraday Fluctuations in Market Betas
\title{Robust Estimation of Realized Correlation:\linebreak{}
New Insight about Intraday Fluctuations in Market Betas\emph{\normalsize{}\medskip{}
}}
\author{\textbf{Peter Reinhard Hansen}{\normalsize{}$^{a}$}\textbf{ and Yiyao
Luo$^{b}$}\bigskip{}
\\
{\normalsize{}$^{a}$}\emph{\normalsize{}University of North Carolina
at Chapel Hill}\textbf{ }\\
{\normalsize{}$^{b}$}\emph{\normalsize{}University of Mississippi}\textbf{}\thanks{Corresponding author: Yiyao Luo. Email: [email removed] Mailing
Address: University of Mississippi, Department of Economics, Oxford,
MS, 38677, USA. We are grateful for helpful comments from Ron Gallant
and seminar participants at UNC, Duke University, and the 2023 Annual
Meeting of SoFiE in Seoul, Korea.}}
\date{\emph{\normalsize{}\today}}
\maketitle
\begin{abstract}
Time-varying volatility is an inherent feature of most economic time-series,
which causes standard correlation estimators to be inconsistent. The
quadrant correlation estimator is consistent but very inefficient.
We propose a novel subsampled quadrant estimator that improves efficiency
while preserving consistency and robustness. This estimator is particularly
well-suited for high-frequency financial data and we apply it to a
large panel of US stocks. Our empirical analysis sheds new light on
intra-day fluctuations in market betas by decomposing them into time-varying
correlations and relative volatility changes. Our results show that
intraday variation in betas is primarily driven by intraday variation
in correlations.
\bigskip{}
\end{abstract}
\textit{\small{}Keywords:}{\small{} Correlation, Pearson, Kendall,
Subsampling, Robustness, Consistency, Epps effect, High-frequency
data, Microstructure, Jump}{\small\par}
\noindent {\small{}\newpage}{\small\par}
\section{Introduction}
The correlation is a measure of association between two variables
that plays a central role in many empirical methods. The correlation
is most commonly estimated by the sample correlation, which is known
as Pearson's $r$. Other classical correlation estimators include
the Quadrant estimator, the Kendall's tau, Spearman's rank correlation,
and the Gaussian rank correlation estimator, see \citet{Kruskal:1958}
for the relationships between these measures and an historical account
of their developments. The choice of estimator involves a tradeoffs
between robustness and efficiency. This tradeoff is influenced by
many features of the underlying distribution, including heteroskedasticity
that is particularly important for many economic applications.
In this paper, we propose a new robust correlation estimator that
is well-suited for heteroskedastic time-series, such as high-frequency
financial data. Time-varying volatility and market microstructure
noise are innate features of high-frequency financial data, and both
features undermine the reliability of standard correlation estimators.
We compare the sensitivity of correlation estimators to departures
from homoskedasticity and show that the Quadrant estimator is the
only estimator that is robust to heteroskedasticity, among the classical
estimators. The other estimators are inconsistent, except in very
special cases. Unfortunately, the Quadrant estimator is rather inefficient.
We recover much efficiency by combining the Quadrant estimator with
subsampling and this makes it possible to improve efficiency while
retaining consistency. We derive theoretical properties of the new
estimator and study it using simulation designs that mimic empirical
high-frequency financial data. We show that the realized correlation
can be very biased as documented in our empirical analysis. We apply
the new estimator to high-frequency data for 22 assets and an exchange-traded
fund that tracks the S\&P 500 index. The empirical results suggest
that the new estimator is more accurate than other estimators, with
the improvements likely resulting from better bias properties. We
combine intraday correlation estimates with estimates of relative
volatility to form an estimate of intraday market beta, as analyzed
in \citet{AndersenThyrsgaardTodorov:2021}. We find substantial variation
in market betas within the trading day, with some stocks having increasing
betas over the trading hours, while others tend to have decreasing
betas. Our empirical results corroborate the finding in \citet{AndersenThyrsgaardTodorov:2021},
even though we use different estimation methods and a different (narrower)
estimation window. Our estimation approach enables us to decompose
the time variation in betas into time variation in correlation and
time-variation in relative volatility. Interestingly, we find that
the variation in betas is mainly driven by time-variation in correlations.
Relative to the market, all assets in our analysis have increasing
correlations and decreasing relative volatilities over the trading
hour. The declines in relative volatilities are very similar across
assets. The relative volatility during the last hour of active trading
is typically between 50\%-75\% of relative volatility during the first
hour of trading. There is far more variation across assets in terms
of their correlations with the market. For many assets their market
correlation is 2-5 times larger during the last hour than during the
first hour. These assets have nearly linearly increasing market betas
during the trading hours. Another set of assets, which are characterized
by high market correlations, have their correlations increase by much
less than 100\% during the day. These assets have, on average, decreasing
market betas during the trading hours. Thus, we document that intraday
variation in both correlation and relative volatility contribute to
the variation in market betas, but the variation across assets is
primarily driven their time-variation in correlations with the market.
Time-varying volatility in high-frequency financial data is well documented,
see e.g. \citet{AndersenBollserslev:1998b}. Similarly, it is well
documented that market microstructure noise can harm realized measures
of volatility, see \citet[1998]{Zhou:1996}\nocite{Zhou:1998}, \citet{zhang-mykland-aitsahalia:05},
\citet{BandiRussell:2006}, and \citet{HansenLunde:JBES2006}. Market
microstructure noise is defined as the difference between the observed
prices and true prices. The latter are characterized by having certain
martingale properties, whereas the former typically entails some degree
of predictability. Market microstructure noise arises from many intricate
aspects of high-frequency data. For instance, noise can arise as artifacts
of imputation methods and recording and rounding errors. These issues
are all important for correlation estimation, see e.g. \citet{Reno2003},
\citet{PrecupIori:2007}, and \citet{MunnixSchaeferGuhr:2011}, and
\citet[2009]{TothKertesz:2007}.\nocite{TothKertesz:2009} The lack
of synchronicity in observation times induces a type of noise that
is particularly important for covariance and correlation estimation.
This will often manifest as the Epps effect, where the sample correlation
decreases as the sampling frequency increases, see \citet{Epps:1979}.
\citet{HayashiYoshida:2005} proposed an estimator that adjusts for
asynchronicity and \citet{VoevLunde:2007} and \citet{GriffinOomen:2011}
proposed related estimators that are robust to additional forms of
noise. Jumps in prices and adversely effect empirical measures, including
realized variances, covariances, and correlations. However, these
effects can be alleviated by truncation methods, see \citet{Mancini:2009}
and \citet{RaymaekersRousseeuw:2021}.
A standard remedy for market microstructure noise in high-frequency
data is sparse sampling. \citet{AndersenBollerslev:1998a} estimated
realized variances using $5$-minute intraday returns and this sampling
frequency appears to offer a reasonably good compromise between bias
and variance in many applications, see e.g. \citet{HansenLunde:JBES2006},
\citet{BandiRussell:2008}, and \citet{LiuPattonSheppard:2015}. Realized
measures that utilize more information include the subsampled realized
variance by \citet{zhang-mykland-aitsahalia:05}, the realized kernel
estimator by \citet{BNHLS:2008}, and the pre-averaging estimators
by \citet{JacodLiMyklandPodolskijVetterPreaverage}. These three approaches,
subsampling, realized kernels, and pre-averaging, lead to the same
class of estimators, aside from minor differences caused by end-effects,
see \citet{BNHLS-SubRK:2011}.
Realized correlations are often computed from multivariate estimators,
such as those proposed in \citet{MalliavinMancino:2002}, \citet{BNS:2004},
\citet{ChristensenKinnebrockPodolskij:2010}, \citet{AitSahaliaFanXiu:2010},
\citet{BNHLS:2011}, and \citet{ChristensenPodolskijVetter:2013},
among others. If volatility varies over the period for which estimators
are computed, then the resulting estimator will be inconsistent, aside
from special cases, as we detail in Section 3.
\subsection{Organization of Paper}
This paper is organized as follows. Section 2 reviews the benchmark
correlation estimators and introduces the subsampled Quadrant estimator.
In Section 3, we present the properties of the estimators, including
efficiency, consistency, and robustness. Section 4 reports the results
of a series of simulation studies based on the Levy and Heston model
adding prevailing microstructure issues and jumps. The empirical illustrations
are presented in Section 5. We extend the correlation estimation of
bivariate variables to the higher dimensional correlation matrices
in section 6. Section 7 concludes.
\section{Population and Empirical Measures of Correlation}
We begin by reviewing classical correlation measures, starting with
the Pearson correlation.
\subsection{Population Measures}
For two random variables, $X$ and $Y$, with finite variances, the
correlation coefficient is defined by
\[
\rho=\frac{\sigma_{XY}}{\sqrt{\sigma_{X}^{2}\sigma_{Y}^{2}}},\qquad\text{where}\quad\sigma_{XY}=\mathrm{cov}(X,Y)=\mathbb{E}[(X-\mu_{X})(Y-\mu_{Y})],
\]
$\mu_{X}=\mathbb{E}(X)$, $\mu_{Y}=\mathbb{E}(Y)$, $\sigma_{X}^{2}=\mathrm{var}(X)$,
and $\sigma_{Y}^{2}=\mathrm{var}(Y)$.
Nowadays, the ``correlation'' is commonly understood to mean $\rho=\sigma_{XY}/\sqrt{\sigma_{X}^{2}\sigma_{Y}^{2}}$,
but $\rho$ is just one of several classical population measures of
the correlation. Another measure is defined from sign-concordances,
\[
\tau=\mathbb{E}[\mathrm{sgn}\{(X-\xi_{X})(Y-\xi_{Y})\}],
\]
where $\mathrm{sgn}(x)$ denotes the sign of $x$, and $\xi_{X}$
and $\xi_{Y}$ are the medians of $X$ and $Y$, respectively. The
parameter $\tau$ is given from the quadrant probabilities of the
recentered variables, $\tilde{X}=X-\xi_{X}$ and $\tilde{Y}=Y-\xi_{Y}$,
since $\tau=\Pr[Z>0]-\Pr[Z<0]$, where $Z=\tilde{X}\tilde{Y}$, and
$\tau$ is the population quantity that is estimated by the quadrant
estimators we use below. For spherical and continuously distributed
variables, we have $q\equiv\Pr[Z>0]=1-\Pr[Z<0]=(\tau+1)/2$, such
that $\tau=2q-1$. A closely related population measure is Kendall's
tau, which is given by
\[
\tau_{K}=\mathbb{E}[\mathrm{sgn}\{(X_{1}-X_{2})(Y_{1}-Y_{2})\}],
\]
where $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ are independent and distributed
as $(X,Y)$. For a continuous bivariate distribution with cdf, $F$,
it can be shown that $\tau_{K}=\mathbb{E}[4\{F(X,Y)-\tfrac{1}{4}\}]$
whereas $\tau=4[F(\xi_{x},\xi_{y})-\tfrac{1}{4}]$. The two quantities,
$\tau$ and $\tau_{K}$, are identical for elliptical distributions.
Other classical correlation measures include the Gaussian rank correlation
and Spearman's rank correlation, where the latter estimates $\eta=12\{\mathbb{E}[F(X)G(Y)]-\tfrac{1}{4}\},$
where $F$ and $G$ are the cumulative distribution functions for
$X$ and $Y$, respectively. We do not include these estimators in
our comparison, because they are not competitive for various reasons
discussed later in the paper.
The population measures, $\rho$, $\tau$, $\tau_{K}$, and $\eta$
are closely related and all have values ranging between $-1$ and
$1$. The exact relation between these quantities depends on the bivariate
distribution of $(X,Y)$. For elliptical distributions we have $\tau(\rho)=\tfrac{2}{\pi}\arcsin\rho$,
such that the inverse mapping is:
\begin{equation}
\rho=\sin(\tfrac{\pi}{2}\tau).\label{eq:Greiner}
\end{equation}
This link function was derived in \citet[p.236]{Greiner:1909}, albeit
the identity is implicit from results in \citet{Sheppard:1899}, who
first related quadrant probabilities to the correlation. Greiner derived
the result under the assumption that $(X,Y)$ are normally distributed,
but (\ref{eq:Greiner}) is valid for a broader class of distributions
that includes all symmetric elliptical distributions for which the
correlation is well defined, such as the multivariate $t$-distribution
with degrees of freedom greater than two, see Proposition \ref{prop:Greiner}.
The link function is also unaffected by skewness as defined by the
moments and cumulants of odd order, see \citet{Kendall:1949}. The
link function in (\ref{eq:Greiner}) makes it possible to translate
an estimate of $\tau$ into an estimate of $\rho$. In general, the
link function between $\tau$ and $\rho$ depends on actual bivariate
distribution and we have shown three examples of the link function
in Figure \ref{fig:Greiner's-link-function}.
\begin{figure}[!h]
\centering{}\includegraphics[width=0.54\textwidth]{figures/Tau2Rho.pdf}\includegraphics[width=0.18\textwidth]{figures/BivariateT.png}\caption{The mapping from $\tau=\mathbb{E}[\mathrm{sgn}(X-\xi_{x})(Y-\xi_{y})]$
to $\rho=\mathrm{corr}(X,Y)$ for three bivariate distributions.\label{fig:Greiner's-link-function}}
\end{figure}
The blue line represents Greiner's link function, (\ref{eq:Greiner}),
while the green dashed and the red dotted lines represent link functions
for two non-elliptical distributions. The green dashed link function,
labelled ``Uniform'' is based on the following bivariate distribution
\[
X=U_{1}-U_{2}\quad Y=\rho X+\sqrt{1-\rho^{2}}(U_{3}-U_{4}),\qquad\text{\ensuremath{\rho\in[-1,1]}}
\]
where $U_{1}$,..., $U_{4}$ are independent and uniformly distributed
on $[0,1]$, and the red dotted link function, labelled ``Exponential''
is based on
\[
X=aZ_{0}-(1-|a|)Z_{1}\quad Y=|a|Z_{0}+(1-|a|)Z_{2},\qquad a\in[-1,1],
\]
where $Z_{0},Z_{1},Z_{2}$ are independent and standard exponentially
distributed. The correlation for this distribution is $\rho(a)=a|a|/[a^{2}+(1-|a|)^{2}]$.
Examples of these distributions are shown in the right panels using
scatterplots with 1,000 observations. The upper right panel is the
bivariate normal distribution with correlation $-0.5$. The middle
right panel is that based on four uniformly distributed random variables
with $\rho=-0.5$, and the lower right panel is based on the exponential
random variables with $a=-0.5$, which happens to translate to the
same correlation, $\rho(-0.5)=-0.5$. Thus Greiner's link function
is a good approximation to the two other link functions in Figure
\ref{fig:Greiner's-link-function}, and may offers a good approximation
to a broader class of distributions than the elliptical distributions.
However, it is also possible to construct a bivariate distribution
whose link function differs to a greater degree from that in (\ref{eq:Greiner}).\footnote{For instance, pathological examples can be created by assigning small
probabilities to extreme events. carefully shifting probability mass
near zero to shift the binary distribution over signs, which will
have negligible without having much impact on $\rho$. }
\subsection{Classical Correlation Estimators: Pearson, Quadrant, and Kendall}
Next, we introduce classical correlations estimators. To simplify
the exposition we use $(x_{i},y_{i})$, $i=1,\ldots,n$, to denote
recentered variables, such that their sample means (or sample medians)
are zero.\footnote{The sample mean is subtracted before applying the Pearson estimator
and the sample median is subtracted if the Quadrant or Kendall estimators
are used. }
The Pearson correlation estimator is the well-know sample correlation,
which takes the form
\[
P=\frac{\sum_{i=1}^{n}x_{i}y_{i}}{\sqrt{\sum_{i=1}^{n}x_{i}^{2}\sum_{i=1}^{n}y_{i}^{2}}}.
\]
This estimator is asymptotically efficient if the data are iid Gaussian.
A drawback of the Pearson estimator is that it is sensitive to outliers,
as we discuss below. More robust estimators of $\rho$ can be constructed
from estimators of $\tau$, such as the quadrant estimator
\[
\hat{\tau}_{Q}=\frac{1}{n}\sum_{i=1}^{n}\mathrm{sgn}(x_{i}y_{i}),
\]
and Kendall's tau coefficient
\[
\hat{\tau}_{K}=\frac{2}{n(n-1)}\sum_{i<j}\mathrm{sgn}([x_{i}-x_{j}][y_{i}-y_{j}]).
\]
Quadrant-based estimation of the correlation was introduced in \citet{Sheppard:1899},
with the relation between $\rho$ and $\tau$ spelled out in \citet{Greiner:1909}.
The asymptotic properties of the quadrant estimator were derived in
\citet{Blomqvist:1950}. \citet{Esscher:1924} introduced the $\hat{\tau}_{K}$
estimator and cited \citet{Greiner:1909} for the link function. This
estimator was rediscovered in \citet{Kendall:1938} and is commonly
known as Kendall's tau coefficient and Kendall rank correlation coefficient.
Note that $\hat{\tau}_{K}$ is the quadrant estimator applied to $\{(X_{i}-X_{j},Y_{i}-Y_{j})\}_{i<j}$,
and it is easy to verify that $\rho=\mathrm{corr}(X_{1},Y_{1})=\mathrm{corr}(X_{1}-X_{2},Y_{1}-Y_{2})$,
if $(X_{1},Y_{1})$ and $(X_{2},Y_{2})$ are independent and identically
distribution.
In this paper, we employ Greiner's link function to map the estimators
of $\tau$ to estimators of $\rho$. The estimators of $\rho$ are
therefore defined by
\[
Q=\sin(\tfrac{\pi}{2}\hat{\tau}_{Q})\qquad\text{and}\qquad K=\sin(\tfrac{\pi}{2}\hat{\tau}_{K}),
\]
respectively. A convenient feature of these two estimators, is that
they bypass the need for estimating the variances of $X$ and $Y$.
In fact, $Q$ and $K$ do not rely on $X$ and $Y$ having finite
moments. For non-elliptical distributions, (\ref{eq:Greiner}) may
not be the appropriate link function, and this type of misspecification
can therefore induce a bias in these estimators. Fortunately, Greiner's
link function does appear to offer a good approximation beyond the
class of elliptical distributions, as illustrated in Figure \ref{fig:Greiner's-link-function}.
In our empirical application we use sparsely sampled financial returns,
which is an application where a Gaussian assumption has some theoretical
justification.
\subsection{A New Correlation Estimator}
Our new estimator is motivated by the empirical situation one encounters
with high-frequency financial data, where market microstructure noise,
jumps, and time-varying volatility pose challenges to the validity
of correlation estimators. While Pearson is the ideal estimator when
the variables are distributed as a bivariate Gaussian distribution,
it is inconsistent under more realistic and commonly accepted assumptions
for intraday returns. The $K$ estimator is more robust, but also
inconsistent under time-varying volatility, while $Q$ is is very
inefficient. This motivates the estimator introduced below.
\subsubsection{Notation with High-Frequency Data}
Let $X(t)$ and $Y(t)$ denote the the observed logarithmically transformed
price processes over some period, such as a trading day. We denote
the intraday returns over a time-interval with length $\delta$ by
\[
\Delta_{\delta}X_{t}=X(t)-X(t-\delta),
\]
and similarly for $\Delta_{\delta}Y_{t}$. In the context of high
frequency data it is common to sample sparsely to mitigate the effects
of market microstructure noise, and a popular choice is to set $\delta$
equal to five minutes. If we normalize the interval of time to be
$[0,1]$ and set $\delta=\frac{1}{n}$, then the correlation estimators
given above, may be applied to $(x_{i},y_{i})=(\Delta_{\frac{1}{n}}X_{\frac{i}{n}},\Delta_{\frac{1}{n}}Y_{\frac{i}{n}})$
for $i=1,\ldots,n$.
Let $N$ denote the number of intraday returns at the highest possible
sampling frequency and suppose, for simplicity, that $N$ is divisible
by $n$, such that $S=N/n\in\mathbb{N}$. Then we can create $S$
distinct grids by shifting the initial observation time to be $t_{s}=s/(Sn)$
for $s=0,\ldots,S-1$. Each grid will have sparely and non-overlapping
returns, and combined we have $N-S+1$ pairs of sparsely sampled returns,
$(\Delta_{\delta}X_{\frac{j}{N}},\Delta_{\delta}Y_{\frac{j}{N}})$,
for $j=S,\ldots,N$.\footnote{For instance, a 6.5 hour long trading day has $n=78$ intraday returns
when partitioned into $5$-minute intervals. By shifting the starting
time we obtain partitions with distinct 5-minute returns, each having
just 77 returns. By shifting the starting time in one-minute increments
we obtain $S=5$ different partitions and a total of $386$ 5-minute
returns.}
\subsubsection{Subsampled Quadrant Estimator}
We are now ready to introduce the subsampled variant of the Quadrant
estimator, defined by
\[
Q_{S}=\sin(\tfrac{\pi}{2}\hat{\tau}_{S})\qquad\text{with}\qquad\hat{\tau}_{S}=\frac{1}{N-S+1}\sum_{j=S}^{N}\mathrm{sgn}(\Delta_{\frac{S}{N}}X_{\frac{j}{N}}\Delta_{\frac{S}{N}}Y_{\frac{j}{N}}).
\]
The estimator does not require $N$ to be divisible by $S$, but if
$N$ is divisible by $S$, then $\hat{\tau}_{S}$ can be expressed
as a simple average of $S$ $\tau$-estimators based on different
grids. This construction is similar to many robust estimators of the
long-run variance. \citet{PolitisRomanoWolf99} noted that the subsampled
sample variance is identical to the moving-blocks estimator and the
jackknife variance estimator, and it is almost identical to the Bartlett
estimator, \nocite{Bartlett:1950}\citet[1950]{Bartlett:1946}, which
is often referred to as the Newey-West estimator in the econometrics
literature.\footnote{\citet[p.98]{PolitisRomanoWolf99}: ``{[}...{]} the variance estimator
$\hat{\sigma}_{\text{\textsc{sub}}}^{2}$ is actually asymptotically
equivalent to the Bartlett kernel estimator{[}...{]}'' and \citet[p.98]{PolitisRomanoWolf99}:
``In addition, $\hat{\sigma}_{\text{\textsc{sub}}}^{2}$ is identical
to the moving blocks bootstrap and/or jackknife variance estimator
of the variance of the sample mean proposed by \citet{Kunsch:89}
and \citet{LiuSingh:92} {[}...{]}''.} In the context of volatility estimation with high-frequency data,
the subsampling idea was first used in \citet{Zhou:1996}. The theoretical
foundation for subsampled realized variances was established in \citet{zhang-mykland-aitsahalia:05}
and \citet{Zhang:2006}, and the close connection between subsampled
estimators and kernel estimators is detailed in \citet{BNHLS-SubRK:2011}.
The subsampled quadrant correlation estimator has several appealing
properties. First, it inherits the robustness of the quadrant estimator
while being more precise than $Q$. The robustness is characterized
by the influence function, which is discussed below. Second, $Q_{S}$
is consistent under to time-varying volatilities. This is important
because time-varying volatility is common in economic time series,
especially in high-frequency financial data. Third, another computationally
attractive feature of the new $Q_{S}$ estimator, is that it relies
on binary variables. This makes it easier to scale this estimator
to large data sets.
\subsubsection{Implementation at Ultra High Frequencies}
One challenge with ultra-high-frequency data is that price increments
can be zero over short time intervals, resulting in $\Delta_{\delta}X\Delta_{\delta}Y=0$.
This issue may be caused by stale prices and rounding to a grid defined
by the minimum tick size. This issue abates quickly with sparse sampling,
and zeros are infrequent in our empirical analysis once we sample
a frequencies below one minute. Most of our emprical results are based
on $\delta=3$-minutes. Still, we will explore the properties of the
estimators at at higher sampling frequencies to gain insight about
them and market microstructure noise. For this reason we need to account
for zero returns, and we do so by redefining the estimator,
\[
\hat{\tau}_{S}=\frac{1}{N_{1}-S+1}\sum_{j=S}^{N}\mathrm{sgn}(\Delta_{\delta}X_{\frac{j}{N}}\Delta_{\delta}Y_{\frac{j}{N}}),\qquad\text{with}\quad N_{1}=\sum_{j}1_{\{\Delta_{\delta}X_{\frac{j}{N}}\Delta_{\delta}Y_{\frac{j}{N}}\neq0\}},
\]
such that we only count non-zero product-pairs.
\section{Properties of the Estimators}
In this section, we establish several properties of the estimators,
and we highlight some of the key advantages that are unique to $Q_{S}$.
We first consider the simple case with iid and normally distributed
variables. This is the situation that arises when the price process
are given from Brownian motions with constant volatility and the observed
prices are measured without error. We then proceed with more realistic
models with time-varying volatility and discuss robustness by means
of the influence function of the estimators. The impact of general
types of market microstructure noise will be analyzed in Section 4.
\subsection{Limit Distributions under Ideal Circumstances}
We begin with the simplest possible situation, where logarithmic price
processes follow Brownian motions with constant volatilities and constant
correlation.
\begin{assumption}
\label{assu:Brownian}Suppose that $(X,Y)$ is given by a bivariate
Brownian motion, such that $(X_{t},Y_{t})\sim N_{2}(0,t\Sigma)$.
\end{assumption}
In this situation, intraday returns, $(\Delta_{\delta}X_{i\delta},\Delta_{\delta}Y_{i\delta})$,
$i=1,\dots,n$ are iid and normally distributed. This is the ideal
situation for the Pearson estimator, because $P$ is the maximum likelihood
estimator of $\rho$ for the sample with the $n$ pairs of observations.
We should therefore expect $P$ to compare favorable to $Q$ and $K$.
It is less obvious how $P$ will compare with $Q_{S}$, because the
latter utilizes the shifted grids of sparsely sampled returns, $(\Delta_{\delta}X_{j\delta/S},\Delta_{\delta}Y_{j\delta/S}),j=S,\dots,N$,
and is thus computed from a larger data set. The asymptotic distribution
of the new estimator is given next.
\begin{thm}
\label{thm:efficiency}Suppose that Assumption \ref{assu:Brownian}
holds and let $S\in\mathbb{N}$ be fixed. Then the subsampled quadrant
correlation estimator is asymptotically normally distributed
\[
\sqrt{n}(Q_{S}-\rho)\overset{d}{\rightarrow}N(0,V_{S}(\rho)),
\]
as $n\rightarrow\infty$ , where
\[
V_{S}(\rho)=(1-\rho^{2})\frac{1}{S}\sum_{s=-S}^{S}\left[\mathrm{asin}^{2}(w_{s})-\mathrm{asin}^{2}(w_{s}\rho)\right],\qquad w_{s}=\tfrac{S-|s|}{S},
\]
which is decreasing in $S$ and bounded from below by
\[
\lim_{S\rightarrow\infty}V_{S}(\rho)=(1-\rho^{2})2\left[\mathrm{asin}^{2}(1)-2\tfrac{\sqrt{1-\rho^{2}}}{\rho}\mathrm{asin}(\rho)-\mathrm{asin}^{2}(\rho)\right].
\]
\end{thm}
The corresponding asymptotic distributions for the estimators, $Q$,
$K$, and $P$, are well known, see e.g. \citet{CrouxDehon:2010}.
For the sake of comparison, these are included below.
\begin{prop}
Suppose that Assumption \ref{assu:Brownian} holds, then as $n\rightarrow\infty$
we have
\begin{eqnarray*}
\sqrt{n}(P-\rho) & \stackrel{d}{\rightarrow} & N(0,V_{P}),\qquad\text{with}\quad V_{P}=(1-\rho^{2})^{2},\\
\sqrt{n}(K-\rho) & \stackrel{d}{\rightarrow} & N(0,V_{K}),\qquad\text{with}\quad V_{K}=(1-\rho^{2})[(\tfrac{\pi}{3})^{2}-4\mathrm{asin}^{2}(\tfrac{\rho}{2})],\\
\sqrt{n}(Q-\rho) & \stackrel{d}{\rightarrow} & N(0,V_{Q}),\qquad\text{with}\quad V_{Q}=(1-\rho^{2})[(\tfrac{\pi}{2})^{2}-\mathrm{asin}^{2}(\rho)].
\end{eqnarray*}
\end{prop}
We can compare the asymptotic variance of $Q_{S}$ to those of the
other estimators. The asymptotic variances depend on $\rho$ and those
for $P$, $K$, and $Q_{S}$ are shown in Figure \ref{fig:avar_asym}.
With $S=1$ we obviously have $V_{Q_{S}}=V_{Q}$.\footnote{With $S=1$ we have $\sum_{s=-S}^{S}\arcsin^{2}(\tfrac{S-|s|}{S})-\arcsin^{2}(\tfrac{S-|s|}{S}\rho)=\arcsin^{2}(1)-\arcsin^{2}(\rho)=\tfrac{\pi^{2}}{4}-\arcsin^{2}\rho$
such that $V_{Q_{S}}=V_{Q}$ as expected.} For a sufficiently large $S$, the subsampled Quadrant estimator
is more accurate than the Kendall estimator. For small values of $\rho$,
$Q_{S}$ is more accurate than $K$ when $S\geq5$, whereas a larger
value of $S$ in required for larger values of $\rho$. The $Q_{S}$
estimator is similar to $P$ for large values of $S$, with $Q_{S}$
having the edge for small values of $\rho$, whereas $P$ has the
edge for large values of $\rho$.
Realized measures are commonly computed from sparsely sampled returns,
such as 5-minute returns, to minimize the impact of market microstructure
noise. There will typically be a large number of observations within
each 5-minute interval, and this makes it possible to use a relatively
large value for $S$.
\begin{figure}[H]
\begin{centering}
\includegraphics[width=0.9\textwidth]{figures/AVAR}
\par\end{centering}
\caption{Asymptotic variances as a function of the true correlation, $\rho$,
for Quadrant and subsampled Quadrant estimators in the upper panel
and subsampled Quadrant, Pearson, and Kendall estimators in the lower
panel.}
\label{fig:avar_asym}
\end{figure}
\subsection{Properties with Time-Varying Volatility}
Time-varying volatility is an intrinsic feature of financial time-series.
For instance, volatility if found to vary substantially in high-frequency
financial data, even within a trading day. Next, we relax the assumption
that volatility is constant and evaluate the effect this has on the
correlation estimators. The asymptotic properties of correlation estimators
stated above need not apply in this context, because they were derived
under constant volatility.
We can illustrate the issues that arise from time-varying volatility
with a simple bivariate Brownian semimartingale.
\begin{assumption}
\label{assu:BSM}Suppose that
\begin{equation}
\left(\begin{array}{c}
X_{t}\\
Y_{t}
\end{array}\right)=\left(\begin{array}{c}
X_{0}\\
Y_{0}
\end{array}\right)+\int_{0}^{t}\sigma(u)\mathrm{d}W(u),\label{eq:ito}
\end{equation}
where $W(u)$ is a bivariate Wiener process with $\mathrm{cor}(\mathrm{d}W_{x},\mathrm{d}W_{y})=\rho$
and
\[
\sigma(u)=\begin{pmatrix}\sigma_{x}(u) & 0\\
0 & \sigma_{y}(u)
\end{pmatrix},
\]
is a squared integrable CADLAG process.
\end{assumption}
The assumption can be generalized in many ways, such as having a random
drift term,\footnote{Formally, we can let the logarithmic price process be defined on the
filtered probability space $(\Omega,\mathcal{F},(\mathcal{F})_{t\in[0,1]},\mathbb{P})$
with a locally bounded predictable drift function, $a(u)$, where
$a$, $\sigma$, and $W$ are adapted to a $\mathcal{F}_{t}$.} but the simple setup presented here suffices to show that traditional
correlation estimators are biased in the presence of time-varying
volatility, and establish that quadrant-based estimators are robust
to time-varying volatility. We will, initially, take the correlation
coefficient to be constant over time. The case with time varying correlation
is discussed below in Section 3.3.
\begin{thm}
\label{theo:Consistency}If Assumption \ref{assu:BSM} holds, then
$Q_{S}\overset{p}{\rightarrow}\rho$ as $\delta\rightarrow0$, whereas
the probability limits for $P$ and $K$ are given by
\begin{eqnarray*}
P & \overset{p}{\rightarrow} & \lambda\rho,\qquad\text{where}\quad\lambda=\tfrac{\int_{0}^{1}\sigma_{x}(u)\sigma_{y}(u)\mathrm{d}u}{\sqrt{\int_{0}^{1}\sigma_{x}^{2}(u)\mathrm{d}u\int_{0}^{1}\sigma_{y}^{2}(u)\mathrm{d}u}},\\
K & \overset{p}{\rightarrow} & \sin\left(\int_{0}^{1}\int_{0}^{1}\arcsin\left(h(u,v)\rho\right)\mathrm{d}u\mathrm{d}v\right),
\end{eqnarray*}
where $h(u,v)=\tfrac{\sigma_{x}(u)\sigma_{y}(u)+\sigma_{x}(v)\sigma_{y}(v)}{\sqrt{\sigma_{x}^{2}(u)+\sigma_{x}^{2}(v)}\sqrt{\sigma_{y}^{2}(u)+\sigma_{y}^{2}(v)}}$.
\end{thm}
The important result from Theorem \ref{theo:Consistency} is that
$Q_{S}$ emerges as the only consistent estimator when volatility
is time-varying. Both $P$ and $K$ are generally inconsistent, except
in the following special case where the two volatility processes are
perfectly collinear.
\begin{figure}[tbh]
\begin{centering}
\includegraphics[scale=0.8]{figures/BiasTVvolatilities}
\par\end{centering}
\caption{Bias in correlation estimators, $P$, $K$, and $Q_{S}$, with two
different degrees of collinearity between the volatilities.\label{fig:BiasTVvol}}
\end{figure}
\begin{cor}
Suppose that Assumption \ref{assu:BSM} holds and that $\sigma_{y}(u)=c\sigma_{x}(u)$
for some $c>0$. Then $P\overset{p}{\rightarrow}\rho$ and $K\overset{p}{\rightarrow}\rho$.
\end{cor}
The results for $P$ and $K$ in this special case are easy to verify,
because perfectly collinearity implies $\lambda=1$ and that
\[
h(u,v)=\frac{c\sigma_{x}^{2}(u)+c\sigma_{x}^{2}(v)}{\sqrt{\sigma_{x}^{2}(u)+\sigma_{x}^{2}(v)}\sqrt{c^{2}(\sigma_{x}^{2}(u)+\sigma_{x}^{2}(v))}}=1,
\]
for all $u,v$.
We illustrate the inconsistency with a simple example. Consider the
functions, $g(u)=\tfrac{1}{5}+\tfrac{4}{5}u$ and $h(u)=\frac{1}{2}(\tfrac{6}{5}+\cos(2\pi u))$,
which we will use to construct volatility paths with varying degrees
of collinearity. The upper left panel in Figure \ref{fig:BiasTVvol}
represents a case with low collinearity, where $\sigma_{x}(u)=g(u)$
and $\sigma_{y}(u)=h(u)$, and the upper right panel corresponds to
a case with high collinearity, where $\sigma_{x}(u)=\frac{3}{10}g(u)+\frac{6}{10}h(u)$
and $\sigma_{y}(u)=h(u)$. The lower panels show the resulting bias
of the correlation coefficients, $P$, $K$, and $Q$, as a function
of the true correlation coefficient, $\rho$. With low collinearity,
the sample correlation, $P$, has a large bias unless $\rho$ is near
zero, and the bias in $K$ is about half that in $P$. These estimators,
$P$ and $K$, are also biased in the example with high collinearity,
but the bias is substantially smaller. The bias of these estimators
are also pronounced in standard simulation design with the Heston
model, as we document in Section \ref{sec:Simulation-Study}.
An important implication of the results in this subsection is that
conventional estimates of correlations between assets are systematically
influenced by the degree of collinearity in their volatilities.
\subsection{Time varying correlations}
The correlation may be time varying, as is the case for volatility.
To accommodate this situation we could modify Assumption \ref{assu:BSM}
and let $\rho(u)$ be a CADLAG process. In this situation, the integrated
correlation, $\rho_{\bullet}=\int_{0}^{1}\rho(u)\mathrm{d}u$, is
a natural object of interest. Unfortunately, none of the correlation
estimators are consistent for $\rho_{\bullet}$. For instance, $Q_{S}$
will estimate $\tilde{\rho}_{\bullet}=\sin[\int_{0}^{1}\mathrm{asin}\{\rho(u)\}\mathrm{d}u]$,
and since $\mathrm{asin}(t)$ is strictly convex for $t>0$ and concave
for $t<0$, it follows that $\tilde{\rho}_{\bullet}\geq\rho_{\bullet}$
if the $\rho(u)\geq0$ and $\tilde{\rho}_{\bullet}\leq\rho_{\bullet}$
if the $\rho(u)\leq0$.
One way to partially account for time-varying correlations is to apply
the correlation estimators over relatively short intervals of time
and aggregate these local estimates to an estimate of $\int_{0}^{1}\rho(u)\mathrm{d}u$.
This approach was used in jump-robust estimation of the integrated
covariance in \citet{BoudtCornelissenCroux:2012jump}. In our empirical
analysis we will also use local estimates of $\rho$ to assess time-variation
in $\rho(u)$.
\subsection{Estimating Integrated Covariance}
Interestingly, it is not advisable to combine the robust correlation
estimator with volatility estimators for the purpose of estimating
the integrated covariance, $\mathrm{IC}=\int\sigma_{xy}(u)\mathrm{d}u$,
which simplifies to $\rho\int\sigma_{x}(u)\sigma_{y}(u)\mathrm{d}u$
when $\rho(u)=\rho$ for all $u$. Now, if we multiply $Q_{S}$ by
consistent estimates of $\sqrt{\int\sigma_{x}^{2}(u)\mathrm{d}u}$
and $\sqrt{\int\sigma_{y}^{2}(u)\mathrm{d}u}$, we will be estimating
$\frac{1}{\lambda}\mathrm{IC}$, instead of $\mathrm{IC}$. Using
$K$ is not advisable either, because it leads to another incorrect
limit. For this problem, localized estimators of spot volatility and
spot correlation can be use, as proposed in \citet{BoudtCornelissenCroux:2012jump}.
In order to be robust to jumps, they combine the MedRV estimator by
\citet{AndersenDobrevSchaumburg2012} and the Gaussian rank correlation.
This is further explored in \citet{VanderVeredas:2016} who employ
additional robust correlation estimators, as a component to estimate
$\mathrm{IC}$. They also combine non-localized estimates of $\sqrt{\int\sigma_{x}^{2}(u)\mathrm{d}u}$
and $\sqrt{\int\sigma_{y}^{2}(u)\mathrm{d}u}$ with a range of correlation
estimators. Some of these combinations will be inconsistent for the
reason stated earlier. This may explain that \citet{VanderVeredas:2016}
find the bivariate realized kernel estimator by \citet{BNHLS:2011}
to be the most accurate estimator of $\mathrm{IC}$ in the absence
of jumps.
\subsection{Influence Function}
The \emph{influence function} can be used to measure an estimator's
sensitivity to data contamination. It measures the sensitivity of
a statistical functional, $R$, to data contamination in a baseline
distribution, $F$, and is defined by
\[
\mathrm{IF}((x_{0},y_{0}),R,F)=\lim_{\eta\searrow0}\frac{R((1-\eta)F+\eta\mathbf{\Delta}_{(x_{0},y_{0})})-R(F)}{\eta},
\]
where $\mathbf{\Delta}_{(x_{0},y_{0})}$ is the Dirac measure at $(x_{0},y_{0})$.
We have $R_{P}(F)=R_{Q}(F)=R_{K}(F)=\rho$ for $F=\Phi_{\rho}$, which
denotes the standard bivariate normal distribution with correlation
equal to $\rho$. From \citet{DevlinGnanadesikanKettenring:1975}
and \citet{CrouxDehon:2010} we have their influence functions.
\begin{prop}
The influence functions of correlation estimators at $\Phi_{\rho}$
are given by
\[
\begin{aligned}\mathrm{IF}((x_{0},y_{0}),R_{Q},\Phi_{\rho}) & =\tfrac{\pi}{2}\sqrt{1-\rho^{2}}(\mathrm{sgn}(x_{0}y_{0})-\tau)\\
\mathrm{IF}((x_{0},y_{0}),R_{K},\Phi_{\rho}) & =2\pi\sqrt{1-\rho^{2}}(2\Phi(x_{0},y_{0})+1-\Phi(x_{0})-\Phi(y_{0})-\tfrac{1}{2}(\tau+1))\\
\mathrm{IF}((x_{0},y_{0}),R_{P},\Phi_{\rho}) & =x_{0}y_{0}-(\frac{x_{0}^{2}+y_{0}^{2}}{2})\rho
\end{aligned}
\]
where $\Phi(\bullet,\bullet)$ denotes the joint CDF for $\Phi_{\rho}$
and and $\Phi(\bullet)$ denote the marginal CDF for a standard normal
distribution.
\end{prop}
The important message from the influence functions is that $Q$ and
$K$ have bounded influence functions whereas $P$ has an unbounded
influence function. This difference motivate their labeling as robust
and non-robust estimators, respectively. The unbounded influence function
of $P$ makes it sensitive to outliers. It is intuitive that $Q$
and $K$ are less sensitive to outliers, since they are computed from
signed variable alone. This limits the harm an outlier can cause to
merely flipping the sign. The analogous results for $Q_{S}$ are qualitative
very similarly, and are presented in the Supplementary Material. One
way to alleviate the sensitivity that $P$ has to outliers is to use
truncation estimators, which is commonly used for estimating realized
variances, see e.g. \citet{Mancini:2009}.
The influence function for the Spearman estimator is also bounded
but can be shown to have a larger bound than $Q$ and $K$, whereas
the Gaussian rank estimator has an unbounded influence function, see
\citet{Rousseeuw:1984}, \citet{BoudtCornelissenCroux:2012gauss},
and \citet{RaymaekersRousseeuw:2021} for details and additional results
on influence functions.
\section{Simulation Study\label{sec:Simulation-Study}}
We compare the estimators in simulation studies that are designed
to emulate the situation we encounter in our empirical analyses with
high-frequency data. We generate the two logarithmic price processes,
$X_{t}^{\ast}$ and $Y_{t}^{\ast}$, using the Heston model:
\begin{equation}
\begin{aligned}\mathrm{d}X_{t}^{\ast} & =\mu_{j}\mathrm{d}t+\sigma_{xt}\mathrm{d}W_{xt},\\
\mathrm{d}\sigma_{x,t}^{2} & =\kappa_{x}(\bar{\sigma}_{x}^{2}-\sigma_{x,t}^{2})\mathrm{d}t+s_{x}\sigma_{x,t}dB_{x,t},
\end{aligned}
\label{eq:heston}
\end{equation}
where $W_{x,t}$ and $B_{x,t}$ are standard Brownian motions with
$\mathrm{cov}(\mathrm{d}W_{x,t},\mathrm{d}B_{xt})=\varrho_{x}dt$,
and $Y_{t}^{\ast}$ is generated similarly with $\mathrm{cov}(\mathrm{d}W_{y,t},\mathrm{d}W_{y,t})=\rho\mathrm{d}t$.
The model is calibrated using the simulation design in Table \ref{tab:Parameters-calibration},
which was previously used in \citet{AitSahaliaFanXiu:2010}. The initial
values for volatility $\sigma_{x,0}^{2}$ and $\sigma_{y,0}^{2}$
are drawn from Gamma distributions, $\Gamma(2\kappa_{x}\bar{\sigma}_{x}^{2}/s_{x}^{2},s_{x}^{2}/2\kappa_{x})$
and $\Gamma(2\kappa_{y}\bar{\sigma}_{y}^{2}/s_{y}^{2},s_{y}^{2}/2\kappa_{y})$,
and the price processes are initialized with $X_{0}^{\ast}=\log(100)$
and $Y_{0}^{\ast}=\log(40)$. The simulated model is a discretized
version with $23,400$ increments, which translates to 1 second observations
over a 6.5 hours period -- the length of a typical trading day.
\begin{table}[H]
\centering{}\caption{\label{tab:Parameters-calibration}Parameters calibration for the
Heston model}
\begin{tabular}{cccccc}
\hline
& $\mu$ & $\bar{\sigma}^{2}$ & $\kappa$ & $s$ & $\varrho$\tabularnewline
\hline
$X$ & 0.05 & 0.16 & 3 & 0.8 & -0.60\tabularnewline
$Y$ & 0.03 & 0.09 & 2 & 0.5 & -0.75\tabularnewline
\hline
\end{tabular}
\end{table}
We present results for two values of the true correlation, $\rho=1/4$
and $\rho=2/3$, which are typical levels of the correlation in our
empirical analysis. In the Supplementary Material we present the corresponding
results for $\rho=1/2$ and $\rho=3/4$.
\subsection{Case without Noise}
We first consider the case where prices are observed without measurement
error. This defines the limit to which we can apply subsampling. For
instance, for sparsely sampled 1-minute returns we can set $S=60$.
In the absence of noise, there is no need to sample sparsely, but
we gain valuable insight about the the estimators by studying their
properties at lower sampling frequencies.
The Heston model generates prices process with time-varying volatilities.
For this reason, we should not expect $P$ and $K$ to be consistent.
While $Q$ and $Q_{S}$ are consistent, they may have a bias in finite
samples, because sampling error in $\hat{\tau}$ and the non-linear
transformation, $\rho=\sin(\frac{\pi}{2}\tau)$, will induce a finite-sample
bias in $Q$ and $Q_{S}$.
\begin{figure}[tbh]
\begin{centering}
\includegraphics[width=0.45\textwidth]{figures/Simulation/dynvol_Heston_mean_rho25}\includegraphics[width=0.45\textwidth]{figures/Simulation/dynvol_Heston_mean_rho66}
\par\end{centering}
\centering{}\includegraphics[width=0.45\textwidth]{figures/Simulation/RMSE/dynvol_Heston_rmse_rho25}\includegraphics[width=0.45\textwidth]{figures/Simulation/RMSE/dynvol_Heston_rmse_rho66}\caption{Means and RMSEs for the estimators as a function of sampling frequencies.
Prices are generated by the Heston model with the true correlation
being $\rho=0.25$ (left panels) and $\rho=0.6\overline{6}$ (right
panels).\label{fig:Plot-the-averages1}}
\end{figure}
The average values of the estimators are shown in the upper panels
of Figure \ref{fig:Plot-the-averages1} for the case were $\rho=0.25$
and $\rho=0.6\overline{6}$. As expected, $P$ and $K$ are biased
as expected, since volatility is time-varying in the Heston model,
which $P$ being substantially more biased than $K$. At the highest
sampling frequency, $P$ and $K$ become very accurate estimates of
incorrect quantities, as defined in Theorem \ref{theo:Consistency}.
The $Q_{S}$ estimator is largely unbiased when returns are sampled
more frequently that every minute. At slower sampling frequencies
a bias begin to emerge in $Q_{S}$, which is a consequence of Jensen's
inequality. The variance of $\hat{\tau}$ increases with the sampling
frequency and the concavity of $\tau\mapsto\sin(\frac{\pi}{2}\tau)$
for $\tau>0$ explains the downwards bias that becomes evident at
slow sampling frequencies. However the bias of $Q_{S}$ is substantially
smaller than those of $K$ and $P$.
The corresponding root mean squared errors (RMSEs) are shown in the
two lower panels. The new estimator has the smallest RMSE, which is
driven by its ability to reduce the bias.
\subsection{Microstructure Issues}
Next, we amend the simulation to mimic features commonly seen in empirical
data. We do so, by adding different forms of market microstructure
noise. Noise will influence estimators in different ways. Noise that
only alters the sign of a small fraction of returns will have minute
impact on the robust estimators, but could have a large impact on
$P$. Rare outliers provide an example of this scenario, and can be
inferred directly from the influence functions for the different estimators.
\subsubsection{Independent Noise}
Independent noise in the price processes can induce the Epps effect.
The independent noise reduces the correlation in returns and this
downwards bias is increasing in the sampling frequency. We simulate
independent noise as follows:
\[
X_{t}=X_{t}^{\ast}+\epsilon_{xt}
\]
where $\epsilon_{xt}\sim iidN(0,\omega_{x}^{2})$ and similar for
$Y_{t}$ with $\epsilon_{xt}$ independent of $\epsilon_{yt}$. Following
similar simulation designs in this literature, see e.g. \citet{BandiRussell:2006}
and \citet{BNHLS:2008}, we set $\omega_{x}^{2}=\xi^{2}\sqrt{T^{-1}\sum_{i=1}^{T}\sigma_{x,i/T}^{4}}$
with $\xi^{2}=0.001$, such that variance of the noise is proportional
to square root of the integrated quarticity.
\subsubsection{Prices with tick-size increments}
In practice, high-frequency financial prices are restricted to a grid
defined by their tick-size. This induces a particular type of market
microstructure noise, as analyzed in \citet{DelattreJacod:1997},
\citet{Horel2007}, \citet{Rosenbaum:2009}, \citet{ManciniGobbi:2012},
\citet{Hansen:MC2015}, \citet{LiMykland:2015}, \citet{HansenHorelLundeArchakov},
and \citet{LiZhangLi:2018}. We will study this phenomenon by letting
observed prices be given by
\[
X_{t}=\alpha\lfloor X_{t}^{\ast}/\alpha\rfloor\qquad\text{and}\qquad Y_{t}=\alpha\lfloor Y_{t}^{\ast}/\alpha\rfloor,
\]
where $\alpha$ defines the coarseness of the grid.\footnote{In reality it is nominal prices, $\exp X_{t}$ and $\exp Y_{t}$ that
are confined to a grid, but it makes no practical difference over
trading day.} In our simulations we let the coarseness be proportional to the level
of volatility, $\alpha=c\sigma$ in order to control the average number
of price changes within a given period of time. The true price processes
are, as before, define by (\ref{eq:heston}).
\subsubsection{Tick size with Noise}
Next we add noise to the grid of observed prices. Specifically we
now observe
\[
X_{t}=\alpha\lfloor X_{t}^{\ast}/\alpha\rfloor+\epsilon_{xt},\qquad\text{with}\quad\epsilon_{xt}=\begin{cases}
0 & \text{with probability }1-p,\\
\pm\alpha & \text{with probability }p/2,
\end{cases}
\]
and similarly for $Y_{t}$ with $\epsilon_{xt}$ and $\epsilon_{yt}$
independent.
\subsubsection{Stale Prices}
We introduce stale pricing using
\begin{equation}
X_{\frac{j}{N}}=\begin{cases}
\alpha\lfloor X_{\frac{j}{N}}^{\ast}/\alpha\rfloor & \text{with probability }1-q,\\
X_{\frac{j-1}{N}} & \text{with probability }q.
\end{cases}\label{eq:StalePrices}
\end{equation}
This will generate ``flat pricing'' and the expected duration between
price updates will be $(1-q)^{-1}/N$.
\subsubsection{Jumps}
Jumps are prevalent in high-frequency prices, and we could generate
such with
\[
X_{t}=X_{t}^{\ast}+\sum_{s\leq t}J_{s}^{x}\qquad Y_{t}=Y_{t}^{\ast}+\sum_{s\leq t}J_{s}^{y},
\]
where $J_{t}^{x}$ and $J_{t}^{y}$ denote jump processes. The impact
that jumps have on the estimators is characterized by their influence
functions. The robust estimators, $Q_{S}$ and $K$, are essentially
unaffected by jumps, whereas $P$ is highly sensitive. Independent
jumps will cause $P$ to be biased towards zero, whereas a co-jump
(a simultaneous jump in both series) will bias $P$ towards $-1$
or $1$, depending on the sign of $J_{s}^{x}J_{s}^{y}$. Co-jumps
in the same direction will cause $P$ to be biased towards one, whereas
co-jumps in the opposite direction will cause $P$ to be biased towards
$-1$. Jumps can be alleviated by truncation methods, see \citet[2009]{Mancini:2001}
and \citet{AndersenDobrevSchaumburg2012}.\nocite{Mancini:2009} Simulation
results with jumps are presented in the Supplementary Material.
\subsection{Simulation Results}
The bias that different types of noise induce on the correlation estimators
are show in Figure \ref{fig:Correlation-signature-plots}. The true
correlation is $\rho=1/4$ in the left panels and $\rho=2/3$ in the
right panels. Results for additional levels of correlation and types
of noise are presented in the supplementary material. Panel (a) in
Figure \ref{fig:Correlation-signature-plots} presents the results
when the efficient prices are contaminated with independent Gaussian
noise with a variance that is about $10^{-3}$ times the square root
of integrated quarticity of the two series. Independent noise is one
(of several ways) to bring about the Epps effect. The independent
noise reduces the correlation between returns, which induces a downwards
bias that increases with sampling frequency, to an extend that all
estimators essentially becomes noisy estimates of zero when computed
with 1-second intraday returns. Independent noise is a good stating
point for studying estimators, but there is overwhelming empirical
evidence that contradicts the independent noise assumption in high
frequencies data, see \citet{HansenLunde:JBES2006}, which is also
the case in our empirical analysis.
The correlation signature plots in our empirical analysis resemble
those in Panel (b) of Figure \ref{fig:Correlation-signature-plots},
where the noise is defined by a rounding error ($\alpha=10^{-4}$),
to resemble the tick size in prices. Interestingly, the rounding error
causes $Q_{S}$ to be upwards bias a higher sampling frequencies.
This is also true for $K$, but to a much lesser extend, whereas $P$
is largely unaffected, but maintains the downwards bias caused by
time-varying volatilities. In Figure \ref{fig:Correlation-signature-plots}
(c) we consider the same level of rounding error ($\alpha=10^{-4}$)
and add additional noise by shifting the price up or down by one tick
size with equal probability, $p/2$ with $p=0.75$. This induces a
downwards bias, which is most pronounced a high sampling frequencies.
Finally, in Figure \ref{fig:Correlation-signature-plots} (d) we add
additional staleness to prices on the grid, as defined by (\ref{eq:StalePrices}),
where one price series remains stale with probability $q_{x}=0.5$
and the other series remains stale with probability $q_{y}=0.8$.
The combined impact of rounding and staleness is a sizable downwards
bias.
\begin{figure}[H]
\begin{centering}
\subfloat[Heston with independent noise]{
\centering{}\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonIndependentNoise_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonIndependentNoise_rho66}}
\par\end{centering}
\begin{centering}
\subfloat[Heston with rounding to a grid]{\begin{centering}
\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonTick0001_0001_NoiseProb0_0_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonTick0001_0001_NoiseProb0_0_rho66}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Heston with rounding and grid-noise]{\begin{centering}
\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonTick0001_0001_NoiseProb075_075_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonTick0001_0001_NoiseProb075_075_rho66}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Heston with rounding and staleness]{\begin{centering}
\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonTick0001_0001_UpdateProb05_08_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/HestonTick0001_0001_UpdateProb05_08_rho66}
\par\end{centering}
}
\par\end{centering}
\caption{Correlation signature plots, with sampling frequency ranging from
1 second to 15 minutes. Observed prices are generated with Heston
models with a layer of noise added. The true correlation is $\rho=1/4$
in left panels and $\rho=2/3$ in right panels. The four types of
noise are: (a) Independent noise ($\xi^{2}=10^{-3}$); (b) rounding
to a grid ($\alpha=10^{-4}$); (c) rounding with grid-noise ($\alpha=10^{-4}$
and $p=0.75$); and (d) rounding with stale prices ($\alpha=10^{-4}$
and $q_{x}=0.50$ and $q_{y}=0.80$).\label{fig:Correlation-signature-plots}}
\end{figure}
\begin{figure}[H]
\begin{centering}
\subfloat[Heston with independent noise]{
\centering{}\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonIndependentNoise_rmse_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonIndependentNoise_rmse_rho66}}
\par\end{centering}
\begin{centering}
\subfloat[Heston with rounding to a grid]{\begin{centering}
\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonTick0001_0001_NoiseProb0_0_rmse_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonTick0001_0001_NoiseProb0_0_rmse_rho66}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Heston with rounding and grid-noise]{\begin{centering}
\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonTick0001_0001_NoiseProb075_075_rmse_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonTick0001_0001_NoiseProb075_075_rmse_rho66}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Heston with rounding and staleness]{\begin{centering}
\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonTick0001_0001_UpdateProb05_08_rmse_rho25}\includegraphics[width=0.4\textwidth]{figures/Simulation/RMSE/HestonTick0001_0001_UpdateProb05_08_rmse_rho66}
\par\end{centering}
}
\par\end{centering}
\caption{RMSE signature plots. Observed prices are generated with Heston models
with a layer of noise added. The true correlation is $\rho=1/4$ in
left panels and $\rho=2/3$ in right panels. The four types of noise
are: (a) Independent noise, (b) rounding to a grid ($\alpha=10^{-4}$)
(c) rounding with grid-noise ($\alpha=10^{-4}$ and $p=0.75$); and
(d) rounding with stale prices ($\alpha=10^{-4}$ and $q_{x}=0.50$
and $q_{y}=0.80$).\label{fig:RMSE-signature-plots}}
\end{figure}
The corresponding root mean squares errors (RMSEs) are reported in
Figure \ref{fig:RMSE-signature-plots}. The new correlation estimator,
$Q_{S}$, tends to have the smallest RMSE, which is also true for
the additional simulation experiments presented in the Supplementary
Material.
\section{Empirical Application}
We apply the correlation estimators to high-frequency data for about
100 assets. We begin by analyzing and comparing their daily correlation
estimates. For instance, we use correlation signature plots to study
market microstructure noise, and explore how sensitive the estimators
are to the choice of sampling frequency, as defined by $\delta$.
Then we turn to estimation of intraday correlations, which we find
to vary substantially over the hours with active trading. Correlations
between stocks and the market are, on average, increasing for all
assets in our sample. We obtain estimates of intraday betas, by combining
the correlation estimates with estimates of relative volatility. We
then proceed to related intraday variation in correlations and betas
with asset characteristics, such as low frequency based market beta,
market capitalization, and book-to-market valuations. This part of
the analysis is done with an expanded set of assets detailed below.
Our sample period covers the period from January 1, 2015 to December
31, 2021 and includes 1,763 trading days. We use NYSE and NASDAQ transaction
prices from the TAQ database that were accessed through the Wharton
Research Data Services (WRDS) system. The data were cleaned following
the guidelines in \citet{BNHLS:2011}, and prices (when unavailable)
were interpolated by the previous-tick methods. We will analyzed 22
stocks and SPY, an exchange traded fund that tracks the S\&P 500 index,
in great details. We label this data set ``Small Universe''. The
22 stocks were selected to be the two largest stocks (by market capitalization)
within each of the eleven GICS\footnote{Global Industry Classification Standard.}
sectors. A larger set of asset of assets, ``Large Universe'' is
used to identify asset-characteristics associated with different patterns
in intraday market betas. The Large universe includes the assets in
the S\&P 100 index, as of {[}date{]}, we excluded two of these assets
from the Large Universe. PYPL (PayPal) was excluded because it only
started trading in 2015 after being spun off eBay, and RTX (formerly
Raytheon Tech) was excluded because it merged with United Technologies,
which was completed in April 2020.
\begin{table}[H]
\caption{Summary Statistics Small Universe\label{tab:StatisticsSmallUniverse}}
\begin{centering}
\begin{tabular*}{1\textwidth}{@{\extracolsep{\fill}}>{\raggedright}m{4.5cm}lS[table-format=4.2]S[table-format=2.2]S[table-format=2.2]S[table-format=2.2]}
\hline
\ \\[-6pt]
Sector (GICS code) & Ticker & \multicolumn{1}{c}{Average} & \multicolumn{1}{c}{Duration} & \multicolumn{2}{c}{Zero-returns (\%)} \\
& & \multicolumn{1}{c}{Price} & \multicolumn{1}{c}{(seconds)} & \multicolumn{1}{c}{$\delta=1$s} & \multicolumn{1}{c}{$\delta=3$m}\\[5pt]
\hline
\ \\[-6pt]
& SPY & 281.89 & 2.28 & 63.39 & 3.73\\[5pt]
Energy & HAL & 33.38 & 5.56 & 89.25 & 8.37\\
(10) & XOM & 72.44 & 3.83 & 82.73 & 6.71\\[3pt]
Materials & LYB & 89.55 & 9.23 & 90.85 & 4.94\\
(15) & NEM & 40.10 & 6.67 & 89.76 & 8.73\\[3pt]
Industrials & AAL & 34.32 & 6.83 & 89.76 & 8.54\\
(20) & UNP & 144.23 & 6.86 & 87.82 & 4.29\\[3pt]
Consumer Discretionary & TSLA & 415.90 & 5.12 & 70.38 & 1.14\\
(25) & AMZN & 1658.92& 5.03 & 76.60 & 0.75\\[3pt]
Consumer Staples & PG & 101.81 & 4.86 & 85.85 & 6.92\\
(30) & WMT & 99.02 & 4.46 & 84.87 & 6.75\\[3pt]
Health Care & JNJ & 131.77 & 4.57 & 83.74 & 5.89\\
(35) & MRK & 68.76 & 5.06 & 87.05 & 7.50\\[3pt]
Financials & JPM & 101.17 & 3.22 & 78.24 & 5.53\\
(40) & WFC & 48.11 & 4.50 & 86.83 & 7.84\\[3pt]
Information Technology & AAPL & 167.52 & 2.27 & 63.05 & 3.57\\
(45) & AMD & 33.45 & 14.87& 82.87 & 19.17\\[3pt]
Communication Services & DIS & 121.95 & 4.19 & 80.95 & 5.18\\
(50) & FB & 181.59 & 2.80 & 68.63 & 3.18\\[3pt]
Utilities & D & 75.08 & 8.04 & 91.35 & 7.92\\
(55) & DUK & 84.94 & 7.47 & 90.61 & 7.26\\[3pt]
Real estate & AMT & 168.88 & 10.18& 91.43 & 4.15\\
(60) & PLD & 72.88 & 9.77 & 92.62 & 8.70\\[5pt]
\hline
\end{tabular*}
\par\end{centering}
Note: Summary statistics for SPY and 22 assets (two from each of the
11 sectors) for the sampling period from January 1, 2015 to December
31, 2021. Average price, average duration between two consecutive
transactions are listed along with the percentage of zero returns
when returns are sampled at 1 second and 3 minutes, respectively.
\end{table}
Table \ref{tab:StatisticsSmallUniverse} presents the summary statistics
for the Small Universe with 22 assets. The exchange traded fund, SPY,
is the most frequently traded asset, followed by AAPL and FB. On average,
these securities have just over 2 seconds between transaction prices.
The price range is an interesting statistic, because the tick-size
is more likely to induce rounding errors and price staleness for assets
trading at low prices. This appears to be relevant for AMD that traded
for less than \$3 in all of 2015 and below \$10 during most of the
first three years in our sample period. This likely explains the many
zero increments. More than 19\% of all 3-minute returns are zero in
this sample period.
The assets in the Large Universe are listed and organized by sectors
in Table \ref{tab:StatisticsLargeUniverse}.
\begin{table}[H]
\caption{Large Universe\label{tab:StatisticsLargeUniverse}}
\begin{centering}
\vspace{0.2cm}
\begin{small}
\begin{tabularx}{\textwidth}{YYYYYYYYYYYYY}
\toprule
Energy & Materials & Industrials & \multicolumn{1}{c}{Consumer} & \multicolumn{1}{c}{Consumer }& Healthcare \\
& & & \multicolumn{1}{c}{Discretionary} & \multicolumn{1}{c}{Staples} &\\
\midrule
& & & & & \\[-3pt]
COP & DOW & BA & AMZN & CL & ABBV \\
CVX & LIN & CAT & BKNG & COST & ABT \\
XOM & & EMR & F & KHC & AMGN \\
& & FDX & GM & KO & BMY \\
& & GD & HD & MDLZ & CVS \\
& & GE & LOW & MO & DHR \\
& & HON & MCD & PEP & GILD \\
& & LMT & NKE & PG & JNJ \\
& & MMM & SBUX & PM & LLY \\
& & UNP & TGT & WBA & MDT \\
& & UPS & TSLA & WMT & MRK \\
& & & & & PFE \\
& & & & & TMO \\
& & & & & UNH \\
& & & & & \\
\midrule
Financials & Information & Telecom. & Utilities & Real Estate &\\
& Technology & Services & & & \\
\midrule
& & & & & \\[-3pt]
AIG & AAPL & CHTR & DUK & AMT & \\
AXP & ACN & CMCSA & EXC & SPG & \\
BAC & ADBE & DIS & NEE & & \\
BK & AMD & GOOGL & SO & & \\
BLK & AVGO & FB & & & \\
BRKB & CRM & NFLX & & & \\
C & CSCO & T & & & \\
COF & IBM & TMUS & & & \\
GS & INTC & VZ & & & \\
JPM & MA & & & & \\
MET & MSFT & & & & \\
MS & NVDA & & & & \\
SCHW & ORCL & & & & \\
USB & QCOM & & & & \\
WFC & TXN & & & & \\
& V & & & & \\
\bottomrule
\end{tabularx}
\end{small}
\par\end{centering}
Note: List of assets in ``Large Universe'', organized by sectors.
\end{table}
\subsection{Estimates of Daily Correlations}
We apply the correlation estimators to daily high frequency data using
calendar-time sampling with frequencies ranging from 1 second to 15
minutes. The resulting correlation signature plots are shown in Figure
\ref{fig:CorrSignaturePlots} for a subset of the assets. These are
the two most actively traded securities, SPY and AAPL, the stock with
most zero returns, AMD, and the two stocks from the Material sector,
LYB and NEM, whose liquidity and percentage of zero returns is more
typical for assets in the Small Universe. Signature plots were introduced
in \citet{andersen-bollerslev-diebold-labys:00a} who plotted the
average realized variance against the sampling frequency used to compute
the underlying intraday returns. Signature plots help identify bias
in the estimators, which tend to be most pronounced at high sampling
frequencies. If the estimator is unbiased over a range of sampling
frequencies, then the signature plot will be roughly flat over that
those sampling frequencies.
\begin{figure}[H]
\begin{centering}
\subfloat[$\mathrm{corr}$(SPY,AAPL)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AAPL_cts}
\par\end{centering}
}\subfloat[$\mathrm{corr}$(SPY,AMD)]{\centering{}\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AMD_cts}}\subfloat[$\mathrm{corr}$(AAPL,AMD)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/AAPL_AMD_cts}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[$\mathrm{corr}$(SPY,LYB)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_LYB_cts}
\par\end{centering}
}\subfloat[$\mathrm{corr}$(SPY,NEM)]{\centering{}\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_NEM_cts}}\subfloat[$\mathrm{corr}$(LYB,NEM)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/LYB_NEM_cts}
\par\end{centering}
}
\par\end{centering}
\caption{Correlation signature plots, where the average correlation estimate
is plotted against sampling frequency for the three estimators, $Q_{S}$,
$P$, and $K$.\label{fig:CorrSignaturePlots}}
\end{figure}
The signature plots in Figure \ref{fig:CorrSignaturePlots} are signature
plot for correlations, which can be used to visualize biases, such
as the Epps effect. Here we observe that many of the plots have patterns
that resemble the effect for rounding to a grid, because $Q_{S}$
often has an upwards bias at high sampling frequencies, while $P$
has a downwards biased. Additional signature plots are presented in
the Supplementary Material, see Figure \ref{fig:SignaturePlotsSmallUniverse}.
We adopt 3-minutes as a common sampling frequency for all estimators.
This is in part motivated by the signature plots tend to be flat for
$\delta\geq$3 minutes, and in part because it makes our results more
comparable to those in ATT.
\begin{table}[tbh]
\caption{Correlation estimates.\label{tab:SectorCorr}}
\begin{centering}
\setlength{\tabcolsep}{5.3pt}
\begin{tabular}{l ccc c ccc c ccc c ccc}
\hline
\\[-5pt]
\multicolumn{1}{c}{Sector code} & \multicolumn{3}{c}{Average} & & \multicolumn{3}{c}{25th quantile} & & \multicolumn{3}{c}{Median} & & \multicolumn{3}{c}{75th quantile}\\
\multicolumn{1}{c}{(Asset pair)} & $Q_S$ & $K$ & $P$ & & $Q_S$ & $K$ & $P$ & & $Q_S$ & $K$ & $P$ & & $Q_S$ & $K$ & $P$ \\[4pt]
\hline
& & & \\[-5pt]
10 {\footnotesize{(HAL,XOM)}} & 0.59 & 0.55 & 0.53 & & 0.49 & 0.45 & 0.42 & & 0.60 & 0.56 & 0.54 & & 0.70 & 0.66 & 0.65\\[2pt]
15 {\footnotesize{(LYB,NEM)}} & 0.14 & 0.11 & 0.09 & & 0.02 & 0.00 & -0.03 & & 0.14 & 0.12 & 0.10 & & 0.25 & 0.23 & 0.22\\[2pt]
20 {\footnotesize{(AAL,UNP)}} & 0.31 & 0.28 & 0.25 & & 0.19 & 0.17 & 0.13 & & 0.29 & 0.27 & 0.24 & & 0.41 & 0.38 & 0.37\\[2pt]
25 {\footnotesize{(AMZN,TSLA)}}& 0.37 & 0.35 & 0.33 & & 0.24 & 0.23 & 0.21 & & 0.35 & 0.34 & 0.33 & & 0.48 & 0.48 & 0.45\\[2pt]
30 {\footnotesize{(PG,WMT)}} & 0.41 & 0.37 & 0.33 & & 0.29 & 0.25 & 0.20 & & 0.40 & 0.36 & 0.33 & & 0.52 & 0.49 & 0.46\\[2pt]
35 {\footnotesize{(JNJ,MRK)}} & 0.56 & 0.52 & 0.49 & & 0.46 & 0.41 & 0.38 & & 0.57 & 0.52 & 0.50 & & 0.66 & 0.63 & 0.61\\[2pt]
40 {\footnotesize{(JPM,WFC)}} & 0.74 & 0.72 & 0.68 & & 0.67 & 0.64 & 0.61 & & 0.76 & 0.72 & 0.70 & & 0.83 & 0.79 & 0.78\\[2pt]
45 {\footnotesize{(AAPL,AMD)}} & 0.40 & 0.37 & 0.33 & & 0.26 & 0.21 & 0.16 & & 0.40 & 0.36 & 0.33 & & 0.55 & 0.52 & 0.49\\[2pt]
50 {\footnotesize{(DIS,FB)}} & 0.34 & 0.31 & 0.27 & & 0.20 & 0.17 & 0.14 & & 0.33 & 0.30 & 0.26 & & 0.48 & 0.45 & 0.42\\[2pt]
55 {\footnotesize{(D,DUK)}} & 0.76 & 0.72 & 0.70 & & 0.70 & 0.66 & 0.63 & & 0.78 & 0.73 & 0.71 & & 0.83 & 0.79 & 0.78\\[2pt]
60 {\footnotesize{(AMT,PLD)}} & 0.52 & 0.48 & 0.44 & & 0.43 & 0.38 & 0.32 & & 0.53 & 0.48 & 0.46 & & 0.63 & 0.59 & 0.57\\[4pt]
\hline
\end{tabular}\medskip{}
\par\end{centering}
Note: Summary statistics for pairs of assets, within each of the 11
sectors: Energy (10), Materials (15), Industrials (20), Consumer Discretionary
(25), Consumer Staples (30), Health Care (35), Financials (40), Information
Technology (45), Communication Services (50), Utilities (55), Real
Estate (60).
\end{table}
Ideally, one would determine an empirical way to select an optimal
sampling frequency, because the optimal sampling frequency likely
varies over time and across assets. We leave this for future research.
Table \ref{tab:SectorCorr} presents summary statistics for correlations
between stocks in the same sector. For each of the three estimators,
we compute the average, median, and interquartile range across the
1,763 daily estimates. For all pairs, these quantities are similar
for the three estimators. The interquartile range is across days in
the sample that predominately is driven by time-variation in the daily
correlation. So, a similar width for the interquartile range should
not be interpreted as the estimators having similar precision. In
the next subsection, we present results that strongly indicate that
$Q_{S}$ is more precise than $K$ an $P$. While the measurements
are similar for the three estimators, we always have $Q_{S}>K>P$.
This ordering is in line with our theoretical results, that time-varying
volatility induces a bias in $P$ than in $K$, and that $P$ is more
biased than $K$.
Next, we estimate daily correlations between each of the 22 stocks
and SPY. The average, median, and interquartile range (over the 1,763)
estimates are shown in Figure \ref{fig:Daily-Correlations-with-SPY}.
Once again we see that the quantities are similar for the three estimators,
as was the case in Table \ref{tab:SectorCorr}, and once again do
we have $Q_{S}>K>P$ uniformly across all assets and across all measurements.
\begin{figure}[H]
\begin{centering}
\includegraphics[width=1\textwidth]{figures/Empirical/SPY_Assets_cts_3min_corrboxplot}
\par\end{centering}
\caption{Daily correlations between assets and market returns. The three estimators
were applied to 1,763 trading days. The average estimate (dash) and
median estimate (bullet) are shown. The vertical lines present the
interquartile range over the 1,763 daily estimates.\label{fig:Daily-Correlations-with-SPY}}
\end{figure}
\subsection{Intra-day Correlations and Market Beta Estimation}
Estimating betas from high frequency data is an active research area,
see e.g. \citet[2006]{AndersenBollerslevDieboldWu:2005}\nocite{ABDW:2006},
\citet{TodorovBollerslev:2010}, \citet{DovononGoncalvesMeddahi:2013},
\citet{HansenLundeVoev:2014}, and \citet{ReiBTodorovTauchen:2015}.
This literature as also documented substantial time variation in the
betas over time. Recently, \citet{AndersenThyrsgaardTodorov:2021}
(ATT) documented systematic time-variation in betas within the trading
day. Specifically, they estimated betas for rolling windows (spanning
two hours) using 3-minute intraday returns. Their local estimate of
beta is simply a local estimator of the covariance between asset and
market returns divided by a local estimates of the quadratic variation
of market returns. The local (time-of-the day) estimates are averaged
over the 2,243 days in their sample period (2010-2018). Interestingly,
ATT found a great deal of variation in the betas within the day. Some
stocks have increasing betas over the trading day while other other
assets had decreasing betas over the day.
We can use correlation estimators to cast new light on the patterns
in intraday market betas, by decomposing the market beta into correlation
multiplied by relative volatility,
\[
\beta_{i,t}=\rho_{i,t}\times\lambda_{i,t},\qquad\lambda_{i,t}=\frac{\sigma_{i,t}}{\sigma_{0,t}},
\]
where $\rho_{i,t}$ is the correlation between the $i$-th asset and
the market and $\sigma_{i,t}$ and $\sigma_{0,t}$ are the volatilities
for the $i$-th asset and the market, respectively. We will estimated
local market betas using local correlation estimators combined with
estimators of relative volatility. Specifically, we compute $P$,
$K$, and $Q_{S}$ and $\lambda$ using a rolling window with 60 minutes
of high frequency data. We will the investigate how much of the intraday
variation in betas is explained by into intraday variation in correlations
and how much can be ascribed to intraday variation in relative volatility.
We estimate $\rho_{i,t}$ with each of the correlation estimators,
$P,$ $K$, and $Q_{S}$, using a rolling window that spans 60 minutes.
Similarly, we estimate the relative volatility, $\lambda_{i,t}$,
with subsampled range-based estimators with truncations, as defined
by
\[
\hat{\lambda}_{i,t}=\frac{\sum_{j\in I_{t}}\left\llbracket \Delta_{\frac{S}{N}}Y_{\frac{j}{N}}\right\rrbracket _{\nu_{y}}}{\sum_{j\in I_{t}}\left\llbracket \Delta_{\frac{S}{N}}X_{\frac{j}{N}}\right\rrbracket _{\nu_{x}}},\qquad\left\llbracket x\right\rrbracket _{\nu}=\begin{cases}
x & \text{if }|x|<\nu\\
0 & \text{otherwise}
\end{cases}\qquad I_{t}=[\lfloor tN\rfloor-W+S,\lfloor tN\rfloor],
\]
where $\nu_{x}$ and $\nu_{y}$ are adaptive thresholds for jump truncation.
These are defined by $\nu_{x}=4\sqrt{\mathrm{BV}_{x}}/n^{0.49}$ and
$\nu_{y}=4\sqrt{\mathrm{BV}_{y}}/n^{0.49}$, where $\mathrm{BV}_{x}=\frac{\pi}{2}\sum_{j=2}^{n}|\Delta_{\delta}X_{j\delta}||\Delta_{\delta}X_{(j-1)\delta}|$
is the jump robust bipower variation estimator of daily integrated
volatility, see \citet{barndorff-shephard:2004BiPower}, and $\mathrm{BV}_{y}$
is defined analogously. We have explored estimation of relative volatility
using the bipower variation measures, with the same thresholds. These
estimated we virtually identical to those of $\hat{\lambda}_{i,t}$.
In our implementation, we have $N=23,400$, $W=3,600$, and $S=180$.
This results in 3,421 overlapping 3-minute returns within each hour
we use to compute the subsampled quantities.
\begin{figure}[H]
\begin{centering}
\subfloat[Correlation (AAPL,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AAPL_cts_corr_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Volatility ratio (AAPL,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AAPL_cts_relvol_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Market $\beta$ (AAPL,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AAPL_cts_beta_kn12}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Correlation (AMD,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AMD_cts_corr_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Volatility ratio (AMD,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AMD_cts_relvol_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Market $\beta$ (AMD,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_AMD_cts_beta_kn12}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Correlation (LYB,SPY)]{\begin{centering}
\includegraphics[width=0.33\textwidth]{figures/Empirical/SPY_LYB_cts_corr_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Volatility ratio (LYB,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_LYB_cts_relvol_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Market $\beta$ (LYB,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_LYB_cts_beta_kn12}
\par\end{centering}
}
\par\end{centering}
\begin{centering}
\subfloat[Correlation (NEM,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_NEM_cts_corr_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Volatility ratio (NEM,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_NEM_cts_relvol_kn12}
\par\end{centering}
}\negthinspace{}\subfloat[Market $\beta$ (NEM,SPY)]{\begin{centering}
\includegraphics[width=0.32\textwidth]{figures/Empirical/SPY_NEM_cts_beta_kn12}
\par\end{centering}
}
\par\end{centering}
\caption{Intraday correlations (left panels), relative volatilities (middle
panels), and market $\beta$ (right panels) for four assets, AAPL,
AMD, LYB, and NEM, relative to SPY that tracks the S\&P 500 index.
Quantities are estimated with a rolling window of data that spans
the 60 minutes leading up to the indicated timestamp. Estimates are
averaged over the days in the sample period. The estimates of market
betas using the methodology in ATT are included in the right panels.\label{fig:RhoLambdaBetaIntraday}}
\end{figure}
Intraday estimates of correlations, relative volatilities, and betas
are show for four assets in Figure \ref{fig:RhoLambdaBetaIntraday}.
All quantities are estimated using a rolling window that spans 60
minutes return. The time-stamp used along the x-axis refers to the
end of the 60 minute period. The estimates are averaged over the 1,763
trading days in the sample. The left panels report the intraday correlation
estimates for $P$, $K$, and $Q_{S}$. A horizontal dashed line indicate
the average correlations over the trading hours. The middle panels
report the relative volatility as defined by $\hat{\lambda}_{i,t}$
above, which is multiplied by the three correlation estimates to obtain
estimates of market betas. These three intraday market betas are shown
in the right panels along with the regression based estimate, based
on the same methodology as ATT. The corresponding results for all
assets in the Small Universe is presented in the Supplementary Material,
Figure \ref{fig:RhoLambdaBetasSmallUniverse}.
We note that the time variation in the estimates of $Q_{S}$ tends
to be smoother than those of $K$ and $P$. This strongly suggests
that $Q_{S}$ is a more accurate than $K$ and $P$. We also note
that $Q_{S}$ tends to be slightly larger than $K$ and $P$, which
may be related to them having a larger variance causing another source
of bias in $K$ and $P$. These smoother lines for $Q_{S}$ and slightly
larger values carries over to the intraday estimates of market betas.
The lines for the regression based estimates of intraday betas are
also less smooth that those based on $Q_{S}$.
We find correlations to be generally increasing over the day, while
relative volatilities are decreasing. Whether their product, the market
beta, is increasing or decreasing will depend on which of the terms
changes the most. Unlike correlations and relative volatility, the
paths for intraday market betas take many different shapes. Some assets
have clearly increasing market beta over the day (e.g. NEM), others
have decreasing market betas (e.g. AMD), and a third group of assets
have market betas that goes both up and down (e.g. AAPL), or stay
relatively flat for a large part of the day (e.g. LYB). It is interesting
to compare the market betas for LYB and NEM, which are both Materials
sector stocks, with trading intensity below the average for stocks
in the Small Universe. Despite these commonalities the intraday beta
patterns for LYB and NEM are very different. The reason can be found
in their intraday correlations. For LYB the correlation only increases
by about 50\% over the trading hours (from about 0.37 to 0.55), whereas
the correlation for NEM increases by nearly 500\% (from about 0.05
to 0.25). A great variety of shapes for time-varying betas are shown
in Figure \ref{fig:RhoLambdaBetasSmallUniverse} in the Supplementary
Material.
\subsection{Decomposing Intraday Variation in Market Betas }
All estimated correlations are positive, we can therefore factorize
the logarithm of intraday market betas, as
\[
\log\beta_{i,t}=\log\rho_{i,t}+\log\lambda_{i,t}.
\]
We use this decomposition to investigate who much of the intraday
variation in market betas can be ascribed to changes in correlations
and changes in relative volatilities. For this purpose, we expand
this part of our analysis to include all assets in the Large Universe.
\begin{figure}[H]
\begin{centering}
\subfloat[]{\begin{centering}
\includegraphics[width=0.49\textwidth]{figures/Empirical/DeltaRhoDeltaBeta_scatterplot}
\par\end{centering}
}\negthinspace{}\subfloat[]{\begin{centering}
\includegraphics[width=0.49\textwidth]{figures/Empirical/DeltaLambdaDeltaBeta_scatterplot}
\par\end{centering}
}
\par\end{centering}
\caption{Scatterplot of changes in intraday betas plotted against changes in
correlations (a) and changes in relative volatility (b). Changes are
defined as the difference between the last hourly estimate of the
trading day and the first hourly estimate of the trading day.\label{fig:DeltaRhoDeltaLambda-vs-DeltaBeta}}
\end{figure}
In Figure \ref{fig:DeltaRhoDeltaLambda-vs-DeltaBeta} we have plotted
changes in intraday market betas against intraday changes in intraday
correlations and against changes in relative volatility, for all assets
in the Large Universe. We use color codes to indicate the GICS industry
sectors for each of the assets. The increments (changes) are defined
by the logarithmic difference between the estimate from the first
hour of trading and the analogous quantity from the last hour of trading.
When changes in intraday betas are plotted against intraday changes
in correlations, it reveals a strong linear relationship between the
two, see left panel of Figure \ref{fig:DeltaRhoDeltaLambda-vs-DeltaBeta}.
In contrast there is only a weak relationship between changes in market
betas and changes in relatively volatilities. In fact, as can be seen
from the range of the x-axis in the right panel of Figure \ref{fig:DeltaRhoDeltaLambda-vs-DeltaBeta},
there is far less cross-sectional variation in the changes in relative
volatility. Overall, we can see that most of the cross sectional variation
in market betas can be ascribed to variation in intraday correlations.
Interestingly, there is a great deal of clustering by sectors in Figure
\ref{fig:DeltaRhoDeltaLambda-vs-DeltaBeta}. In terms of changes in
intraday correlations, there is a large degree of sector-specific
separation, whereas in terms of changes in relative volatilities there
are notable variation within all sectors, as evident by asset dispersion
along the x-axis.
In the Supplementary material, Figure \ref{fig:DeltaRhoLambdaBeta-vs-BetaSizeBTM},
we have explored the intraday variation in greater details. For instance,
we plot changes in intraday market correlations, relative volatility,
and market betas against Fama-French type variables. We do not detect
a strong association with key characteristics such as market capitalization
and book-to-market ratios.
In Figure \ref{fig:MarketBetaScatterPlots}, we present scatterplots
of the intraday market betas against conventional market betas, which
are computed from daily returns. The left panel has the market beta
for the first hour of trading and the right panel has the market beta
for the last trading hour plotted against the low frequency market
beta. Not surprisingly, do we find a strong relationship between the
low-frequency market betas and the intraday market betas. The scatterplots
in Figure \ref{fig:MarketBetaScatterPlots} corroborates findings
in ATT, who found market betas to be less disperse at the end of the
day, than the beginning of the day.
\begin{figure}[H]
\begin{centering}
\subfloat[First hour]{\begin{centering}
\includegraphics[width=0.49\textwidth]{figures/Empirical/FirstHourBeta_DailyBeta}
\par\end{centering}
}\negthinspace{}\subfloat[Last hour]{\begin{centering}
\includegraphics[width=0.49\textwidth]{figures/Empirical/LastHourBeta_DailyBeta}
\par\end{centering}
}
\par\end{centering}
\caption{Market beta for the first and last hour plotted against the conventional
market beta, which is computed from daily returns.\label{fig:MarketBetaScatterPlots}}
\end{figure}
\section{Concluding Remarks}
The correlation coefficient is a fundamental measure of linear dependence
with broad applications across various fields of empirical analysis.
For instance, in modern finance, it has a central role in risk management,
portfolio selection, and the pricing of derivatives.
In this paper, we have introduced a novel robust correlation estimator
that is particularly well-suited for high-frequency financial data
analysis. We have shown that the sample correlation, $P$, and Kendall's
tau, $K$, are inconsistent under time-varying volatility, while the
quadrant estimator is robust to time-varying volatilities. The subsampled
quadrant estimator, $Q_{S}$, inherits the consistency of the quadrant
and is far more efficient, because it leverages additional high-frequency
data. The theoretical properties we established for the estimators
are supported by simulation-based evidence and an extensive empirical
analysis spanning seven years of high-frequency return data for about
100 securities.
The empirical analysis also offers valuable insights into the time-varying
nature of market betas within a trading day. Market betas can be expressed
as the product of the correlation (with market returns) and relative
volatility. We have documented that the time-variation in market betas
within the day is mainly driven by time-variation in intra-day correlations.
While the estimator, $Q_{S}$, is particularly well-suited for high-frequency
financial data analysis, it may also be useful for other time-series
with time-varying volatility, or time series that are prone to outliers
and noise. The $Q_{S}$ estimator might also be useful for nonparametric
estimation of the leverage leverage effect, as analyzed in \citet{KalninaXiu:2017}.
There are several ways the $Q_{S}$ estimator could be extended and
possibly improved. For instance, there might be more efficient ways
to handle zero returns, such as distinguishing between cases were
both returns are zero and cases were just one of the returns is zero.
A multivariate version of the $Q_{S}$ estimator would be interesting
to explore. Constructing a correlation matrix from univariate correlation
estimates, need not result in a positive definite matrix. So, a subsequent
matrix projection to the set of positive definite correlation matrices
might be needed, such as those proposed by \citet{Higham:2002} and
\citet{QiSun:2006}.