Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
124,557 characters · 25 sections · 62 citation commands
A multifactor regime-switching model for inter-trade durations in the limit order market
\newgeometry{top=1.25in,left=1in,right=1in,bottom=1.25in}
\tnotetext[t1]{All co-authors agree with the contents of the manuscript. We thank to the National Science Foundation, DMS-1612501 for financial support. Declarations of interest: none.}
\ead{[email removed]}
\ead{[email removed]}
\ead{[email removed]}
\address[HNU]{Center of Economics, Finance and Management Studies, Hunan University, China} \address[SBU]{Department of Applied Math, Stony Brook University, United States} \address[CUHK]{Institute of Data and Decision Analytics,The Chinese University of Hong Kong, Shenzhen, China }
\fntext[fn1]{Zhicheng Li and Haipeng Xing are equally contributed as the first author.} \cortext[cor1]{Corresponding author: Xinyun Chen, Institute of Data and Decision Analytics, The Chinese University of Hong Kong, Shenzhen, 2001 Longxiang Road, Longgang District, Shenzhen, China, 86-0755-23517025.}
In the last two decades, there has been a rapid development of algorithmic trading and high-frequency trading in financial markets. The evolution of the market from human involvement to computer control, from operating in time frames of minutes to time scales of microseconds, has changed the market in a fundamental way (o2015high). In this paper, we document a bimodal phenomenon that is prevalent in the distributions of inter-trade duration for stocks listed in the NASDAQ limit order market. The two modes are significantly different in their scales: one is of the order of $10^{-4}$ seconds, and the other is of the order of $10^0$ seconds. To the best of our knowledge, our paper is the first to document such a phenomenon in high-frequency duration data. This phenomenon, together with other empirical evidence, suggests that there are two regimes associated with the dynamics of inter-trade duration, and the regime switchings, therefore, reflects the dynamics of the market microstructure in a high-frequency world.
To better understand the bimodal phenomenon, we propose a two-state multifactor regime-switching duration (MF-RSD) model with transition probability matrices depending on the information in the limit order book. We study the properties of the model, including the ergodicity, bimodality, and exogeneity, and use the expectation-maximization (EM) algorithm to make an inference on the model parameters. Simulation studies validate our estimation method and show that the MR-RSD model can successfully generate the bimodal distribution of inter-trade durations. Then, we use the proposed model to study the inter-trade durations in the NASDAQ limit order market. The results shows that the MF-RSD model not only fits the data well and but also has a good out-of-sample performance in predicting the inter-trade durations. We use 5000 in-sample data to estimate the model parameters and then do 1-step-ahead forecasting of the next inter-trade duration for each single stock. A benchmark comparison with some classical duration models, such as the autoregressive conditional duration (ACD) model by engle1998autoregressive and the Markov-switching multifractal inter-trade duration (MSMD) model by chen2013markov, shows that the our-of-sample performance of our MF-RSD model is better than that of the other two in predicting the arrival time of the next trade.
We have analyzed the underlying regimes that are estimated from the MF-RSD model and find that these two regimes are related to the endogenous liquidity provision and consumption of HFTs. Specifically, we use the permanent price change and the realized price spread to represent the profit of taking and providing liquidity, and see their relationship between the underlying regimes. Our results show that the profit of taking liquidity is large when the market in the short-duration regime and the HFTs are probably using aggressive trading strategies, while in the long-duration regime when the HFTs mainly act as the passive market makers, the profit of providing liquidity is relatively large. Moreover, we regress the short-term price change on the regime levels and the regime-switching probabilities and find that a large price change happens in a short time after the short-duration regime, and there will be a stronger price movement if the market just switches from the long-duration regime to the short-duration regime. This result suggests that the HFTs change the trading strategy to taking liquidity because they foresee that a large price change is going to occur subsequently.
We also study some economic hypotheses in the market microstructure theory by applying the MF-RSD model to 25 Nasdaq stocks in a whole month. In particular, we investigate the sensitivity of the transition probabilities of inter-trade durations in the MF-RSD model to various LOB variables and summarize which types of market information the traders or their algorithms learn from and react to in the LOB. Most of the results are consistent with the existing findings in the literature. For instance, 1) order book depth imbalance is an indicator for price movement and a large depth imbalance would lead HFTs to actively trade for earning a profit, which causes short inter-trade durations; 2) HFTs are more likely to provide liquidity as market makers when price spread is large. At that time, market orders are initiated by non-HFTs and the inter-trade duration would be more likely in the long-duration regime; and 3) a large trade volume reflects a strong trading willingness on one side of the market, which is a signal of informed trading. Therefore, HFTs would actively follow up and trade when they observe the volume of the last trade is very large.
Besides, several new findings are obtained. First, the impact of order book depth imbalance on the regime switchings of inter-trade durations is significant (positive for inter-trade durations switching from the long-duration regime to the short duration regime and negative for the opposite direction) mostly for the stocks whose price spread is tight, while it is not obvious for the stocks whose spread is slack. This is because the depth imbalance between best ask and best bid is only informative for price movement when the price spread between them is small. Second, the effect of price spread is asymmetric. A large price spread tends to keep inter-trade duration stay in the long-duration regime if the preceding inter-trade duration is long. However, a large price spread is not a significant signal which leads the inter-trade duration to switch into the long-duration regime from the short-duration regime. Third, an increase in price spread, which is contemporaneously associated with a change in mid-price, is shown to have a significant effect on the regime switching of inter-trade durations, i.e., it is significantly negative for inter-trade durations switching from the long-duration regime to the short-duration regime and significantly positive for inter-trade durations switching from the short-duration regime to the long one.
The remainder of the paper is organized as follows. Section 2 presents the related literature. Section 3 discusses the data we use and provides the empirical facts of inter-trade durations, in particular, the bimodal distribution of inter-trade durations. In Section 4 we introduce our MF-RSD model and discuss the model's implied properties. The estimation method and a simulation study are also given in Section 4. In Section 5 we implement the empirical analysis of the NASDAQ limit order market and summarize the main findings. Section 6 gives our concluding remarks.
Over the past twenty years, many financial duration models have been proposed. Most of them aim to capture the empirical properties of inter-trade durations. One pioneering work is the ACD model by engle1998autoregressive, which expresses the conditional expectation of duration as a linear function of past durations and past conditional expectations of durations, and successfully explains the observed clustering effect of inter-trade durations. After that, many varieties of ACD extensions were developed, such as the logarithmic ACD model by bauwens2000logarithmic, the threshold ACD model by zhang2001nonlinear, the Markov-switching ACD model by hujer2002markov and the stochastic conditional duration (SCD) model by bauwens2004stochastic. These models can generate a rich collection of nonlinear dynamics that fit different duration series. Recently, chen2013markov proposed the MSMD model, in which the stochastic intensity can be decomposed into multiple fractals and each of them follows a distinct hidden Markov process, capture the long memory property of inter-trade durations that has been discussed in both the theoretical and applied literature jasiak1999persistence,diebold2001long,DEO20103715.
Another strand of papers considers other variables' effects on the dynamics of inter-trade durations by incorporating them into the modeling, which shed light on our work. For instance, engle2000econometrics and bauwens2003asymmetric have jointly modeled the inter-trade durations and other events of interest, such as the price process. The MF-RSD model in our paper has a similar spirit as the regime-switching model in diebold1994regime, and we link the regimes of inter-trade durations with the LOB variables via time-varying transition probabilities. The method of making the Markov transition matrix time-varying and dependent on some variables is also employed in kim2008estimation, kang2014estimation, and chang2017new, while they further considered and addressed a potential endogeneity problem in which the innovations in determining regime switching are possibly correlated with the observed time series.
Our work is also built on some works which study information learning in financial markets. In traditional market microstructure theory, traders learn information about security from market data, especially the trade-related information kyle1985continuous, glosten1985bid,hasbrouck1991measuring. However, in the high-frequency world, the basic unit of market information becomes limit orders rather than trades hautsch2012market, o2015high, lo2015resiliency. Many papers have studied the informativeness in the LOB both empirically and theoretically. harris2005information showed that the limit orders in the NYSE are informative about price changes and that NYSE specialists would use this information to provide liquidity. cont2014price investigated the price impact of order book events using NYSE TAQ data and found that price changes are driven by the order book imbalance. lipton2013trade and cartea2018enhancing studied the informativeness of order book imbalance in detail and found that it is a good predictor for the arrival of trades. Recently, aliyev2018learning build a market microstructure model in which the liquidity providers not only learn about the fundamental value of the asset but also learn about the extent of informed tradings through order flow imbalance. The model theoretically explains why order imbalance sometimes destabilizes markets and provides more understanding on the dynamic process of how markets digest order imbalance.
One more strand of research related to our paper focuses on HFTs' trading strategies and the influences of LOB factors on them. Some studies menkveld2013high, hagstromer2013diversity, LI2005533 mentioned that a significant proportion of high-frequency traders (HFTs) employ market-making strategies and, consequently, are very sensitive to the transient fluctuation in market liquidity. On the other hand, some papers brogaard2014high, HFreview pointed out that HFTs would also use aggressive directional strategies when they anticipate the direction of order flow or price movement in the short run. Furthermore, several studies have shown that HFTs switch the trading strategies between supplying liquidity and consuming liquidity according to different LOB statuses. For example, carrion2013very found that HFTs are good at timing and hence provide liquidity when the price spread is large and take liquidity when the spread is small. goldstein2017high showed that HFTs condition their strategy on order book imbalance, i.e., they supply liquidity on the thick side of the order book and demand liquidity on the thin side. van2019high found that HFTs trade with large institutional orders, i.e., they initially lean against these orders but eventually change direction and take a position in the same direction as the most informed institutional orders. Recently, foster2019microliquidity used a new market microstructure model to explore a fragile liquidity when HFTs switch from liquidity provision to liquidity consumption on the basis of unexpected information signals.
Our data are downloaded from LOBSTER (\href{https://lobsterdata.com/}{https://lobsterdata.com/}), which provides high-quality LOB data for all NASDAQ stocks since June 2007, based on NASDAQ's Historical TotalView-ITCH data. LOBSTER simultaneously generates two files. One is a `message' file, which contains indicators for the type of event causing an update of the LOB in the requested price range, with decimal precision in nanoseconds ($10^{-9}$ second). The other file is an `order-book' file that records the evolution of the LOB up to the requested number of levels at the time when the `message' file is updated. Through the `message' file, we can easily calculate the time length between two consecutive trades, i.e., the inter-trade duration. And from the `order-book' file, we are able to construct various LOB variables.
The sample period of our study is January 2013, which contains 21 trading days, and we have randomly selected 25 stocks from the NASDAQ-100 index with various market capitalizations. A simple description of these selected stocks is presented in Table (ref) in the Appendices. Because the frequency of our data is ultra-high, and we are particularly interested in the intra-day dynamics of inter-trade durations, we will analyze each stock in each trading day separately. To give a clear illustration, we summarize our results for the 25 stocks but provide detailed results for the Microsoft Corporation (MSFT) in the rest of our paper.
In Figure (ref), we first plot the series of MSFT inter-trade durations on January 2nd, 2013.\footnote{As the inter-trade duration series exhibits a significant intra-day calendar effect, which will be elucidated in Section V, the data that we use throughout the paper are the adjusted inter-trade durations that have been adjusted by the intra-day calendar effect.} There are more than 6000 inter-trade durations on that day. Some of them are extremely short and in the order of $10^{-6}$ seconds, and some of them are very long and in the order of 100 seconds. We perform a histogram of these inter-trade durations in the first subplot of Figure (ref), which shows a heavy tailedness and that the majority is less than seconds. In fact, the inter-trade durations that are less than 1 second account for 65% of the total share. Hence, in the second subplot of Figure (ref), we scrutinize the inter-trade durations by checking the histogram of their common logarithms and find a bimodal distribution with two distinct modes. Moreover, the inter-trade durations of the remaining 24 stocks also exhibit similar bimodal distributions, and we will give a detailed discussion in the next subsection.
The bottom left panel of Figure (ref) presents an exponential duration quantile-quantile (Q-Q) plot for the MSFT inter-trade durations, which indicates a nonexponential duration distribution with a long right tail and that the distribution is over-dispersed (i.e., the standard deviation exceeds the mean), in contrast to the equality that would be obtained if the trade events follow a classical point process and the durations were exponentially distributed haipeng2016durations. In particular, the mean of MSFT inter-trade durations is 4.07 seconds, while the standard deviation is 9.54 seconds. Another empirical property of inter-trade durations is the clustering effect, which has been discussed in engle1998autoregressive, jasiak1999persistence, and laixing2008. Here, we illustrate this effect by showing a random sample of 300 consecutive inter-trade durations in Figure (ref), from which we can see that the clustering effect appears in a manner in which short (or long) inter-trade durations are followed by short (or long) inter-trade durations.
We have investigated the distributions of inter-trade durations for all 25 stocks in our sample. Figure (ref) shows the histograms of their common logarithms,\footnote{It plots the histograms for the calendar-effect-adjusted inter-trade durations on the same trading day, i.e., January 2nd, 2013. In the Appendices, we also provide the histograms for the raw inter-trade durations and the aggregated inter-trade durations in one month. They all show bimodal distribution.} and they all exhibit a similar bimodal pattern. To locate the two modes, we fit these empirical inter-trade duration distributions by a mixture of inverse Gaussian distributions. The results are plotted in Figure (ref), from which we can see that the left modes are approximately $10^{-4}$ s, and the right modes are located between $10^{-1}$s and $10^{1}$ s. To the best of our knowledge, our paper is the first to document such a phenomenon in high-frequency duration data. This newly found bimodal distribution of inter-trade durations, together with the clustering effect we have shown, suggest that there may be two regimes associated with the dynamics of inter-trade durations, corresponding to the quick and slow trading periods.
Based on hasbrouck2013low, which shows that the high-frequency trading firms have effective-latency in milliseconds by using NASDAQ data in the year 2007, we think that the clustering of very short inter-trade durations (most of the time below $10^{-1}$ s) is caused by HFTs. Moreover, according to some literature mentioned in Section 2 brogaard2014high,carrion2013very, we conjecture that the regime switchings between the long inter-trade durations and the short inter-trade durations are driven by HFTs' behaviors of changing trading strategies for profitability. When the market is relatively stable, HFTs mainly use market making strategies to earn profit from the price spread between the best ask and the best bid.\footnote{Sometimes HTFs' profit is supplemented by a rebate from exchange for resting liquidity.} At that time, trades are mostly initiated by non-HFTs and the time interval between two trades would be relatively long. While when HFTs receive some signal that probably indicates a short-term price movement, they may switch to aggressive trading strategies and trade in the direction of price movement to gain profit. Consequently, the market enters a regime in which HFTs consume liquidity and the inter-trade durations become very short.
Given the empirical properties and the bimodal distribution of inter-trade durations, we propose the following multifactor regime-switching duration (MF-RSD) model with two hidden states. Let $t_{i}$ be the calendar time for the $i$-th trade in a trading day for one particular stock, where $t_{0}<t_{1}<\cdots<t_{n}$ and $t_{0}$ is the opening time for the market. The $i$-th inter-trade duration is defined as \[ y_{i}=t_{i}-t_{i-1}, \] and there are two regimes for the inter-trade durations. Under each regime, $y_i$ is an i.i.d. random variable drawn from a prespecified distribution, i.e.,
$s_i$ is a two-state Markov chain with a probability transition matrix $$ \mathbb{P}_{i}=\left(
\right), $$
in which $P_{jk,i} = P(s_{i}=k | s_{i-1}=j)$ is the transition probability from state $j$ to state $k$ (where $j,k=1,2$ ) for the $i$-th trade.
We think the transition probabilities in our model are time-varying and depend on some exogenous variables. Denote ${\bf x}_{i-1}$ the vector that contains some lagged variables which have been realized on or before the $(i-1)$-th trade, and we concretely assume that the $P_{jk,i}$ satisfies the following logistic forms,
Hence, multiple factors can be incorporated into the model to consider their effects to the regime switchings of inter-trade durations. In our study, we mainly focus on the impact of LOB factors on the dynamics of inter-trade durations. Furthermore, considering the range of inter-trade durations and the simplicity of estimation, we assume that the distribution $f_k(y)$ $(k=1,2)$ is inverse Gaussian (IVG). Specifically, assume that conditional on state $k$, the probability density function for $y_{i}$ is as follows:
where $\mu_k$ and $\lambda_k$ are the respective mean and shape parameters for two IVGs. Suppose that $s_i=1$ corresponds to the short-duration regime, and we should have $\mu_1<\mu_2$.
Compared with other inter-trade duration models, our MF-RSD model has several advantages in modeling high-frequency duration data. First, it uses a parsimonious structure to directly capture the observed bimodal distribution and clustering of inter-trade durations, which helps us to understand the dynamics of market more intuitively. Second, by introducing the covariates that potentially determine the evolution of inter-trade durations, we can improve the model's predictive power by using the values of covariates that are realized before the $i-$th inter-trade. Finally, given the identified relationships between the covariates and the arrival time for trades, the MF-RSD model can also be used to test some economic hypotheses on the limit order market.
Here we study some implied properties of the MF-RSD model. We begin by showing the stationarity, ergodicity, and over-dispersion. Then, we present the conditions under which the MF-RSD model can generate the bimodality. Finally, we discuss the rationality of assuming that the covariates that determine the regime switchings of inter-trade durations are exogenous.
If we have the factor series $\mathbf{x}_i$ is a stationary sequence in a compact set, by the Theorem 5.5 of orey1991markov, we can conclude that the underlying regime state $s_i$ for inter-trade duration is stationary and ergodic in the sense that $\lim_{i\to\infty}P(s_{i}=k|s_{0}=j)\to\pi_{k}$ for $j,k\in\{1,2\}$, with $\pi_{k}\geq0$ and $\sum_{k}\pi_{k}=1$. In particular, $\pi=(\pi_{1},\pi_{2})'$ satisfies that $\pi P=\pi$, $P=E[\mathbb{P}(\mathbf{x}_i)].$
Moreover, let $s^{*}$ follows the stationary distribution of $s_{i}$, then we can write the stationary distribution of $y^{*}$ as $y(s^{*})$. Denote the random variable $y(k)\sim f(\boldsymbol{\theta_{k}})$ and define the dispersion $d_{k}=\frac{\text{Var}(y(k))}{E[y(k)]^{2}}$. Suppose that $E[y(k)]$ are not equal for $k=1,2$. Then we have the dispersion of $y^{*}$ that, $d=\frac{\text{Var}(y^{*})}{E[y^{*}]^{2}}>\min_{k}d_{k}.$ Because
The last inequality follows the Jensen's Inequality. Hence, if the individual $f(\boldsymbol{\theta_{k}})$ already has over-dispersion, the stationary distribution of $y^{*}$ must be over-dispersed.
A bimodal distribution mostly arises as a mixture of two different unimodal distributions; however, certain requirements need to be met. According to the Theorem 2 in dovsla2009conditions which studies a mixture of two general unimodal densities, we have derived the following proposition for a mixture of two IVGs:
The proof for Proposition 1 is given in the Appendices. In our MF-RSD model, the above conditions are easy to be satisfied. The stationary probability for $s_i=1$ is the mixture ratio $p$. The IVG under the short-duration regime has a smaller mode $M_1$. Specifically, we can assume $\mu_1<\mu_2$ and $\lambda_1<\lambda_2$ \footnote{They are consistent with our empirical results}. Hence, when $y$ is between $M_1$ and $M_2$ (it is certainly greater than $N_1$ and $N_2$, which are negative), $R(y)$ in (ref) is very likely to be negative, especially when $\mu_1<<\mu_2$. Moreover, as in (ref), $\lim_{y\to M_1}R(y)=+\infty$ and $\lim_{y\to M_2}R(y)=+\infty$, $R(y)$ will first decrease to a negative value from $M_1$ to $y^{\star}$ and increase back to positive infinity when $y$ approaches $M_2$. Therefore, there must be two roots for the equation $R(y)=0$ in the interval $(M_1,M_2)$, which are $y_1$ and $y_2$. The mixture ratio $p$, which is the stationary probability for $s_i=1$, should be in the range $(p_1,p_2)$.
In our MF-RSD model, we assume that the covariates that drive the dynamics of inter-trade durations are exogenous. This assumption is reasonable in the following two senses.
First, according to hautsch2011econometrics, the event probability of a point process $N(t)$ depends on information that is (at least instantaneously) known prior to $t$. If variables are known and constant prior to an event $i$ , we call them time-invariant covariates, and they can be considered weakly exogenous. Conversely, the time-varying covariate process $\boldsymbol{x}(t)$ can continuously change between two consecutive events. To be weakly exogenous for $y_{i}$, the process $x(t)$ must be c\`adl\`ag with $\boldsymbol{x}_{\breve{N}(t)+1}=\boldsymbol{x}(t_{i-1})$ for all $t$ with $t_{i-1}<t\leq t_{i}$. In our case, the duration event $y_{i}$ is defined as $y_{i}\coloneqq t_{i}-t_{i-1},$ and $P_{jk,i}=P(s_{i}=k|s_{i-1}=j)$ represents the switching probability for the underlying state of event $i$ changing from $j$ to $k$ (here, $j,k\in\{1,2\}$). Thus, the factors $\mathbf{x}_{i-1}$ that have been realized on or before $t_{i-1}$ can be considered weakly exogenous to the occurrence of event $i$.
Second, if we think the factors $\mathbf{x}_{i-1}$ which represent the LOB information up to $(i-1)$-th trade are timely revealed to market participants, especially to the HFTs. Then, their actions are timely made conditional on the market information. As the occurrence of trade is triggered by just one action of a single trader, the covariates $\mathbf{x}_{i-1}$ that were realized on or before the $(i-1)$-th trade can be considered as exogenous to the decision process of the market participant who initiates the $i$-th trade.
The direct maximization of the likelihood function of the observations is difficult to implement, as we cannot observe the underlying states $s_{i}$ and $\{s_{i}\}_{i=1}^{n}$ has $2^{n}$ possible realizations. Hence, we adopt the standard expectation-maximization (EM) algorithm. In the E-step, given the information of $\mathbf{x}_{1:n}=\{\mathbf{x}_{i-1}\}_{i=1}^{n}$ and the model's old parameters $\Theta^{(m)}$ ($m=1,2,\ldots$ is the number of iterations), the expected complete-data log-likelihood is
where $\mathcal{F}$ represents information generated by $\{\mathbf{x}_{i},y_{i}\}_{i=1}^{n}$ and old parameters $\Theta^{(m)}$. $\rho$ is the conditional stationary probability for state 1, i.e., $\rho=\mathbb{P}(s_{1}=1|\mathcal{F})$. The conditional posterior distributions $\mathbb{P}(s_{i}=2,s_{i-1}=1|\mathcal{F})$, $\mathbb{P}(s_{i}=1,s_{i-1}=2|\mathcal{F})$, $\mathbb{P}(s_{i-1}=1|\mathcal{F})$, and $\mathbb{P}(s_{i-1}=2|\mathcal{F})$ can be calculated using the the forward-backward algorithm \footnote{diebold1994regime has shown the steps to calculate and implement this forward-backward algorithm.}.
Then, in the M-step, we update the model parameters by maximizing the expected log-likelihood (ref),
We repeat the E-step and M-step until the model parameters converge, i.e.,
By plugging the density function (ref) into the expected log-likelihood function (ref), the maximization step (ref) yields the following analytical expressions for the model parameters \footnote{Detailed derivations are shown in the Appendices.}:
and the regression coefficients $\boldsymbol{\hat{\beta}}_{12}$, $\boldsymbol{\hat{\beta}}_{21}$ satisfy the following equations:
Given the assumption that the covariates $\mathbf{x}_{i-1}$ are exogenous to the duration process and inter-trade duration $y_i$ is i.i.d conditional on the underlying state $s_i$, the maximum likelihood (MLE) will obtain consistent estimators, and they asymptotically converge to the true parameters. However, we adopt the EM algorithm instead of directly maximizing the likelihood because of the complexity in constructing the log-likelihood function in the presence of underlying hidden states. Hence, the estimation consistency relies on the convergence of the EM estimator to the global maximizer of the log-likelihood function.
Define the log-likelihood for the observed data as $L(\Theta)=L(Y|\mathbf{X},\Theta)$, in which $Y:=y_{1:n}$ and $\mathbf{X}:=\mathbf{x}_{1:n}$. Then, the MLE estimator directly maximizes $L(\Theta)$, and the EM algorithm iteratively increases the value of $L(\Theta)$ at each step by maximizing the expected complete-data log-likelihood in (ref), which can be redefined as
According to wu1983convergence, if the unobserved complete-data specification can be described by an exponential family, the function $Q(\psi | \phi)$ is continuous in both $\psi$ and $\phi$. Then, the limits of the EM sequence are the stationary points (local maxima) of the log-likelihood $L(\Theta)$. This condition obviously holds, as the IVGs belong to the exponential family and the transition probabilities are in logistic form. Moreover, if $L(\Theta)$ is unimodal, the EM estimator converges to the global maximizer of $L(\Theta)$.
The variance-covariance matrix of estimators is not a byproduct of the EM algorithm. To obtain that, we use the supplement EM (SEM) method proposed by meng1991using. The SEM method employs the fact that the rate of convergence of the estimators in the EM is governed by the fractions of missing information that would increase the variability of the estimation. In particular, the desired observed variance-covariance matrix $V$, which is also the inverse of the observed information matrix, can be expressed as the sum of the covariance matrix of the expected complete-data, i.e., $\mathbf{I}_{\text{EC}}^{-1}$, and a variance inflation part $\Delta V$,
where $\mathbf{I}_{\text{EC}}$ is the expectation of complete-data information matrix $\mathbf{I}_{\text{C}}(\Theta)$ over the conditional distributions evaluated at $\hat{\Theta}$, which can be calculated in the standard EM method. The variance inflation part $\Delta V$ in (ref) can be written as a function of $DM$ as follows:
where $I$ is simply the identity matrix and $DM$ is the matrix rate of EM convergence, whose calculation is shown in detail in meng1991using. Hence, the standard errors of the estimators are the square roots of the diagonal elements of $V$.
We perform a simulation study in this part to validate our MF-RSD model and the estimation method. We provide an example in which the regime-switching probabilities are driven by a one-dimensional factor $x$, which is piecewise constant and takes alternative values among $\{0.4, -0.3, 0.6, -0.5\}$, as shown in Figure (ref). Hence, the regime switching probabilities are determined as following:
where $1\le j \neq l \le 2$. The model parameters $\mu_k, \lambda_k, \boldsymbol{\beta_{jl}}$ ($k=1,2$) are shown in Table (ref). These values are chosen so that moderate regime-switching probabilities are generated from the model. We then simulate $y_i$ $(1\le i \le n)$ for $n=9000$ and show the results in Figure (ref). Note that the simulated $y_i$ exhibits a large variation and that the histogram in the bottom left panel shows a clear bimodal distribution, which demonstrates that our model can indeed generate bimodality. We then use the EM algorithm to estimate the model and show the estimated values in Table (ref), from which we can see that the estimates are quite close to the true values with low standard errors, and the confidence intervals of estimates basically cover the true values. We also show the simulated and estimated states $s_i$ $(1\le i \le 1000)$ in the bottom right panel of Figure (ref), from which we find that the difference between the true and estimated states are quite small, and the underlying states capture the clustering of long (or short) inter-trade durations.
In the following, we implement the real data analysis for the inter-trade durations in the NASDAQ limit order market. The inter-trade durations usually contain the intra-day calendar effect, i.e., inter-trade durations tend to be shorter early and late in the day and longer around noon. Based on the standard dummy variable procedure (e.g.,ghysels2004stochastic), we find that that the intra-day calendar effect is a common phenomena for all stocks in our sample (shown in Section (ref) in the Appendices.). Therefore, prior to the analysis, we have adjusted inter-trade durations for the calendar effects.
For each stock in our sample, we aggregate the inter-trade durations in January 2013 and provide the descriptive statistics in Table (ref). These statistics include count, min (the minimum), three quantiles, max (the maximum), mean, standard deviation, skewness, kurtosis, and over-dispersion (O.D.), which is the standard deviation divided by the mean. Table (ref) suggests the following: 1) There is a large variation in their counts and maximum values; 2) The minimum values are approximately $10^{-6}$ seconds, and the mean values are approximately 1-10 seconds; 3) Their median value has a small variation, with most of them are around or below 0.1 seconds; and 4) They all exhibit a large dispersion as the values of their over-dispersion are significantly greater than one.
To identify the factors that may impact the regime switchings of inter-trade durations, we focus the covariates in our model on the LOB factors for the following reasons. First, many studies advocate that the order flows are determined by the state of the order book hautsch2012market,huang2015simulating,abergel2015long; therefore, factors reflecting the state of the LOB are thought to determine the flow of market orders and hence the inter-trade durations. Second, the information flow revealed by the change of LOB contribute a major part of information learning for the market participants harris2005information, cao2009information, especially for the HFTs who take the speed advantage to trade on the information. Third, a lot of research has found that the algorithm traders in the limit order market are indeed using LOB factors as signals to set their trading strategies. LI2005533,obizhaeva2013optimal,hendershott2013algorithmic,goldstein2017high, so these factors must play an important role on the arrival time of trades, i.e., the inter-trade durations.
Let $p_a$ and $p_b$ be the prices (dollar prices divided by 10) for the best ask and the best bid, respectively, and $v_a^1$ and $v_b^1$ be the depth at the best ask and the best bid, respectively. We consider the following four LOB factors as the covariates in our MF-RSD model.
In Table (ref), we show some summary statistics of these LOB factors. The minimum and maximum values of $DI$ are 0 and 1, and the minimum values of $PS$ and $TV$ are always $0.1$ (1 cent) and $0.001$ (1 share), respectively; we do not report these values in the table. Since the factor $PM$ is a dummy variable, we only report its mean, or equivalently the ratio of the times when it equals to 1. From Table (ref), we find that the percentiles of $DI$ are very similar across stocks. The factors $TV$ and $PM$ do have some differences from stock to stock, while the most divergent factor is $PS$. For some stocks, such as CMCSA, FB, INTC, MSFT, TXN, VOD and YHOO, the factor $PS$ are most time fixed at 0.1 (the minimum tick size). But for some other stocks, such as GOOG, ISRG, and REGN, $PS$ varies much and its values can be very large.
We first implement the in-sample analysis for the inter-trade durations of MSFT on January 2, 2013, according to the following regression models for the transition probabilities:
The model is estimated using the EM procedure as described in Section 4 and the results are reported in Table (ref). In addition to the full model - Model 6, we also estimate the submodels (Models 1-5). The distribution parameters estimated from the full model are very close to those from the submodels, and the estimated factor coefficients in the full model are having similar signs and significance levels as those in the submodels, suggesting that there is no strong multicollinearity in the LOB factors. In the rest of the empirical study, we shall only report the results using the full model (M6).
From Table (ref), we can see that the estimated mean of the long-duration regime is much larger than that of the short-duration regimes, i.e., $\hat{\mu}_2>>\hat{\mu}_1$. Based on the estimated values for the parameters of two IVGs, we also calculate the modes of the two distributions. They are around $8\times10^{-5}$ seconds and $1$ second, which are consistent with the bimodal distribution of inter-trade durations discussed in Section 3. Moreover, we have noticed the following results for the LOB factors. First, the estimated coefficients for the $DI$ is significantly positive for ${P}_{12}$ ($\hat{\beta}_{12,1}>0$) and significantly negative for ${P}_{21}$ ($\hat{\beta}_{21,1}<0$). Second, the $PS$ is shown to be both negative for ${P}_{12}$ and ${P}_{21}$ while its coefficient is only significant in ${P}_{12}$. Third, $\hat{\beta}_{12,3}$ is significantly less than 0 and $\hat{\beta}_{21,3}$ is significantly greater than 0, suggesting that the $TV$ is a significant factor for the regime switchings of inter-trade durations. Last, the factor $PM$ seems to be insignificant in this case because both $\hat{\beta}_{12,4}$ and $\hat{\beta}_{21,4}$ are not significantly different than 0. Nonetheless, the LOB factors fairly impact the dynamics of inter-trade duration via regime-switching probabilities, and we expect that our MF-RSD model can enhance the prediction of inter-trade durations by considering their effects. Moreover, we'd like to understand more about the economics implication of the effects of LOB factors on the regime switchings of inter-trade durations, which will be discussed in detail in Section (ref).
Given the estimated values of model parameters, we examine the model prediction power by first considering a 1-step-ahead out-of-sample prediction for the underlying states. The 1-step-ahead prediction of $s_i$ is given by
and the transition probabilities at $i$ are given by
which can be calculated given the information up to $i-$th trade.
We use the MSFT inter-trade duration series on January 2, 2013, to demonstrate the above prediction. The model parameter is estimated using the first 5000 samples, and 1-step-ahead forecasting is made for the next 200 durations. Figure (ref) shows the comparison between the estimated states and the 1-step-ahead forecasts. The estimated states are the underlying states inferred from the real data, and the forecasted states are 1-step-ahead out-of-sample prediction.\footnote{The states jump between two values, i.e., state 1 (the short-duration regime) and state 2 (the long-duration regime)} From the result, we can see that the forecasted states almost capture the variation tendency of the underlying estimated states.
Furthermore, the 1-step-ahead prediction for the inter-trade duration is expressed as
To see the predictive performance of our model, we compare our results in predicting the next inter-trade duration with those of two classical duration models, i.e., the ACD model by engle1998autoregressive and the MSMD model by chen2013markov. The ACD model we considered has the following form: $$ y_{i} =\varphi_{i}\varepsilon_{i}, \qquad \varphi_{i} =\omega+\alpha y_{i-1}+\beta\varphi_{i-1}, $$
where $\varepsilon_{i}$ follows an inverse Gaussian distribution. The MSMD model we considered has five levels, and the following form:
in which $\varepsilon_{i}$ also follows an inverse Gaussian distribution.
Table (ref) shows the comparison among our model, the ACD(1,1) model and the MSMD(5) model. We use the BIC as an indicator to compare their in-sample fitness. From the results, we find that the in-sample fitness of our MF-RSD model is slightly worse than that of the ACD(1,1) model but better than that of the MSMD(5) model. This is probably because the ACD model has the most parsimonious structure and the least number of parameters. For the out-of-sample prediction power, we use the RMSE between the real duration and the predicted duration as a measurement. Nonetheless, from the results in Table (ref), our MF-RSD model performs significantly better than the other two models.
In the next, we want to see the characteristics of the underlying regimes that are estimated from our MF-RSD model. In Figure (ref), we plot the first MSFT 300 inter-trade durations on January 2, 2013, along with their estimated levels of the underlying regimes. It shows that the estimated regimes capture the clustering of the long (or short) inter-trade durations, and there are significant switchings between the long-duration regime and the short-duration regime. Given the estimated underlying regimes, we will diagnose the two regimes and check whether they are related to different channels of profitability for the HFTs, because we think that in the long-duration regime, HFTs mainly provide liquidity and gain profit from the price spread as market makers, while in the short-duration regime, HFTs aggressively take liquidity and earn profit as speculators because of the short-term price movement.
To make things comparable, we divide the trading day into 360 one-minute intervals and calculate the ratio of the number of short inter-trade durations to the total number of trades in each interval, which is defined as the HF ratio. We categorize the short and long inter-trade durations based on which regime they belong to. The LF ratio is then defined as the ratio of long inter-trade durations and hence equals to 1-HF ratio. Thus, a high HF (LF) ratio means that the short-duration (long-duration) regime is dominated in the interval. Following carrion2013very, we use the permanent price change and the realized price spread as the metrics of profit. The permanent price change is defined as the absolute percentage change of midpoint price in the one-minute interval, which measures the profit of taking liquidity, and the realized price spread is the average effective price spread minus the permanent price change, which measures the profit of supplying liquidity. We also include the order submission rate and the cancellation rate to see the market activity level. As the time length of the interval is fixed, we simply use the numbers of submissions and cancellations in one minute to represent the rates.
We still use MSFT on January 2, 2013 as an example. For each one-minute interval, we calculate the HF ratio, the LF ratio, the order submissions, the order cancellations, the permanent price change, and the realized price spread. In Table (ref), we present their correlation matrix, from which we can observe that the HF ratio is positively correlated with the order submissions, cancellations, and the permanent price change, but negatively correlated with the realized price spread. The LF ratio is having the opposite relationships with those 4 metrics. Thus, the profit of taking liquidity is large when the market in the short-duration regime in which the HFTs are using aggressive trading strategies. While in the period with a high LF ratio, i.e., when HFTs mainly supply liquidity, the profit of supplying liquidity is larger as the realized price spread is positively correlated with the LF ratio.
Furthermore, we implement the below regression to see which regime or which regime-switching action is followed by a significant short-term price movement.
In the regression, $\Delta P_{i,t(+10s)}$ is the absolute value of the percentage change of midpoint price in the next 10 seconds after $i-$th trade.\footnote{As we mainly care about the HFTs' behaviors, we think 10 seconds is long enough at the high-frequency level.} $\mathbf{1}_{s_i=1}$ is the indicator which equals to 1 if the $i-$th trade belongs to the short-duration regime. $\hat{P}_{12,i+1}$ is the estimated transition probability for the next inter-trade duration switching from the short-duration regime to the long-duration regime, and $\hat{P}_{21,i+1}$ is the estimated regime transition probability for the next inter-trade duration on the contrary; both of them are calculated by equation ((ref)) according to our MF-RSD model.
The regression results are presented in Table (ref). From it, we can see that the coefficient for $\mathbf{1}_{s_i=1}$ is positive. Hence the short-duration regime will be followed by a larger price change compared with the long-duration regime. Nonetheless, from the last column of Table (ref), we note that the effect for the transition probability from the long-duration regime to the short-duration regime is even stronger, as the coefficient of $\hat{P}_{21,i+1}$ is significantly greater than the coefficient of $\mathbf{1}_{s_i=1}$. This means that a large price change follows the short-duration regime, while a stronger price movement will occur subsequently if the market is switching from the long-duration regime to the short-duration regime. We think this switching could be predicted by the HFTs, so they would take this opportunity to trade on the signal and to earn a profit as price speculators. However, the effect for $\hat{P}_{12,i+1}$ is not significant, so the price change is relatively small if the HFTs turn from the aggressive trading strategy to the passive one.
At last, we have studied the informativeness of LOB factors to the trading activities by applying the MF-RSD model to the 25 NASDAQ stocks in our sample for 21 trading days (January 2013). For each stock and each factor, we count the number of days in which the factor coefficient in the regression (ref) is significant. The results are presented in Table (ref) and the main findings are as follows.
Some of our findings are in harmony with those of the existing research on market microstructure. First, many papers LI2005533,goldstein2017high,van2019high suggest that order imbalance is a good predictor for price movement, and algorithm traders would use it as a trading indicator. A larger imbalance is a signal for future price movement and hence encourages traders (in particular HFTs) to actively trade to gain profit (or prevent loss). This is why, in some cases, the effect of depth imbalance is significantly negative for inter-trade durations switching from the short-duration regime to the long-duration regime and significantly positive for the switching from the long-duration regime to the short-duration regime. Second, according to ranaldo2004order,carrion2013very,hendershott2013algorithmic, price spread is an indicator of liquidity and measures the profit (cost) of providing (taking) liquidity. A large price spread indicates illiquidity of the asset, and the larger the price spread is, the more transaction cost liquidity takers face and meanwhile the more profit liquidity providers have. Thus, when the price spread becomes larger, people are more reluctant to initialize market orders and the inter-trade durations are more likely to be long. Third, trade volume is indeed a momentum variable that reflects trading willingness o1995market,hasbrouck1991measuring. A large trade volume would not only keep the market in an active trading period (the short-duration regime) but also lead the market to switch from a passive trading period to an active one. Finally, the effect of price movement seems to violate the negative correlation between price movement and duration length manganelli2005duration,furfine2007inter. However, in our high-frequency data, perhaps the change in mid-price in the last trade is not a momentum variable for future price movement. Moreover, we find that the change in mid-price just occurs because the existing limit orders at best bid or best ask are eliminated by the incoming trade, and consequently the price spread between the best ask and the best bid increases. Thus, we suspect that its effect on the regime switchings of inter-trade durations is probably resulted from another channel, i.e., an increase in the price spread.
In addition, we obtain some new findings. First, the depth imbalance plays an important role mostly when the price spread is small. A possible explanation is that when the price spread is large, the depth imbalance is less informative as it is relatively costless for traders to submit orders at the best ask/bid. Moreover, within the revealed price spread, perhaps there are some hidden limit orders that make the true depth imbalance between the best ask and the best bid unknowable. Only when the price spread is tight, the depth imbalance reveals more information and becomes a valid indicator for the subsequent price movement. Next, a large price spread is more likely to keep inter-trade durations in the long-duration regime rather than to reverse inter-trade durations from the short-duration regime to the long-duration regime, as we have observed that the effect of price spread is commonly negative for $P_{21}$ but is not obvious for $P_{12}$. This finding suggests that traders are more sensitive to the size of $PS$ when the market is in the long-duration regime, in which HFTs mainly provide liquidity and slow traders consume liquidity. The slow traders seem to be more concerned with a high transaction cost induced by a large $PS$ when the market is relatively stable. However, when the market enters the short-duration regime in which HFTs aggressively take liquidity and earn a profit because of the price movement, a large price spread would not impede the HFTs' trading aggressiveness and play a significant role in the arrival time of trades. Finally, as the effect of $PM$ is commonly positive for $P_{12}$ and $PM=1$ is accompanied by an increase of price spread, whether the price spread has just increased could be a driving force that leads inter-trade durations to switch to the long-duration regime. This finding may affirm the viewpoint in obizhaeva2013optimal that the optimal strategy for algorithm traders depends more on the resilience properties of supply/demand such as the change in bid-ask spread, rather than its static property.
To further verify the mechanism of LOB factors on the dynamics of inter-trade durations and our explanation, we implement the below `naive' panel data regression to see the relationship between the LOB factors and the price change in the next 10 seconds. The regression model is like following:
where $i$ and $t$ indicate the $i-$th stock and the $t-$th trade respectively, and $\Delta P$ is the short-term price change, the same as in the equation (ref). Here we just use the data of 25 stocks on the same day, i.e., January 2, 2013. The panel data is unbalanced because different stocks have different numbers of trades.
The regression results are presented in Table (ref). As we can see, for the full sample, the LOB factor $DI$ and $PM$ don't show a significant effect on the price change. This finding advocates that the depth imbalance is overall not informative and the price movement of last trade is not a momentum variable for the future price movement. Nevertheless, a large $TV$ predicts a large price change because the $TV$ is significantly positive for the $\Delta P$, which is in line with our finding that a large trade volume will induce short inter-trade durations as HTFs treat it as a signal for future price movement. The coefficient of $PS$ is significant and positive for $\Delta P$, however, we think the $PS$ could be endogenously correlated with the price change as the effective price spread ($PS$) consists of the permanent price change ($\Delta P$) and the realized price spread, which compensate liquidity providers costs on two aspects, i.e., the adverse select cost and the `real friction' carrion2013very. Thus, we exclude $PS$ in the second regression and the result shows that the other three LOB factors have similar effects. We further divided the 25 stocks into two subgroups to verify the effect of $DI$. One subgroup has stocks with relatively tight price spreads,\footnote{There are 12 stocks in total. 75% percentile of the $PS$ for each stock is less than or equal to 0.2.} the other subgroup has the remaining stocks, whose price spreads are relatively slack. From the last two columns in Table (ref), we can observe that the $DI$ in the first subgroup is significantly positive, while it is insignificant in the second subgroup. The finding strongly supports the conclusion that the depth imbalance is informative and a valid indicator for HFTs only when the price spread is small, and a large depth imbalance in such case is very likely to indicate the following price movement.
Moreover, we have provided robustness tests for the findings on the effects of the LOB factors. The tests include the analysis of NASDAQ stocks in a different month, the analysis of directional trades, the analysis of $DI$ conditional on the tightness of $PS$, the analysis with a substitution of $DI$ by the aggregated depth imbalance, and the analysis with a substitution of $PM$ by a dummy variable for the increase in price spread. These robustness test are presented in Section (ref) the Appendices, and the results are very consistent with our findings.
In this paper, we have proposed a multifactor regime-switching duration (MF-RSD) model for the inter-trade durations in the high-frequency limit order market. The motivation is that we have found a common bimodal distribution of inter-trade durations for the stocks listed in the NASDAQ market. The simulation study of MF-RSD model not only validates our estimation method but also shows that the model can replicate the most empirical facts of inter-trade durations, including the newly found bimodal distribution. Furthermore, based on the empirical analysis, we find that the underlying regimes are related to the endogenous liquidity provision and consumption by the HFTs in the market. Using the factor analysis, the MF-RSD model is also capable to identify the types of market information that algorithm traders learn from and react to in high frequency, which causes the inter-trade durations switching between the short-duration and the long-duration regime.
The contribution of the MF-RSD model in analyzing the high-frequency inter-trade duration data in the LOB can be summarized in the following three aspects. First, the out-of-sample test shows that the performance of the MF-RSD model in predicting inter-trade durations is significantly improved compared with the two benchmark duration models. This is because, through the identified impacts of the LOB factors and the parsimonious structure of the two regimes, our model is able to predict whether the arrival time of the next trade is relatively long or short. Second, given the estimated regimes and the estimated regime-switching probabilities by the MF-RSD model, we find empirical evidence that the switchings between the short-duration regime and the long-duration regime result from the HFTs' behavior of changing strategies between providing liquidity and taking liquidity, driven by the different channels of profitability. Specifically, the short-duration regime is followed by a relatively large price movement, which corresponds to a high profit of taking liquidity, while the long-duration regime is positively correlated with the realized price spread, which measures the profit of providing liquidity. Third, the MF-RSD model is also useful in testing economic hypotheses in the market microstructure theory, and our results are consistent with some findings in the prevalent literature. They are as follows: 1) order book imbalance is applicable to predict the price movement in the short term, and in some cases, a larger depth imbalance would lead to a shorter inter-trade duration; 2) a larger price spread tends to maintain inter-trade duration in the long-duration regime because it reflects higher transaction costs for liquidity demanders; and 3) a larger trade volume is a good momentum variable that reveals a strong trading willingness, which normally induces a shorter duration for the next trade. Moreover, we have some new findings added to the literature. They are: 1) the impact of depth imbalance on the regime-switching of inter-trade duration is spread-dependent and is significant when the stock's price spread is tight; 2) a large price spread is not a reverse signal for inter-trade durations switching from the short-duration regime to the long-duration regime if they were staying in the short-duration regime; and 3) whether the price spread has just increased in the last trade is nonetheless a reverse signal for the regime switching of inter-trade durations, through our analysis of the impact of the mid-price movement.
Our work has made progress in analyzing the high-frequency duration data. Nevertheless, it leaves a great deal to future research. For example, our LOB data is rich in terms of having records of order submissions and cancellations, we can further investigate the impact of those activities on the arrival time of trades. To achieve that, a new econometric model is probably needed. Moreover, some event studies could be implemented to directly identify the HFTs' behaviors of switching trading strategies if we can obtain the data that contain exogenous shocks to the market, which helps to verify our findings on the regime-switching of inter-trade durations. We can also test whether inter-trade durations in other high-frequency financial markets have similar properties, in particular, whether the bimodal distribution is a general phenomenon, and a theoretical market microstructure model is certainly needed if so.