Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
120,389 characters · 10 sections · 77 citation commands
Permutation-based tests for discontinuities in event studies
\thispagestyle{empty}
KEYWORDS: event study, infill asymptotics, jump, permutation tests, randomization tests, semimartingale.
JEL classification codes: C12, C14, C22, C32.
\setcounter{page}{1}
Many econometric problems can be expressed in terms of the continuity or the discontinuity of certain component in the underlying economic model. In an influential paper, chow1960 tested the temporal stability in the demand for automobiles, and subsequently stimulated a large literature on structural breaks in time series analysis; see, for example, andrews1993, stock1994, baiperron1998, and many references therein. In microeconometrics, the regression discontinuity design (RDD) has been extensively used for causal inference. This literature identifies and estimates an average treatment effect by evaluating discontinuities of conditional expectation functions of outcome and treatment variables at a cutoff point of the running variable; see imbenslemieux2008 and leelemieux2010 for comprehensive reviews. \footnote{ Coincidentally, the RDD was first proposed by TC1960 around the same time as the Chow test.} Meanwhile, a more recent high-frequency financial econometrics literature has been devoted to studying discontinuities, or jumps, in various financial time series (e.g., price, volatility, trading activity, etc.). The high-frequency jump literature is pioneered by BNS06, who propose the first nonparametric test for asset price jumps using high-frequency data in an infill asymptotic setting. More recently, fomc study the jumps of volatility and trading intensity in high-frequency jump regressions (jur) that closely resemble the classical RDD.
Although these strands of literature involve apparently different terminology and technical tools, they share a common theme: The econometric goal is to learn about differences in the data generating processes between two subsamples separated by the cutoff. imbens2011 emphasize that these subsamples should be “local” to the cutoff point, which is quite natural given the nonparametric nature of discontinuity inference ( hahn2001). The issue under study is thus a local version of the classical two-sample problem. Correspondingly, the related inference is often carried out using nonparametric two-sample t-tests, which are based on kernel regressions in the RDD (hahn2001, imbens2011, CCT2014 ) or, in the same spirit, spot high-frequency estimators (FN1996, comterenault1998, jacodprotter2012, jur, fomc) in the infill time-series setting.
In an ideal scenario in which the subsamples separated by the cutoff are i.i.d.\ and independent of each other, the permutation test is an excellent tool to detect differences in their distributions. In particular, standard results for randomization inference (lehmannromano2005) indicate that a permutation test implemented with any arbitrary test statistic is finite-sample valid under these conditions. The recent literature has investigated the properties of permutation tests {to detect differences between two samples} under less ideal conditions. One example is canaykamat2017, who consider an RDD and show that permutation-based inference is asymptotically valid to detect discontinuities in the distribution of the baseline covariates at the cutoff. These authors implement their test with a finite number of observations that are located closest to the cutoff, effectively forcing them to concentrate on a small neighborhood of the cutoff as the sample size grows. In the same spirit, cattaneo/Tit/VB:17 propose using permutation-based inference to detect discontinuities at the cutoff under the “local randomization framework” introduced in cattaneo/etal:15. Outside of the RDD literature, chungromano2013 and diciccio/romano:2017 investigate the asymptotic properties of permutation-based inference to test for differences in specific distributional features of two samples, such as the mean or the correlation coefficient. It is important to note that all of the references mentioned in this paragraph presume cross-sectional data.
{In the context of time-series applications, there is an active literature on change-point tests implemented via permutations. This approach was first suggested by antoch/huskova:2001 and later pursued by other authors. See huskova:2004 or horvath/rice:2014 for surveys of this literature. While most of this literature imposes independent errors, some allow for limited forms of weak dependence ({kirch/steinebach:2006, kirch:2007, and jentsch/pauly:2015}). In contrast, our econometric setting accommodates essentially unrestricted persistence and nonstationarity in the underlying state processes (e.g., volatility), which better suits our interest on their dynamics over short time windows around economic news events. In the context of machine-learning methods, chernozhukov/wuthrich/zhu:2018 propose using permutations to implement conformal inference that allows for time series data.}
Set against this background, our main goal in this paper is to establish a general theory for permutation-based discontinuity tests, with a special emphasis on event studies based on time-series data. To capture the “local” nature of this problem, we adopt an infill asymptotic framework, under which the inference concentrates on observations “close” to the event time. Specifically, we consider the Cram\'{e}r-von Mises test statistic formed as the squared $L_{2}$ distance between the empirical cumulative distribution functions for the two local subsamples near the cutoff, and compute the critical value via a standard permutation algorithm. As explained earlier, if the data were i.i.d., the behavior of this permutation test would follow directly from standard results for randomization inference. This “off-the-shelf” theory, however, is not applicable here because time-series data observed in a short event window can be serially highly dependent.
The main theoretical contribution of the present paper is to establish the asymptotic validity of the permutation test in this non-standard setting. The theory has two components. The first is a new general result for permutation test. Specifically, we link the (feasible) permutation test formed using the original data with an infeasible test constructed in a “coupling” problem that involves conditionally i.i.d.\ coupling variables. Since the latter resembles the classical two-sample problem, the infeasible test controls size exactly under the coupling null hypothesis (i.e., coupling variables in the two subsamples are homogeneous), and is consistent under the complementary alternative hypothesis. Under a proper notion of coupling, which is customized for the permutation test, we show that the feasible test inherits the same asymptotic rejection properties from the infeasible one. Since this result is of independent theoretical interest that is well beyond our subsequent analysis in the infill time-series setting, we frame the theory under general high-level conditions so as to facilitate other types of applications.
The second component of our analysis pertains to specializing the general result to the infill time-series setting designed for event-study applications. The event-study framework is particularly relevant for studying macroeconomic and financial shocks, including monetary shocks triggered by FOMC announcements (cochranepiazzesi2002, nakamura2018QJE), or \textquotedblleft natural disasters\textquotedblright\ such as the ongoing COVID-19 pandemic. Following lixiu2016 and fomc, we model observed data using a general state-space framework, in which the observations are discretely sampled from a latent state process \textquotedblleft contaminated\textquotedblright\ by random disturbances. This model has been used to model variables such as asset returns, trading volume, duration, and bid-ask spread, and readily accommodates both continuously and discretely valued variables. Under this state-space model, the temporal discontinuity in the data's distribution is mainly driven by the jump of the latent state process (e.g., asset volatility, trading intensity, and propensity of informed trading), which can be detected by the permutation test. Under easy-to-verify primitive conditions, we construct coupling variables and apply the aforementioned general theory to establish the permutation test's asymptotic validity.
We recognize two advantages of the proposed permutation test in comparison with the standard approach based on the nonparametric “spot” estimation of the underlying state process. Firstly, the permutation test attains asymptotic size control even if the number of observations in each subsample is fixed.\footnote{For similar type of results in the context of RDD; see cattaneo/etal:15, cattaneo/Tit/VB:17, canaykamat2017, and bugni/canay:2021.} This remarkable property is reminiscent of the finite-sample exactness of the permutation test in the classical two-sample problem for i.i.d.\ data. In contrast, the nonparametric estimation approach works in a fundamentally different way, as it relies on the asymptotic (mixed) normality of the estimator, which in turn requires the sizes of the local subsamples to grow to infinity. In empirical applications, however, it is often desirable to use a short time window, either to reduce the effect of confounding factors in the background, or simply because of the lack of observations soon after the occurrence of the economic event (say, in a real-time research situation). Not surprisingly, the conventional inference based on asymptotic Gaussianity often results in large size distortions in this \textquotedblleft small-sample\textquotedblright\ scenario, as we demonstrate concretely in a realistically calibrated Monte Carlo experiment (see Section (ref)). Meanwhile, the permutation test exhibits much more robust size control in finite samples.
The second advantage of the permutation test is its versatility: The same test can be applied in many different empirical contexts without any modification. On the other hand, the nonparametric estimation approach often relies on specific features of the problem, and needs to be designed on a case-by-case basis. Therefore, the proposed permutation test may be particularly attractive in new empirical environments for which tests based on the conventional approach are not yet developed or not yet well-understood. In Section (ref), we illustrate this point more concretely in the context of testing for volatility jumps. In that case, the standard approach relies crucially on the assumption that the price shocks are Brownian in its design of the spot volatility estimator and the associated t-statistic, and it cannot be adapted easily to accommodate a more general setting with L\'{e}vy-driven shocks.\footnote{As explained by bns2001, these more general processes offer the possibility of capturing important deviations from Brownian shocks and for flexible modelling of dependence structures. However, to the best of our knowledge, the estimation and inference of the spot volatility (i.e., the scaling process) in the non-Brownian case remains to be an open question in the literature. There is some limited work on the inference of integrated volatility functionals for the non-Brownian case (see todorov2012) which demonstrates various distinct complications in the non-Brownian setting.} The permutation test, on the other hand, is valid even in the latter, more general, setting.
That being said, we stress that the proposed permutation test is a complement, rather than substitute, for the conventional nonparametric estimation method, because it has two limitations. One is that the permutation test focuses exclusively on hypothesis testing, without producing a point estimate for the jump of the state process (e.g., volatility) of interest, whereas the estimate is a by-product of the conventional approach. In addition, the proposed permutation test is purely nonparametric and it does not exploit any parametric structure that one may be willing to impose. It is therefore conceivable that in certain semiparametric settings, more efficient tests may be designed to exploit a priori model restrictions. Put differently, the aforementioned versatility of the permutation test may come with an efficiency cost. A better understanding about the robustness-efficiency tradeoff might be an interesting topic for future research.
{In an empirical illustration, we apply the permutation test to a recent sample of high-frequency intraday returns of the SPY ETF for the S&P 500 index. Specifically, we focus on four important FOMC announcements during the ongoing COVID-19 pandemic, and test whether each announcement induces discontinuities in volatility, trading activity, and two measures of market illiquidity. We document robust empirical evidence for discontinuities in volatility and trading activity. We also find evidence for announcement-induced discontinuity in transaction cost (measured by bid-ask spread), but not in market impact (gauged by Amihud's measure).} This application highlights one of the main advantages of the proposed test, namely, it is applicable for a broad variety of high-frequency observations modeled in distinct ways, which is unlike, for example, the conventional t-test designed specifically for testing volatility jumps in the Brownian setting.
The rest of the paper is organized as follows. We present the asymptotic theory for the permutation test in Section (ref). Section (ref) reports the test's finite-sample performance in Monte Carlo experiments, and Section (ref) presents the empirical illustration. Section (ref) concludes. The appendix contains all proofs.
Notation. We use $\left\Vert x\right\Vert $ to denote the Euclidean norm of a vector $x$. For any real number $a$, we use $\lceil a \rceil$ to denote the smallest integer that is larger than $a$. For any constant $p\geq 1$, $\left\Vert \,\cdot \,\right\Vert _{p}$ denotes the $L_{p}$ norm for random variables. For two real sequences $a_{n}$ and $b_{n}$, we write $a_{n}\asymp b_{n}$ if $ a_{n}/C\leq b_{n}\leq Ca_{n}$ for some finite constant $C\geq 1$.
We first prove a new result that is broadly useful for establishing the asymptotic validity of permutation tests. Because of its independent theoretical interest, we develop the theory under high-level conditions. In Section (ref), below, we shall specialize this general result in event-study applications under a more specific infill time-series setting, for which the existing theory on permutation tests is not applicable.
Consider an array $(Y_{n,i})_{i\in \mathcal{I}_{n}}$ of $\mathbb{R}$-valued observed variables defined on a probability space $\left( \Omega ,\mathcal{F} ,\mathbb{P}\right) $, which may be either “raw” data or preliminary estimators. Our econometric goal is to decide whether two subsamples $ (Y_{n,i})_{i\in \mathcal{I}_{1,n}}$ and $\left( Y_{n,i}\right) _{i\in \mathcal{I}_{2,n}}$ have \textquotedblleft significantly\textquotedblright\ different distributions, where $(\mathcal{I}_{1,n},\mathcal{I}_{2,n})$ is a partition of $\mathcal{I}_{n}\subseteq \mathbb{Z}$. For ease of exposition, we assume that $ \mathcal{I}_{1,n}$ and $\mathcal{I}_{2,n}$ contain the same number of observations, denoted by $k_{n}$.\footnote{ All of our results can be easily extended to the case when $\mathcal{I}_{1,n} $ and $ \mathcal{I}_{2,n}$ have different sizes, but with the same order of magnitude.} We stress from the outset that $k_{n}$ may either be fixed or grow to infinity in the subsequent analysis. As such, our analysis speaks to not only the classical finite-sample analysis of permutation tests, but also the large-sample analysis routinely used in econometrics.
To implement the test, we first estimate the empirical cumulative distribution functions (CDF) for the two subsamples using
We then measure their difference via the Cram\'{e}r--von Mises statistic, given by
For a significance level $\alpha \in \left( 0,1\right) $, we compute the critical value via a standard permutation algorithm as in lehmannromano2005, which we specify in Algorithm 1 below. We use $\pi $ to denote a permutation of the elements of $\mathcal{I}_{n}$, that is, a bijective mapping from $\mathcal{I}_{n}$ to itself. Let $G_{n}$ denote the collection of all possible permutations of $\mathcal{I}_{n}$, with $M_{n}$ being its cardinality.
Algorithm 1. Step 1. For each permutation $\pi \in G_n $, compute the permuted test statistic $\widehat{T}_{n}(\pi )$ as $\widehat{T} _{n}$, but with $(Y_{n,i})_{i\in \mathcal{I}_{n}}$ replaced by $(Y_{n,\pi \left( i\right) })_{i\in \mathcal{I}_{n}}$.
Step 2. Order $\{\widehat{T}_{n}(\pi ):\pi \in G_{n}\}$ as $ \widehat{T}_{n}^{(1)}\leq \cdots \leq \widehat{T}_{n}^{(M_{n})}$. Set $ \widehat{T}_{n}^{\ast }=\widehat{T}_{n}^{(k)}$ for $k=\left\lceil M_{n}(1-\alpha )\right\rceil $.
Step 3. If $\widehat{T}_{n}>\widehat{T}_{n}^{\ast }$, reject the null hypothesis. If $\widehat{T}_{n}<\widehat{T}_{n}^{\ast }$, do not reject the null hypothesis. If $\widehat{T}_{n}=\widehat{T}_{n}^{\ast }$, reject the null hypothesis with probability $\hat{p}_{n}\equiv (M_{n}\alpha - \widehat{M}_{n}^{+})/\widehat{M}_{n}^{0}$, where $\widehat{M}_{n}^{+}$ and $ \widehat{M}_{n}^{0}$ are the cardinalities of $\{j:\widehat{T}_{n}^{(j)}> \widehat{T}_{n}^{\ast }\}$ and $\{j:\widehat{T}_{n}^{(j)}=\widehat{T} _{n}^{\ast }\}$, respectively. The resulting test then rejects according to the critical function $ \hat{\phi}_{n}\equiv 1\{\widehat{T}_{n}>\widehat{T}_{n}^{\ast }\}+\hat{p} _{n}1\{\widehat{T}_{n}=\widehat{T}_{n}^{\ast }\}$. $\Box $
If the data $(Y_{n,i})_{i\in \mathcal{I}_{n}}$ are i.i.d., then the null hypothesis of the classical two-sample problem holds, and lehmannromano2005 implies that the aforementioned permutation test has exact size control in finite samples. This is a remarkable property of the permutation test, as it holds without requiring any specific distributional assumptions on the data. In contrast to the classical two-sample problem, however, we shall not assume that the data are independent, or even \textquotedblleft weakly\textquotedblright\ dependent (e.g., mixing). As mentioned in the Introduction, the main goal of this paper is to study the permutation test for time-series data observed within a short event window (say, a few days or hours), which can be serially highly dependent in practice. Our key theoretical insight is that the permutation test is still asymptotically valid if the data $(Y_{n,i})_{i\in \mathcal{I}_{n}}$ can be approximated, or \textquotedblleft coupled,\textquotedblright\ by another collection of variables that are conditionally independent, as formalized by the following assumption.
Assumption (ref) lays out the high-level structure for bridging our analysis with the classical theory on permutation tests, which we carry out in Theorem (ref) below. Condition (i) sets up the \textquotedblleft coupling\textquotedblright\ problem, which corresponds to a conditional version of the classical two-sample problem, treating the $(U_{n,i})_{i\in \mathcal{I}_{1,n}}$ and $(U_{n,i})_{i\in \mathcal{I}_{2,n}}$ variables as \textquotedblleft data\textquotedblright. In part (a) of Theorem (ref) , we consider the situation in which both subsamples have the same conditional distribution. In this case, our coupling variables $ (U_{n,i})_{i\in \mathcal{I}_{n}}$ give rise to an infeasible permutation test that can be analyzed as a classical two-sample problem. In particular, this infeasible permutation test attains the exact finite-sample size under our conditions.
This infeasible test, however, only plays an auxiliary role in our analysis, because our interest is on the feasible test $\hat{\phi}_{n}$ formed using the original $(Y_{n,i})_{i\in \mathcal{I}_{n}}$ data. Therefore, a key component of our theoretical argument in Theorem (ref) is to show that the feasible test for the original data inherits asymptotically the same rejection properties from the infeasible test. Conditions (ii) and (iii) in Assumption (ref) are introduced for this purpose. Specifically, condition (ii) requires the variable $U_{n,i}$ to be non-degenerate, in the sense that its conditional probability mass within any small $\left[ x-\eta ,x+\eta \right] $ interval is of order $O\left( \eta \right) $ in probability.\footnote{ {Condition (ii) is satisfied if $ U_{n,i}$ has a conditional probability density that is uniformly bounded in probability.}} Condition (iii) specifies the requisite approximation accuracy of the coupling variables. We note that this condition is easier to hold when $k_n$ is smaller, because the joint coupling requirement would involve a smaller number of variables and the $o_p(k_n^{-2})$ error bound is easier to attain. This condition can be verified under more primitive conditions pertaining to the smoothness of underlying processes and {an upper bound on the growth rate of $k_{n}$}, as detailed in Section (ref).\footnote{ In our applications, we can often verify condition (iii) with $\widetilde{Y} _{n,i}=Y_{n,i}$. Nonetheless, allowing $\widetilde{Y}_{n,i}\neq Y_{n,i}$ is useful when $Y_{n,i}$ is itself an estimator. For example, if $ (Y_{n,i})_{i\in \mathcal{I}_{n}}$ is a finite collection of estimators that converge jointly in distribution, then the coupling required in Assumption (ref)(iii) can be obtained via Skorokhod representation. As such, the theory developed in canaykamat2017 can be cast in our framework with $\mathcal{G}_n$ being the trivial information set and $k_n$ being a fixed constant, although our anti-concentration condition in Assumption (ref)(ii) is slightly stronger than the continuity condition in canaykamat2017.}
Theorem (ref) characterizes the asymptotic rejection probabilities of the feasible test $\hat{\phi}_{n}$ under the null and alternative hypotheses of the two-sample problem for the coupling variables. Part (a) pertains to the situation in which the two subsamples of coupling variables, $ (U_{n,i})_{i\in \mathcal{I}_{1,n}}$ and $(U_{n,i})_{i\in \mathcal{I}_{2n}}$, have the same conditional distribution, which corresponds to the null hypothesis. In this case, the theorem shows that the asymptotic rejection probability of the feasible test is equal to the nominal level $\alpha $. It is relevant to note that this result holds whether $k_n$ is fixed or divergent. This property is clearly reminiscent of the permutation test's finite-sample exactness in the classical setting.
Part (b) of Theorem (ref) concerns the power of the feasible test $\hat{ \phi}_{n}$. It shows that the feasible test rejects with probability approaching one when the conditional distributions of the two coupling subsamples, $Q_{1,n}$ and $Q_{2,n}$, are different, in the sense that their \textquotedblleft distance\textquotedblright\ measured by $\int \left( Q_{1,n}(x)-Q_{2,n}(x)\right) ^{2}d\overline{Q}_{n}\left( x\right) $ is asymptotically non-degenerate, where the mixture distribution $\overline{Q} _{n}$ captures approximately the distribution of the permuted data.\footnote{When the two subsamples have different sizes, the same result goes through with $\overline{Q}_n$ defined as the sample-size weighted average between $Q_{1,n}$ and $Q_{2,n}$, {given by $\overline{Q}_n=(Q_{1,n}|\mathcal{I}_{1,n}| + Q_{2,n}|\mathcal{I}_{2,n}|)/(|\mathcal{I}_{1,n}|+|\mathcal{I}_{2,n}|)$.}} This consistency-type result requires that the information available from each subsample grows with the sample size, i.e., $k_{n}\rightarrow \infty $. This result appears to be new in the context of permutation-based tests under a fixed alternative for the coupling variables. In particular, we note that an analogous result is unavailable in canaykamat2017, as they restrict attention to an asymptotic framework with a fixed $k_n$, which makes a consistency-type result unavailable. Our proof relies on applying lehmannromano2005 to the infeasible test, for which we use the coupling construction developed by chungromano2013 to show that the so-called hoeffding:1952 condition is satisfied. We note that this argument is used to establish the consistency of the permutation test rather than its asymptotic size property.
Theorem (ref) establishes the relation between the rejection probability of the feasible test $\hat{\phi}_{n}$ and the homogeneity (or the lack of it) across the two coupling subsamples $(U_{n,i})_{i\in \mathcal{ I}_{1,n}}$ and $(U_{n,i})_{i\in \mathcal{I}_{2,n}}$. This result does not speak directly to hypotheses formulated in terms of the original $\left( Y_{n,i}\right) _{i\in \mathcal{I}_{n}}$ observations. Rather, its theoretical significance is to \textquotedblleft absorb\textquotedblright\ all generic technicalities stemming from the (feasible) permutation test, which in turn considerably simplifies our overall analysis. The residual issue for any specific application is to explicitly construct the coupling variables and translate their homogeneity in terms of the primitive structures of the original empirical problem, which can be done using domain-specific techniques. We provide general results along this line in the infill time-series context, as detailed in Section (ref) below.
To help anticipate the general discussion, it is instructive to sketch the scheme in a basic running example. Let $Y_{n,i}=\Delta _{n}^{-1/2}(P_{\left( i+1\right) \Delta _{n}}-P_{i\Delta _{n}})$ be the scaled increment of the asset price process $P_{t}$ over the $i$th sampling interval. Let $\tau $ be a \textquotedblleft cutoff\textquotedblright\ time point of interest that is known to the researcher (e.g., the announcement time of a news release) and $i^{\ast }$ be the unique integer such that $\tau \in \lbrack i^{\ast }\Delta _{n},(i^{\ast }+1)\Delta _{n})$.\footnote{ The integer $i^{\ast }$ depends on $n$. We suppress this dependence in our notation for simplicity and to avoid having nested subscripts.} We consider two index sets $\mathcal{I}_{1,n}=\{i^{\ast }-k_{n},\ldots ,i^{\ast }-1\}$ and $\mathcal{I}_{2,n}=\{i^{\ast }+1,\ldots ,i^{\ast }+k_{n}\}$, which collect observations before and after the cutoff, respectively. We consider an asymptotic setting in which these subsamples are \textquotedblleft local\textquotedblright\ in calendar time, that is, $ k_{n}\Delta _{n}\rightarrow 0$. Note that this implies that $\Delta _{n}\rightarrow 0$, which means that we are considering an infill asymptotic setting. If $P_{t}$ is an It\^{o} process with respect to an information filtration $(\mathcal{F}_{t})_{t\geq 0}$, we may represent $Y_{n,i}$ as
where $b_{t}$ is the drift process, $\sigma _{t}$ is the stochastic volatility process, and $W_{t}$ is a standard Brownian motion.\footnote{ Note that $\mathcal{I}_{n}$ does not include the $i^{\ast }$th return observation. Therefore, although the returns in ((ref)) do not contain price jumps, an event-induced price jump is allowed to occur at time $\tau $.} If the $\sigma _{t}$ process is smooth (e.g., H\"{o}lder continuous) in a local neighborhood before $\tau $, then the volatility throughout the pre-event subsample $\mathcal{I}_{1,n}$ is approximately $ \sigma _{(i^{\ast }-k_{n})\Delta _{n}}$. Further recognizing that the drift term is negligible relative to the Brownian component, we can approximate $ Y_{n,i}$ for each $i\in \mathcal{I}_{1,n}$ using the coupling variables
where $\mathcal{MN}$ denotes the mixed normal distribution. Since the Brownian motion has independent and stationary increments, it is easy to see that the coupling variables $(U_{n,i})_{i\in \mathcal{I}_{1,n}}$ are $ \mathcal{F}_{(i^{\ast }-k_{n})\Delta _{n}}$-conditionally i.i.d. Moreover, if the volatility process $\sigma _{t}$ does not jump at the cutoff time $ \tau $, we may follow the same logic to extend the approximation in ((ref)) further to $i\in \mathcal{I}_{2,n}$. In other words, if the volatility process process does not jump then the coupling variables $ (U_{n,i})_{i\in \mathcal{I}_{n}}$ are conditionally i.i.d., which corresponds to the situation in part (a) of Theorem (ref). On the other hand, if the volatility process jumps at time $\tau $, say by a constant $ c\neq 0$, then the coupling variables for the $\mathcal{I}_{2,n}$ subsample will instead take the form $U_{n,i}=\left( \sigma _{(i^{\ast }-k_{n})\Delta _{n}}+c\right) (W_{\left( i+1\right) \Delta _{n}}-W_{i\Delta _{n}})$. In this case, the two subsamples of $U_{n,i}$'s have distinct conditional distributions (i.e., mixed normal with different conditional variances), corresponding to the scenario in part (b) of Theorem (ref).
Within the context of this illustrative example, we can further clarify a key feature of the proposed test that holds more generally. It is not aimed at detecting \textquotedblleft small\textquotedblright\ time-variations in the distribution of the observed data. In fact, by allowing the drift $b_{t}$ and the volatility $\sigma _{t}$ to be time-varying, a smooth form of heterogeneity is always built in. The test instead detects abrupt changes, or discontinuities, in the evolution of the distribution, which can be more plausibly associated with the \textquotedblleft lumpy\textquotedblright\ information carried by the underlying economic announcement, as emphasized by nakamura2018JEP. Specifically in this example, the asset returns are locally centered Gaussian (due to the assumption that the price is an It\^{o} process), and hence, the temporal discontinuity in the return distribution manifests itself as a volatility jump. The empirical scope of our permutation test, however, is far beyond volatility-jump testing depicted in this illustration, as we shall demonstrate in the remainder of the paper.
We now specialize the general Theorem (ref) into an infill asymptotic time-series setting that is particularly suitable for event studies. By introducing a mild additional econometric structure, we shall establish the asymptotic validity of the permutation test under more primitive conditions that are easy to verify in a variety of concrete empirical settings. As in the running example above, we consider an event occurring at time $\tau \in [i^{\ast }\Delta _{n},(i^{\ast }+1)\Delta _{n})$, which separates two subsamples indexed by $\mathcal{I }_{1,n}=\{i^{\ast }-k_{n},\ldots ,i^{\ast }-1\}$ and $\mathcal{I} _{2,n}=\{i^{\ast }+1,\ldots ,i^{\ast }+k_{n}\}$, respectively. All limits in the sequel are obtained under the infill asymptotic setting with $\Delta _{n}\rightarrow 0$.
We suppose that the data are generated from an approximate state-space model of the form
where the state process $\zeta _{t}$ is c\`{a}dl\`{a}g, adapted to a filtration $\mathcal{F}_{t}$, and takes values in an open set $\mathcal{Z} \subseteq \mathbb{R}^{\dim (\zeta )}$; $(\epsilon _{n,i})_{i\in \mathcal{I} _{n}}$ are i.i.d.\ random disturbances taking values in some (possibly abstract) space $\mathcal{E}$; $g\left( \cdot ,\cdot \right) $ is a \textquotedblleft smooth\textquotedblright\ transform; and $R_{n,i}$ is a residual term that is negligible relative to the leading term $g\left( \zeta _{i\Delta _{n}},\epsilon _{n,i}\right) $ in a proper sense detailed below. A simpler version of this state-space model without the $R_{n,i}$ residual term has been used by lixiu2016 and fomc, among others, for modeling market variables such as trading volume and bid-ask spread. By introducing the $R_{n,i}$ {residual} term, we can use a unified framework to accommodate a broader class of models, which in particular include increments of an It\^{o} semimartingale. We now revisit the model in ((ref)) as the first illustration.
Example 1 (Brownian Asset Returns). We represent the It \^{o}-process model ((ref)) for asset returns in the form of ((ref)) by setting $\zeta _{t}=\sigma _{t}$, $\epsilon _{n,i}=\Delta _{n}^{-1/2}(W_{\left( i+1\right) \Delta _{n}}-W_{i\Delta _{n}})$, and $ g\left( z,\epsilon \right) =z\epsilon $. The resulting residual term has the form
Under mild and fairly standard regularity conditions, it is easy to show that $\max_{i \in \mathcal{I}_n}|R_{n,i}|$ is $o_p(1)$. On the other hand, the leading term $g\left( \zeta _{i\Delta _{n}},\epsilon _{n,i}\right) $ has a non-degenerate centered mixed Gaussian distribution with conditional variance $\sigma _{i\Delta _{n}}^{2}$. $\square $
This running example further illustrates the distinct roles played by $ \zeta_t$, $\epsilon_{n,i}$, and $R_{n,i}$ in our state-space model ((ref) ). The leading term $g\left( \zeta _{i\Delta _{n}},\epsilon _{n,i}\right) $ captures the “main feature” of the observed data; in addition, since the $ \epsilon _{n,i}$ disturbance terms are i.i.d., any \textquotedblleft large\textquotedblright\ change in the empirical distribution across the two subsamples must be attributed to the time-$\tau $ discontinuity in the state process $\zeta _t$. From this description, it follows that the hypothesis test for the continuity of the distribution of the main feature of the observed data can be formulated as
where $\Delta \zeta _{\tau }\equiv \zeta _{\tau }-\zeta _{\tau -}\equiv \zeta _{\tau }-\lim_{s\uparrow \tau }\zeta _{s}$ denotes the jump of the state process at time $\tau $.
With the state-space model ((ref)) in place, we can design more primitive sufficient conditions for establishing the asymptotic validity of the permutation test under the hypotheses in ((ref)). We need some additional notation to describe these conditions. For each fixed $z\in \mathcal{Z}$, let $f_{z}\left( \cdot \right) $ and $F_{z}\left( \cdot \right) $ denote the probability density function (PDF) and the CDF of the random variable $g\left( z,\varepsilon _{n,i}\right) $, respectively. It is also convenient to introduce a \textquotedblleft shifted\textquotedblright\ version of $\zeta _{t}$ defined as $\tilde{\zeta}_{t}\equiv \zeta _{t}-\Delta \zeta _{\tau }1\{t\geq \tau \}$, which has the same increments as $\zeta _{t}$ over time intervals not containing $\tau $.
Assumption (ref) entails regularity conditions pertaining to the random disturbance terms, which are often easy to verify in concrete examples as demonstrated later in this subsection. Assumption (ref) imposes a set of smoothness conditions that permits the approximation of the observed data using properly constructed coupling variables.\footnote{ Note that the assumption is framed in a localized fashion using the stopping times $(T_{m})_{m\geq 1}$, which is a standard technique for weakening the regularity condition in the infill asymptotic setting. See jacodprotter2012 for a comprehensive discussion on the localization technique.} Specifically, condition (i) requires that the random function $ z\mapsto g\left( z,\epsilon _{n,i}\right) $ is Lipschitz in $z$ over compact sets under the $L_{2}$ distance. The $a_{n}$ sequence captures the scale of the Lipschitz coefficient. In many applications, we can verify this condition simply with $a_{n}\equiv 1$, but allowing $a_{n}$ to diverge to infinity is sometimes necessary (see Example 2 below). Condition (ii) states that the $\zeta _{t}$ process is locally compact (up to each stopping time $ T_{m}$) and, upon removing the fixed-time discontinuity at $\tau $, it is $ (1/2)$-H\"{o}lder continuous under the $L_{2}$ norm. This H\"{o} lder-continuity requirement can be easily verified using well-known results provided that the $\tilde{\zeta}$ process is an It\^{o} semimartingle or a long-memory process (see jacodprotter2012 and liliu2020). Condition (iii) imposes the requisite assumptions on the residual terms. In some applications, this condition holds trivially with $ R_{n,i}\equiv 0$, but, more generally, it needs to be verified on a case-by-case basis using (relatively standard) infill asymptotic techniques. Theorem (ref), below, establishes the size and power properties of the permutation test under the hypotheses described in ((ref)).
This theorem is proved by verifying the high-level conditions in Theorem (ref) with properly constructed coupling variables analogous to those in equation ((ref)). The condition $a_{n}k_{n}^{3}\Delta _{n}^{1/2}=o(1) $ mainly requires that the window size $k_{n}$ does not grow too fast, which ensures the closeness between the coupling variables and the original data. In the typical case with $a_{n}=1$, it reduces to $k_{n}=o(\Delta_{n}^{-1/6})$.\footnote{This sufficient condition for the growth rate of $k_n$ is different from the conditions needed for conventional asymptotic-Gaussian-based spot inference, which requires {$k_n \asymp \Delta_n^{-\iota}$ for some $\iota\in(0,1/2)$}. For the permutation test, $k_n$ may be fixed or grow to infinity. However, in the latter case, our condition on the growth rate of $k_n$ is more stringent than what is needed for the conventional spot inference theory.} In general, a larger $k_n$ allows one to utilize more data, but the associated longer event window may also lead to a larger nonparametric bias, and hence, a more severe size distortion. Part (a) shows that the permutation test attains the desired asymptotic level under the null hypothesis in ((ref)). Again, we stress that the test has valid asymptotic size control even in the \textquotedblleft small-sample\textquotedblright\ case with fixed $k_{n}$. As in Theorem (ref), the \textquotedblleft large-sample\textquotedblright\ condition $k_{n}\rightarrow \infty $ is needed for establishing the consistency of the test under the alternative, as shown in part (b).
In the remainder of this subsection, we use a few prototype examples to demonstrate how the proposed test may be used in various empirical settings. In particular, we show how to cast the specific problems into the approximate state-space model ((ref)), and discuss how to verify our sufficient regularity conditions. We start by revisiting the running example.
Example 1 (Brownian Asset Returns, Continued). Recall that $\epsilon _{n,i}\equiv \Delta _{n}^{-1/2}(W_{\left( i+1\right) \Delta _{n}}-W_{i\Delta _{n}})$, $\zeta _{t}=\sigma _{t}$, and $g\left( z,\epsilon \right) =z\epsilon $. In this context, the hypothesis testing problem in ( (ref)) represents a test of the continuity of the volatility process $ \sigma _{t}$ at time $t=\tau$, i.e.,
We suppose that the volatility process $\sigma _{t}$ is non-degenerate by setting its domain to $\mathcal{Z}=\left( 0,\infty \right) $. Since the Brownian motion has independent increments with respect to the underlying filtration, the disturbance term $\epsilon _{n,i}$ satisfies Assumption (ref)(i). In addition, for each point $z\in \mathcal{Z}$, the random variable $f\left( z,\epsilon _{n,i}\right) $ has an $\mathcal{N}\left( 0,z^{2}\right) $ distribution. It is then easy to see that conditions (ii) and (iii) in Assumption (ref) hold for any compact subset $\mathcal{K} \subseteq \mathcal{Z}$ (note that $\mathcal{K}$ is necessarily bounded away from zero). To verify Assumption (ref), first note that $g\left( z,\epsilon _{n,i}\right) -g(z^{\prime },\epsilon _{n,i})=(z-z^{\prime })\epsilon _{n,i}$, and hence, $\left\Vert g\left( z,\epsilon _{n,i}\right) -g(z^{\prime },\epsilon _{n,i})\right\Vert _{2}=|z-z^{\prime }|$. Assumption (ref)(i) thus holds for $a_{n}=1$. It is well known that $\sigma _{t}$ is locally $(1/2)$-H\"{o}lder continuous under the $L_{2}$ norm if it is an It\^{o} semimartingale or a long-memory process; if so, Assumption (ref)(ii) is satisfied if the $\sigma _{t}$ and $\sigma _{t}^{-1}$ processes are both locally bounded. Finally, to verify Assumption (ref)(iii), we assume that the drift process $b_{t}$ is locally bounded. It is then easy to show via routine calculations that $\max_{i\in \mathcal{I}_{n}}|R_{n,i}|=O_{p}(k_{n}^{1/2}\Delta _{n}^{1/2})$. Since the condition $a_{n}k_{n}^{3}\Delta _{n}^{1/2}=o(1)$ in Theorem (ref) implies that $O_{p}(k_{n}^{1/2}\Delta _{n}^{1/2})=o_{p}(k_{n}^{-2})$, we have $\max_{i\in \mathcal{I}_{n}}|R_{n,i}|=o_{p}(k_{n}^{-2})$ as needed in Assumption (ref)(iii). All conditions in Theorem (ref) are now verified, and this shows that the permutation test $\hat{\phi}_{n}$ is asymptotically valid for testing the null hypothesis $\Delta \sigma _{\tau }=0$. $\square $
Example 1 shows that the permutation test $\hat{\phi}_{n}$ is asymptotically valid for testing the presence of a volatility jump. This is a relatively familiar problem in the literature. It is therefore useful to contrast the proposed permutation test with the standard approach, which is based on nonparametric \textquotedblleft spot\textquotedblright\ estimators of the asset price's instantaneous variances before and after the event time given by, respectively,
Assuming $k_{n}\rightarrow \infty $ and $k_{n}^{2}\Delta _{n}\rightarrow 0$, it can be shown that (see jacodprotter2012)
Thus, we can test $H_0: \Delta \sigma _{\tau }=0$ by comparing the t-statistic $k_{n}^{1/2}\left( \hat{\sigma}_{\tau }^{2}-\hat{\sigma}_{\tau -}^{2}\right) /\sqrt{2\hat{\sigma}_{\tau }^{4}+2\hat{\sigma}_{\tau -}^{4}}$ with critical values based on the standard normal distribution.
Two remarks are in order. First, note that the asymptotic size control of the standard approach relies on the asymptotic normal approximation ((ref)), which depends crucially on $k_{n}\rightarrow \infty $ (in addition to having $\Delta_n\to 0$) because the underlying central limit theorem is obtained by aggregating a \textquotedblleft large\textquotedblright\ number of martingale differences. Hence, the t-test may suffer from severe size distortion when $k_{n}$ is relatively small. This issue is empirically relevant because an applied researcher may use a short time window to capture short-lived “impulse-like” dynamics and/or to minimize the impact of other confounding economic factors in the background. Moreover, for \textquotedblleft real-time\textquotedblright\ applications, the researcher may have no choice but to use a small $k_{n}$ simply because of the limited amount of available data soon after the event time $\tau $. In sharp contrast, the permutation test controls asymptotic size even when $ k_{n}$ is fixed. This remarkable property is inherited from the coupling two-sample problem, in which the permutation test controls size exactly regardless of whether $k_{n}$ is fixed or grows to infinity.
The second and perhaps practically more important difference between the two tests is that the permutation test is more versatile. Under the spot-estimation-based approach, both the design of the spot estimators in ( (ref)) and the convergence in ((ref)) depend heavily on the fact that the increments of the Brownian motion are not only i.i.d., but also Gaussian. Gaussianity is obviously essential for the conventional approach because, among other things, it ensures that the instantaneous variance of the normalized returns are well-defined.\footnote{ Recall that many distributions used in continuous-time models do not have finite second moments. For example, within the class of stable distributions, the Gaussian distribution is the only one with a finite second moment. Moreover, Gaussianity also implies that the variance of $ \Delta _{n}^{-1}(W_{i\Delta _{n}}-W_{(i-1)\Delta _{n}})^{2}$ is 2, which explains the \textquotedblleft 2\textquotedblright\ factor in the denominator of the t-statistic.} The permutation test, on the other hand, only exploits the i.i.d.\ property of the Brownian shocks, without relying on {their} Gaussianity. Therefore, the permutation test readily accommodates a more general model for asset returns with L\'{e}vy shocks, as we demonstrate in the following example.
Example 2 (L\'{e}vy-driven Asset Returns). We generalize the model in Example 1 by replacing the Brownian motion $W$ with a L\'{e}vy martingale $L$, so that the asset return has the form
In this case, we define the random disturbance as $\epsilon _{n,i}\equiv \Delta _{n}^{-1/\beta }(L_{\left( i+1\right) \Delta _{n}}-L_{i\Delta _{n}})$ for some constant $\beta \in (1,2]$. The more general normalizing sequence $ \Delta _{n}^{-1/\beta }$ is used to ensure that $\epsilon _{n,i}$ has a non-degenerate distribution. For instance, if $L$ is a stable process, we take $\beta $ to be its jump-activity index, so that $\epsilon _{n,i}$ has a centered stable distribution (recall that the Brownian motion is a stable process with index $\beta =2$). We treat the value of $\beta $ as unknown. Since the permutation test is scale-invariant with respect to the data, we can nonetheless regard the normalized return $Y_{n,i}=\Delta _{n}^{-1/\beta }(P_{\left( i+1\right) \Delta _{n}}-P_{i\Delta _{n}})$ as directly observable (because tests implemented for $P_{\left( i+1\right) \Delta _{n}}-P_{i\Delta _{n}}$ and $Y_{n,i}$ are identical). To apply our theory, we represent $Y_{n,i}$ using the state-space model ((ref)) with $\zeta _{t}=\sigma _{t}$, $g\left( z,\epsilon \right) =z\epsilon $, and the residual term given by
Recognizing that the scaled L\'{e}vy increments $(\epsilon _{n,i})_{i\in \mathcal{I}_{n}}$ are i.i.d., we can verify Assumptions (ref) and (ref) using similar arguments as in Example 1 but with $a_{n}=\Delta _{n}^{1/2-1/\beta }$, which depicts the rate at which $\Vert \epsilon _{n,i}\Vert _{2}$ diverges. In particular, the condition $ a_{n}k_{n}^{3}\Delta _{n}^{1/2}=o(1)$ requires $k_{n}$ to obey $ k_{n}=o(\Delta _{n}^{(1/\beta -1)/3})$. Then, we can apply Theorem (ref) to show that the permutation test $\hat{\phi}_{n}$ is asymptotically valid for testing the discontinuity in the volatility process $\sigma _{t}$ at time $\tau $, regardless of whether the driving L\'{e}vy process is a Brownian motion or not. {To our knowledge, our test is the first in the literature that accommodates L\'evy-type shocks in the context of testing for volatility jumps.} $\square $
So far, we have illustrated the use of the permutation test for high-frequency asset returns data. Under the settings of Examples 1 and 2, the distributional change of asset returns is mainly driven by the time-$ \tau $ discontinuity in volatility, and hence, the permutation test is effectively a test for volatility jumps. Example 2, in particular, highlights the versatility and robustness of the permutation test compared with the conventional approach based on spot estimation. Going one step further, we now illustrate how to apply the permutation test to other types of economic variables.
Example 3 (Location-Scale Model for Volume). Consider a simple model for trading volume, under which the volume within the $i$th sampling interval\ is given by $Y_{n,i}=\mu _{i\Delta _{n}}+v_{i\Delta _{n}}\epsilon _{n,i}$. The $\mu_t$ location process captures the local mean, or trading intensity, and the $v_t$ scale process captures the time-varying heterogeneity in the order size. This location-scale model fits directly into the state-space model ((ref)) with $\zeta _{t}=(\mu _{t},v_{t})$, $ g\left( (\mu ,v),\epsilon \right) =\mu +v\epsilon $, and $R_{n,i}\equiv 0$. Let $\mathcal{F}_{t}$ be the filtration generated by the $\zeta _{t}$ process. If $\epsilon _{n,i}$ is independent of the $\zeta_t$ process and has finite second moment and bounded PDF, then it is easy to verify Assumptions (ref) and (ref) with $a_{n}=1$. Theorem (ref) thus implies that the permutation test is valid for testing the discontinuity in $\zeta _{t}=\left( \mu _{t},v_{t}\right) $ at time $\tau $ . $\square $
The location-scale structure in Example 3 is by no means essential in applications, because the permutation test is valid provided that the more general conditions in Assumptions (ref) and (ref) hold. This illustration is pedagogically convenient, in that it permits a straightforward verification of our high-level conditions. That being said, this example does reveal a limitation of our theory developed so far. That is, the data variable needs to be continuously distributed, as required in Assumption (ref)(ii) (which in turn is related to Assumption (ref)(ii)). Observed data in actual applications are invariably discrete, but this continuous-distribution assumption is often deemed as a reasonable approximation to reality. In some situations, however, the discreteness in the data is more salient. For example, the trading volume of a relatively illiquid asset may take values as small integer multiples of the lot size (e.g., 100 shares).\footnote{ This issue has become less important in the equity market as retail investors can now trade a single share, or even a fractional share, of a stock. However, the lot size is still relevant for less liquid assets such as option contracts or for equity data from earlier sample periods.} This motivates us to directly confront the discreteness in the data, as detailed in the next subsection.
The extension will be carried out in similar steps as the theory developed above. We start with modifying the general result in Theorem (ref) to accommodate discretely valued observations; we then specialize the general theory to a high-frequency setting under more primitive conditions. Recall that $Q_{j,n}\left( \cdot \right) $ denotes the $\mathcal{G}_{n}$-conditional distribution function of the coupling variable $U_{n,i}$ for $i\in \mathcal{I}_{j,n}$ and $j\in \left\{ 1,2\right\} $, and $\overline{Q}_{n}=(Q_{1,n}+Q_{2,n})/2$.
Theorem (ref) establishes exactly the same asymptotic properties for the permutation test as Theorem (ref), but under different conditions: it does not impose the anti-concentration requirement for the coupling variable (i.e., Assumption (ref)(ii)), and the \textquotedblleft distance\textquotedblright\ between the observed data and the coupling variable is measured by the probability mass of $\{\widetilde{Y}_{n,i}\neq U_{n,i}\}$. These modifications seem natural for the discrete-data setting.
Next, we specialize the general result in Theorem (ref) to the state-space model ((ref)), starting with some motivating examples. The first is an alternative model for the trading volume that explicitly features discretely valued data, which shows an interesting contrast to Example 3.
Example 4 (Poisson Model for Volume). Let $Y_{n,i}$ be the trading volume of an asset within the $i$th sampling interval. Following andersen1996, we model the discretely valued volume using a Poisson distribution with time-varying mean. To form a state-space representation, let $(\epsilon _{n,i}(t))_{t\geq 0}$ be a copy of the standard Poisson process on $\mathbb{R}_{+}$, independent across $i$, and let $\zeta _{t}$ be the time-varying mean process independent of the $\epsilon _{n,i}$'s. We then set $Y_{n,i}=\epsilon _{n,i}\left( \zeta _{i\Delta _{n}}\right) $, which, conditional on the $\zeta $ process, is Poisson distributed with mean $\zeta _{i\Delta _{n}}$. This representation is a special case of ((ref) ), with $g(\zeta,\epsilon) = \epsilon(\zeta)$ being a time-change and $ R_{n,i} = 0$. We also note that although the $\epsilon _{n,i}$'s are assumed to be i.i.d., the $(Y_{n,i})_{i\in\mathcal{I}_n}$ series can be highly persistent through its dependence on the stochastic mean process $\zeta _{t}$ . $\square $
To further broaden the empirical scope, we consider another example concerning the bid-ask spread of asset quotes. This example is econometrically interesting because of its resemblance to the discrete-choice models (e.g., probit and logit) commonly used for modeling binary and multinomial data.
Example 5 (Bid-Ask Spread). Let $Y_{n,i}$ be the bid-ask spread of an asset at time $i\Delta _{n}$. For a liquid asset, the spread is often maintained at 1 tick (e.g., 1 cent), but it may widen to several ticks due to a higher level of asymmetric information or dealer's inventory cost. For ease of exposition, we suppose that $Y_{n,i}$ is a binary variable taking values in $\{1,2\}$, while noting that a multinomial extension is straightforward. Motivated by the classical discrete-choice models, we model the spread as $Y_{n,i}=1+1\left\{ \zeta _{i\Delta _{n}}\geq \epsilon _{n,i}\right\} $, and suppose that the variables $(\epsilon _{n,i})_{i\in \mathcal{I}_{n}}$ are i.i.d.\ and independent of the $\zeta _{t}$ process. With the CDF of $\epsilon _{n,i}$ denoted by $F_{\epsilon }(\cdot )$, we have $\mathbb{P}\left( Y_{n,i}=2|\zeta _{i\Delta _{n}}\right) =F_{\epsilon }(\zeta _{i\Delta _{n}})$. Evidently, upon redefining $\zeta _{t}$ as $ F_{\epsilon }\left( \zeta _{t}\right) $, we can assume that $\epsilon _{n,i}$ is uniformly distributed on the $[0,1]$ interval without loss of generality. This normalization in turn allows us to interpret $\zeta _{t}$ as the stochastic propensity of a \textquotedblleft wide\textquotedblright\ spread, which may serve as a measure of market illiquidity. $\square $
We now proceed to establish the asymptotic validity of the permutation test for the hypotheses described in ((ref)) for discretely valued observations; see Theorem (ref) below. Since the state-space representation ((ref)) holds with the residual term $R_{n,i}=0$ in the examples above, it seems reasonable to avoid unnecessary redundancy by restricting our analysis to a simpler version given by
We replace Assumption (ref) with the following assumption, where we recall that for each $z\in \mathcal{Z}$, $F_{z}\left( \cdot \right) $ denotes the CDF of the random variable $g\left( z,\varepsilon _{n,i}\right) $ and $\tilde{\zeta}_{t}=\zeta _{t}-\Delta \zeta _{\tau }1\{t\geq \tau \}$.
Theorem (ref) depicts the same asymptotic behavior of the permutation test as in Theorem (ref). The sufficient conditions of these results differ mainly in how to gauge the closeness between the data and the coupling variable, as manifest in the difference between Assumption (ref)(i) and Assumption (ref)(i). The latter is easy to verify under more primitive conditions in concrete settings. Specifically, in Example 4, we note that $|g(z,\epsilon _{n,i})-g(z^{\prime },\epsilon _{n,i})|$ is a Poisson random variable with mean $\left\vert z-z^{\prime }\right\vert $, and hence, $\mathbb{P}(g(z,\epsilon _{n,i})\neq g(z^{\prime },\epsilon _{n,i}))=1-\exp (-\left\vert z-z^{\prime }\right\vert )\leq \left\vert z-z^{\prime }\right\vert $ as desired. In Example 5, we can use $ \epsilon _{n,i}\sim \text{Uniform}\left[ 0,1\right] $ to deduce that
which, again, verifies Assumption (ref)(i). Therefore, in the context of Examples 4 and 5 above, the permutation test is asymptotically valid for detecting discontinuities in trading activity and illiquidity, respectively.
Our Monte Carlo experiment is based on the setting of Example 2. We simulate the (log) price process according to $dP_{t}=\sigma _{t}dL_{t}$ under an Euler scheme on a 1-second mesh, and then resample the data at the $\Delta _{n}=1$ minute frequency. We simulate $ L$ either as a standard Brownian motion or as a (centered symmetric) stable process with index $\beta =1.5$. To avoid unrealistic price path, we truncate the stable distribution so that its normalized increment $\Delta _{n}^{-1/\beta }\left( L_{i\Delta _{n}}-L_{(i-1)\Delta _{n}}\right) $ is supported on $\left[ -C,C\right] $, and we consider $C\in \{10,20,30\}$ to examine the effect of the support. The unit of time is one day.
To simulate the volatility process, we first simulate two volatility factors according to the following dynamics (see BT):
where $B_{1,t}$ and $B_{2,t}$ are independent standard Brownian motions that are also independent of $L_{t}$, $\rho =-0.7$ captures the negative correlation between price and volatility shocks (namely the \textquotedblleft leverage\textquotedblright\ effect). { The $V_{1}$ volatility factor is highly persistent with a half-life of 2.5 months, while the $V_{2}$ volatility factor is quickly mean-reverting with a half-life of only one day.} The constant $c$ determines the size of the volatility jump at the event time $\tau $. In particular, $c=0$ corresponds to the null hypothesis, and we consider a range of $c$ values in $ (0,5]$ in order to trace out a power curve for the corresponding alternative hypotheses. The range of the $c$ parameter is calibrated according to fomc's\ (fomc) empirical estimates for FOMC announcements.\footnote{ Specifically, fomc estimate the average jump size of $\log (\sigma _{t})$ for the S&P 500 ETF around FOMC announcements to be 1.037 (see Table 3 of that paper). This suggests that $ \sigma _{\tau }^{2}/\sigma _{\tau -}^{2}=$ $(\exp (1.037))^{2}\approx 8$ on average, corresponding to $c\approx 3.5$ in this Monte Carlo design.}
We note that the two volatility factors, $V_{1}$ and $V_{2}$, capture the slow- and fast-mean-reverting volatility dynamics, respectively, with the former having \textquotedblleft smoother\textquotedblright\ sample paths than the latter. With this in mind, we simulate $\sigma _{t}$ using two models:
In finite samples, Model A features relatively smooth volatility paths, which is close to the \textquotedblleft ideal\textquotedblright\ scenario underlying the infill asymptotic theory. Meanwhile, Model B generates more realistic, and rougher, sample path for $\sigma $, providing a nontrivial challenge for the proposed inference theory.
We implement the permutation test at the 5% significance level, with the window size $k_{n}\in \{15,30,60,90\}$. The six-fold increase from the smallest window size to the largest one represents a considerable range that allows us to explore the robustness of the proposed test with respect to the $k_n$ tuning parameter.\footnote{We also implemented simulations in which the two subsamples, $\mathcal{I}_{1,n}$ and $\mathcal{I}_{2,n}$, have different sample sizes $k_{1,n},k_{2,n}\in \{15,30,60,90\}$. As anticipated in footnote (ref), these results are quantitatively similar to those with a common sample size, i.e., $k_{1,n}=k_{2,n}$. These additional results are omitted for brevity but are available upon request.} The critical value is computed as in Remark (ref) based on 1,000 i.i.d.\ permutations. For comparison, we also implement the standard (two-sided) t-test based on ((ref)). Rejection frequencies are computed based on {2,000} Monte Carlo trials.
We first examine the size properties of the permutation test $\hat{\phi}_n$ and the t-test based on ((ref)). Table (ref) reports the rejection frequencies of these tests under the null hypothesis (i.e., $c=0$) for various data generating processes. Column (1) corresponds to the case with $L$ being a standard Brownian motion, and columns (2), (3), and (4) report results when $L$ is a truncated stable process with the truncation parameter $C=10$, $20$, and $30$, respectively.
The top panel of the table shows results from Model A, where the volatility is solely driven by the “slow” factor. Quite remarkably, the rejection frequencies of the permutation test are very close to the 5% nominal level for all specifications of $L$ and, importantly, for a wide range of the window size $k_n$. In contrast, the rejection rates of the t-test appear to be far more sensitive to the choice of $k_n$. As we increase $k_n$ from 15 to 90, the rejection rate increases from 1.1% to 4.8% when $L$ is a Brownian motion. A similar pattern emerges when $L$ is a truncated stable process, except that the rejection rates now exceed the nominal level and reach 7.4% and 9.1% in columns (3) and (4), respectively. It is relevant to note that the t-test is not formally justified when $L$ is not a Brownian motion.
The more challenging case is Model B with the two-factor volatility dynamics. Looking at the bottom panel of Table (ref), we find that the permutation test still has rejection rates that are quite close to the nominal level, although we see a slight over-rejection of 6.4% when $k_n = 90$. This is likely due to the fact that the approximation error in the coupling has nontrivial impact when the window size is large. That being said, we note that the benchmark t-test is more severely affected by this bias issue, with rejection rates reaching 9.1% and 13.6% when $k_n = 60$ and $k_n = 90$, respectively.
Figure (ref) plots the power curves of the permutation test and the t-test for various $k_{n}$'s in Model A and Model B. We note that the four specifications of $L$ produce qualitatively similar results. We see that the rejection frequencies increase with the window size $k_{n}$ and the jump size $c$, which is expected from our consistency result obtained under $k_n\to \infty$. The permutation test appears to be less powerful than the t-test under the alternative hypothesis. This is a natural consequence of the typical trade-off between efficiency and robustness to the assumptions. The t-test is based on the spot variance estimator, which is “locally” the maximum-likelihood estimator of the spot variance under the Brownian shocks. The asymptotic validity and efficiency of the t-test rely on the Brownian assumption. In contrast, the permutation test is asymptotically valid regardless of whether the shocks are Brownian or not. As expected, this robustness is costly in terms of power. On the flip side, the power advantage of the t-test comes at the cost of size distortion when shocks are non-Brownian, which can be large, as shown in Table (ref).
Overall, we find that the permutation test controls size remarkably well under the null hypothesis. Although it appears to be less powerful than the t-test, it does not suffer from the latter's size distortion which can be severe in the two-factor volatility model. Our results suggest that, given its robustness, the permutation test is a useful complement to the conventional test based on spot estimation and asymptotic Gaussian approximation.
We apply the proposed permutation test to a recent sample of high-frequency price and volume observations of the S&P 500 ETF (NYSE: SPY) as an empirical illustration. The sampling frequency is one minute; the data source is the TAQ database. With the permutation test, we are interested in testing distributional discontinuities of the ETF's return, trading volume, and two measures of illiquidity for several FOMC announcements during the COVID-19 pandemic. This setting highlights one of the key merits of the proposed test, namely, it is applicable for a broad variety of high-frequency observations modeled in distinct ways. This is in sharp contrast to the conventional t-test based on (ref), which is designed specifically for testing volatility jumps and whose validity relies on the assumption of Brownian shocks. We construct the high-frequency volume series as the total number of shares within each one-minute trading session. The illiquidity measures of interest include Amihud's measure defined as the ratio between absolute return and {dollar} trading volume (amihud2002illiquidity) and the bid-ask spread averaged within each one-minute trading session.
We focus on four important FOMC announcements in the 2020--2021 sample period that are related to distinct aspects of the Federal Reserve's monetary policy during the COVID-19 pandemic. The first is the announcement made on March 3, 2020, which was also the first FOMC announcement after COVID-19 hit the United States. The Fed stated its decision to lower the federal funds rate by 1/2 percentage point as its first response to counter the pandemic's negative impact on the economy. The second event occurred on December 16, 2020. At that time, being concerned with the rising long-term yield, many market participants expected that the Fed might implement the so-called “operation twist” to tame the steepening of the yield curve. But this turned out not to be on the Fed's agenda, and so, the Fed's “inaction” may be deemed as a shock relative to the market's anticipation. The third case pertains to the announcement on March 17, 2021. During the press conference, the Fed Chairman suggested that the central bank would be unlikely to raise {the} rate in the next 2--3 years, which may be regarded as a forward guidance on the target rate. The final example is the announcement on September 22, 2021, when the Fed officially declared its intention to taper the large-scale asset purchase program.
Figure (ref) plots the asset return and trading volume of SPY over one-hour windows centered at these announcement times. For ease of comparison, we plot the return and volume data for the four events on the same scale. These plots immediately reveal the highly distinct market conditions at those times. This highlights the usefulness of adopting a high-frequency event-study research design, which allows us to investigate each event separately, rather than pooling information across different announcements under a likely fragile homogeneity assumption. We also observe several interesting patterns regarding how the market {responds} to the “lumpy” information embedded in the announcements. We generally see a rise in trading activity after the announcement. The price also tends to fluctuate more in the post-announcement window, although the March 3, 2020 event may be an exception as the market was already quite volatile even before the announcement.
As mentioned above, we implement the permutation test {described in Algorithm 1} on the ETF's returns, trading volume, Amihud's measure, and bid-ask spread constructed on the one-minute sampling frequency. For ease of interpretation, we consider the non-randomized version of the test described in Remark (ref). We consider two event windows: $k_n = 10$ {minutes} or $30$ {minutes}. Recall that each FOMC announcement contains two stages. The first is an immediate release of a short summary on the Federal Reserve's webpage, which will be further detailed in the Fed {Chairman}'s opening statement during the first (roughly) 10 minutes of the press conference. The second part is a Q&A session in which the {Chairman} {responds} to questions from the media, with the first few generally being more important. Given this setup, the shorter $k_n=10$ window allows us to focus on the immediate impact of the FOMC statement, whereas the longer $k_n=30$ window further covers the “more subtle” policy information conveyed to the public during the Q&A session. It is worth noting that the 30-minute event window is also adopted in prior work on the high-frequency identification of monetary policy shocks; see nakamura2018QJE.
Table (ref) reports the results of the permutation test. Recall that the permutation test applied to asset returns may be interpreted as a test for volatility jumps, as explained in Example 1 (with Brownian shocks) or Example 2 (with more general L\'evy shocks).\footnote{Prior work on volatility jumps (see, e.g., fomc and litodorovzhang2021) focuses exclusively on the case in which volatility is the scaling factor of Brownian shocks. Our permutation test is related to the prior work, but is valid under a more general notion of volatility with respect to non-Brownian shocks.} At a significance level of 10%, the permutation test rejects the null of continuity for all events except the one on March 3, 2020. The latter non-rejection is consistent with our previous observation that the volatility of SPY was high even before the announcement. Meanwhile, the test also strongly rejects the null hypothesis of distributional continuity for the volume series, echoing the burst of trading activity seen in Figure (ref).
The permutation test applied to the two illiquidity measures generates mixed results. We find some moderate evidence for the distributional discontinuity of Amihud's measure shortly after the March 3 and December 16 announcements in 2020. For the case with {a} 30-minute window, we do not reject the null of continuity for any of the announcements. The overall evidence {suggests} that the FOMC announcements under study did not lead to {an} abrupt change in the market impact coefficient gauged by Amihud's measure. Needless to say, this finding per se does not imply that the liquidity condition is unchanged after the announcement, as the notion of liquidity is a multifaceted concept. Indeed, we see that the permutation test applied to the bid-ask spread always strongly rejects the null of continuity. The post-announcement spread tends to be larger than its pre-announcement level, suggesting that it is significantly more costly to trade during {the} post-announcement trading session.
All in all, the empirical illustration above demonstrates how the proposed permutation test may be used to test for distributional discontinuities for a variety of market variables. This type of versatility is not easily attained by existing methods in the high-frequency econometrics literature. We also see that interesting empirical findings may be obtained even with a small number of observations, which confirms the {practical} relevance of allowing the $k_n$ window to be possibly fixed in our asymptotic theory for the permutation test.
In this paper, we propose using a permutation test to detect discontinuities in an economic model at a cutoff point. Relative to the existing literature, we show that the permutation test is well suited for event studies based on time-series data. While nonparametric t-tests have been widely used for this purpose in various empirical contexts, the permutation test proposed in this paper provides a distinct alternative. Instead of relying on asymptotic (mixed) Gaussianity from central limit theorems, we exploit finite-sample properties of the permutation test in the approximating, or “coupling”, two-sample problem.
We demonstrate that our new theory is broadly useful in a wide range of problems in the infill asymptotic time-series setting, which justifies using the permutation test to detect jumps in economic variables such as volatility, trading activity, and liquidity. Compared with the conventional nonparametric t-test, the proposed permutation test has several distinct features. First, the permutation test provides asymptotic size control regardless of whether the sizes of the local subsamples are fixed or growing to infinity. In the latter case, we also establish that the permutation test is consistent. Second, the permutation test is versatile, as it can be applied without modification to many different contexts and under relatively weak conditions.
Throughout the proofs, we use $K$ to denote a positive constant that may change from line to line, and write $K_{p}$ to emphasize its dependence on some parameter $p$. For any event $E\in \mathcal{F }$, we identify it with the associated indicator random variable.
As explained in Remark (ref), our main results extend beyond the Cram\'er-von Mises statistic in (ref). We characterize the relevant class of test statistics by the following high-level assumption.
Assumption (ref) is satisfied for a large class of test statistics, which includes the Cram\'er-von Mises and Kolmogorov-Smirnov statistics. In fact, the proof of Theorem (ref) shows that the Cram\'er-von Mises statistic satisfies Assumption (ref), and an analogous argument can be used to extend this to the Kolmogorov-Smirnov statistic. We now briefly describe the assumption. Assumption (ref)(a) is an essential ingredient to our methodology. In turn, Assumption (ref)(b) is a mild regularity condition that limits the influence that a few sample observations can have on the test statistic, and is only required to establish our consistency result.
The result proves Theorem (ref) for any test statistic that satisfies Assumption (ref). Since Theorem (ref) is the key to all of the results in the paper, this effectively implies that our findings extend to the class of statistics characterized by Assumption (ref), as claimed in Remark (ref).