Ava C. Blake, Nivika A. Gandhi, Anurag R. Jakkula
arXiv 21 Sep 2025 · Finance — Statistical Finance
arXiv:2510.03236 · PDF · DOI · OpenAlex · Extracted main text
Accurate prediction of financial market volatility is critical for risk management, derivatives pricing, and investment strategy. In this study, we propose a multitude of regime-switching methods to improve the prediction of S&P 500 volatility by capturing structural changes in the market across time. We use eleven years of SPX data, from May 1st, 2014 to May 27th, 2025, to compute daily realized volatility (RV) from 5-minute intraday log returns, adjusted for irregular trading days. To enhance forecast accuracy, we engineered features to capture both historical dynamics and forward-looking market sentiment across regimes. The regime-switching methods include a soft Markov switching algorithm to estimate soft-regime probabilities, a distributional spectral clustering method that uses XGBoost to assign clusters at prediction time, and a coefficient-based soft regime algorithm that extracts HAR coefficients from time segments segmented through the Mood test and clusters through Bayesian GMM for soft regime weights, using XGBoost to predict regime probabilities. Models were evaluated across three time periods--before, during, and after the COVID-19 pandemic. The coefficient-based clustering algorithm outperformed all other models, including the baseline autoregressive model, during all time periods. Additionally, each model was evaluated on its recursive forecasting performance for 5- and 10-day horizons during each time period. The findings of this study demonstrate the value of regime-aware modeling frameworks and soft clustering approaches in improving volatility forecasting, especially during periods of heightened uncertainty and structural change.
appendix boundary found by appendix_command · 89% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Zhang, Y., L. Lei, and Y. Wei (2020) Forecasting the chinese stock market volatility with international market volatilities: The role of regime switching | 0.928 | 4 | 3 | 100% |
| 2 | Prakash, A., N. James, M. Menzies, and G. Francis (2021, May) (2021) Structural clustering of volatility regimes for dynamic trading strategies | 0.405 | 1 | 1 | 100% |
| 3 | Corsi, F (2009) A simple approximate long-memory model of realized volatility | 0.405 | 1 | 1 | 100% |
| 4 | Ding, Y., D. Kambouroudis, and D. G. McMillan (2025) Forecasting realised volatility using regime-switching models | 0.405 | 1 | 1 | 100% |
| 5 | Hu, N., X. Yin, and Y. Yao (2025) A novel har-type realized volatility forecasting model using graph neural network | 0.405 | 1 | 1 | 100% |
| 6 | Li, X., D. Li, Y. Cheng, and W. Li (2024) Forecasting the volatility of educational firms based on har model and lstm models considering sentiment and educational policy | 0.405 | 1 | 1 | 100% |
| 7 | Luo, J., T. Klein, Q. Ji, and C. Hou (2022) Forecasting realized volatility of agricultural commodity futures with infinite hidden markov har models | 0.405 | 1 | 1 | 100% |
Showing the top 7 of 7 scored citations.