Lena Sasal, Tanujit Chakraborty, Abdenour Hadid
arXiv 8 Sep 2022 · Machine Learning · 40 citations (OpenAlex)
arXiv:2209.03945 · PDF · DOI · OpenAlex · Extracted main text
Deep learning utilizing transformers has recently achieved a lot of success in many vital areas such as natural language processing, computer vision, anomaly detection, and recommendation systems, among many others. Among several merits of transformers, the ability to capture long-range temporal dependencies and interactions is desirable for time series forecasting, leading to its progress in various time series applications. In this paper, we build a transformer model for non-stationary time series. The problem is challenging yet crucially important. We present a novel framework for univariate time series representation learning based on the wavelet-based transformer encoder architecture and call it W-Transformer. The proposed W-Transformers utilize a maximal overlap discrete wavelet transformation (MODWT) to the time series data and build local transformers on the decomposed datasets to vividly capture the nonstationarity and long-range nonlinear dependencies in the time series. Evaluating our framework on several publicly available benchmark time series datasets from various domains and with diverse characteristics, we demonstrate that it performs, on average, significantly better than the baseline forecasters for short-term and long-term forecasting, even for datasets that consist of only a few hundred training samples.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | D. B. Percival and A. T. Walden, Wavelet methods for time series ana… (2000) vol. 4 | 1.000 | 6 | 3 | 100% |
| 2 | M. Panja, T. Chakraborty, U. Kumar, and N. Liu, “Epicasting: An ense… (2022) Epicasting: An ensemble wavelet neural network (ewnet) for forecasting epidemics | 1.000 | 5 | 3 | 100% |
| 3 | A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gom… (2017) Attention is all you need | 1.000 | 5 | 3 | 100% |
| 4 | R. Hyndman, A. B. Koehler, J. K. Ord, and R. D. Snyder, Forecasting… (2008) | 0.811 | 4 | 2 | 100% |
| 5 | M. Aminghafari and J.-M. Poggi, “Forecasting time series using wavel… (2007) Forecasting time series using wavelets | 0.737 | 3 | 2 | 100% |
| 6 | J. Faraway and C. Chatfield, “Time series forecasting with neural ne… (1998) Time series forecasting with neural networks: a comparative study using the air line data | 0.737 | 3 | 2 | 100% |
| 7 | R. J. Hyndman and G. Athanasopoulos, Forecasting: principles and pra… (2018) | 0.737 | 3 | 2 | 100% |
| 8 | D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski, “Deepar:… (2020) Deepar: Probabilistic forecasting with autoregressive recurrent networks | 0.737 | 3 | 2 | 100% |
| 9 | G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff,… (2021) A transformer-based framework for multivariate time series representation learning | 0.737 | 3 | 2 | 100% |
| 10 | R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-… (2021) Monash time series forecasting archive | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 48 scored citations.