In this talk, we will discuss the aforementioned paper from AAAI ‘23, which compares the effectiveness of transformer-based architectures to simple linear (NN) models and concludes that linear models outperform transformers in long-term time series forecasting tasks. They present a hypothesis as to why this is the case and provide a few suggestions for future work. We will discuss their experiments and results. We will also briefly go over a recent set of papers that use transformers, that have shown promising results in long-term time series forecasting, though with some major caveats. We will end with a discussion about a literature gap that exists in the domain of (long-term) time series forecasting, pertaining to the lack of comparison with ESN or even LSTM/GRU on these tasks.
Additional resources: https://arxiv.org/abs/2205.13504
NOTE: This talk follows the talk on September 12. The slides below are the same for both talks.