Prediction of Stock Price Time Series using Transformers
Resumo
This work presents an implementation of the Transformer on the problem of predicting stock prices from time series. The model is compared with ARIMA and a neural network with LSTM cells. We hypothesize that, due to the powerful memory capacity and association between series values, the Transformer would be able to achieve better results than other shallow or deep solutions. The data used in the experiments is the average daily prices of 8 shares of the Ibovespa index in the period of 2008. The obtained results corroborated the hypothesis of superiority of the Transformer which predicted the stock prices with higher accuracy in 60% of the times.
Referências
R. Batra and S. M. Daudpota. Integrating stocktwits with sentiment analysis for better prediction of stock price movement. In 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), pages 1–5. IEEE, 2018.
G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
S. J. Brown, W. N. Goetzmann, and A. Kumar. The dow theory: William peter hamilton’s track record reconsidered. The Journal of finance, 53(4):1311–1333, 1998.
M. Caron and O. Müller. Hardening soft information: A transformer-based approach to forecasting stock return volatility. In 2020 IEEE International Conference on Big Data (Big Data), pages 4383–4391. IEEE, 2020.
F. Chollet et al. Keras, 2015. URL https://github.com/fchollet/keras.
D. Daiya and C. Lin. Stock movement prediction and portfolio management via multi-modal learning with transformer. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3305–3309. IEEE, 2021.
Q. Ding, S. Wu, H. Sun, J. Guo, and J. Guo. Hierarchical multi-scale gaussian transformer for stock movement prediction. In IJCAI, pages 4640–4646, 2020.
Y. Li, S. Lv, X. Liu, and Q. Zhang. Incorporating transformers and attention networks for stock movement prediction. Complexity, 2022, 2022.
B. Lim, S. Ö. Arık, N. Loeff, and T. Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4):1748–1764, 2021.
J. Liu, H. Lin, X. Liu, B. Xu, Y. Ren, Y. Diao, and L. Yang. Transformer-based capsule network for stock movement prediction. In Proceedings of the First Workshop on Financial Technology and Natural Language Processing, pages 66–73, 2019.
B. G. Malkiel and E. F. Fama. Efficient capital markets: A review of theory and empirical work. The journal of Finance, 25(2):383–417, 1970.
D. Othan, Z. H. Kilimci, and M. Uysal. Financial sentiment analysis for predicting direction of stocks using bidirectional encoder representations from transformers (bert) and deep learning models. In Proc. Int. Conf. Innov. Intell. Technol., volume 2019, pages 30–35, 2019.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
E. Ramos-Pérez, P. J. Alonso-González, and J. J. Núñez-Velázquez. Multi-transformer: A new neural network-based architecture for forecasting s&p volatility. Mathematics, 9(15):1794, 2021.
T. G. Smith et al. pmdarima: Arima estimators for Python, 2017. URL http://www.alkaline-ml.com/pmdarima. [Online; Acessado em 12/11/2022].
G. Van Rossum and F. L. Drake Jr. Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam, 1995.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
S. J. Brown, W. N. Goetzmann, and A. Kumar. The dow theory: William peter hamilton’s track record reconsidered. The Journal of finance, 53(4):1311–1333, 1998.
M. Caron and O. Müller. Hardening soft information: A transformer-based approach to forecasting stock return volatility. In 2020 IEEE International Conference on Big Data (Big Data), pages 4383–4391. IEEE, 2020.
F. Chollet et al. Keras, 2015. URL https://github.com/fchollet/keras.
D. Daiya and C. Lin. Stock movement prediction and portfolio management via multi-modal learning with transformer. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3305–3309. IEEE, 2021.
Q. Ding, S. Wu, H. Sun, J. Guo, and J. Guo. Hierarchical multi-scale gaussian transformer for stock movement prediction. In IJCAI, pages 4640–4646, 2020.
Y. Li, S. Lv, X. Liu, and Q. Zhang. Incorporating transformers and attention networks for stock movement prediction. Complexity, 2022, 2022.
B. Lim, S. Ö. Arık, N. Loeff, and T. Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4):1748–1764, 2021.
J. Liu, H. Lin, X. Liu, B. Xu, Y. Ren, Y. Diao, and L. Yang. Transformer-based capsule network for stock movement prediction. In Proceedings of the First Workshop on Financial Technology and Natural Language Processing, pages 66–73, 2019.
B. G. Malkiel and E. F. Fama. Efficient capital markets: A review of theory and empirical work. The journal of Finance, 25(2):383–417, 1970.
D. Othan, Z. H. Kilimci, and M. Uysal. Financial sentiment analysis for predicting direction of stocks using bidirectional encoder representations from transformers (bert) and deep learning models. In Proc. Int. Conf. Innov. Intell. Technol., volume 2019, pages 30–35, 2019.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
E. Ramos-Pérez, P. J. Alonso-González, and J. J. Núñez-Velázquez. Multi-transformer: A new neural network-based architecture for forecasting s&p volatility. Mathematics, 9(15):1794, 2021.
T. G. Smith et al. pmdarima: Arima estimators for Python, 2017. URL http://www.alkaline-ml.com/pmdarima. [Online; Acessado em 12/11/2022].
G. Van Rossum and F. L. Drake Jr. Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam, 1995.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Publicado
06/08/2023
Como Citar
COSTA, Lorenzo D.; MACHADO, Alexei M. C..
Prediction of Stock Price Time Series using Transformers. In: BRAZILIAN WORKSHOP ON ARTIFICIAL INTELLIGENCE IN FINANCE (BWAIF), 2. , 2023, João Pessoa/PB.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 85-95.
DOI: https://doi.org/10.5753/bwaif.2023.230239.
