Estimating Forecast Difficulty in N-BEATS using Latent Representations and Meta-Learning

• Lauryane Santos Siqueira UFS
• Leonardo Nogueira Matos UFS
• Paulo Salgado Gomes de Mattos Neto UFPE
• Filipe Coelho de Lima Duarte UFPB

DOI: https://doi.org/10.5753/eniac.2026.31201

Resumo

Forecast difficulty estimation is an important challenge for improving the reliability of time series forecasting systems. This work investigates whether latent representations extracted from the interpretable N-BEATS model, time series characteristics, and recent error history can be combined to estimate the difficulty of individual forecasts. We formulate this task as a supervised meta-learning problem in which each forecast is represented as an instance in a meta-dataset. Experiments on the M4 benchmark show that supervised metamodels consistently outperform latent-space and signal-based baselines, achieving Spearman correlations up to 0.895, ROC-AUC values up to 0.960, and F1-scores up to 0.780. Ablation analyses indicate that recent error history is the most informative source of information, while latent representations provide complementary signals. Furthermore, selective prediction experiments demonstrate that the proposed scores effectively prioritize reliable forecasts, reducing prediction risk under low-coverage scenarios.

Referências

Arvanitidis, G., Hansen, L. K., and Hauberg, S. (2017). Latent space oddity: on the curvature of deep generative models. arXiv preprint arXiv:1710.11379.

Bojanowski, P., Joulin, A., Lopez-Paz, D., and Szlam, A. (2017). Optimizing the latent space of generative networks. arXiv preprint arXiv:1707.05776.

Bradley, A. P. (1997). The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7):1145–1159.

Casolaro, A., Capone, V., Iannuzzo, G., and Camastra, F. (2023). Deep learning for time series forecasting: Advances and open problems. Information, 14(11):598.

Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., et al. (2015). Xgboost: extreme gradient boosting. R package version 0.4-2, 1(4):1–4.

Chinchor, N. and Sundheim, B. M. (1993). Muc-5 evaluation metrics. In Fifth Message Understanding Conference (MUC-5): Proceedings of a Conference Held in Baltimore, Maryland, August 25-27, 1993.

Cox, D. R. (1958). The regression analysis of binary sequences. Journal of the Royal Statistical Society Series B: Statistical Methodology, 20(2):215–232.

Ferchichi, A., Chihaoui, M., Toujani, R., Ferchichi, A., and Hendaoui, F. (2025). Evidential uncertainty quantification with multiple deep learning architectures for spatiotemporal drought forecasting. Neural Computing and Applications, 37(15):8773–8797.

Freeborough, W. and van Zyl, T. (2022). Investigating explainability methods in recurrent neural network architectures for financial time series data. Applied Sciences, 12(3):1427.

Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. (2023). A survey of uncertainty in deep neural networks. Artificial intelligence review, 56(Suppl 1):1513–1589.

Geifman, Y. and El-Yaniv, R. (2017). Selective classification for deep neural networks.

Guo, T., Lin, T., and Lu, Y. (2018). An interpretable lstm neural network for autoregressive exogenous model. arXiv preprint arXiv:1804.05251.

Hyndman, R. J. and Athanasopoulos, G. (2021). Forecasting: Principles and Practice. OTexts, 3 edition.

Hyndman, R. J. and Koehler, A. B. (2006). Another look at measures of forecast accuracy. International journal of forecasting, 22(4):679–688.

Jing, T., Zheng, P., Xia, L., and Liu, T. (2022). Transformer-based hierarchical latent space vae for interpretable remaining useful life prediction. Advanced Engineering Informatics, 54:101781.

Makridakis, S., Spiliotis, E., and Assimakopoulos, V. (2020). The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1):54–74. M4 Competition.

Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y. (2020). N-beats: Neural basis expansion analysis for interpretable time series forecasting.

Papacharalampous, G., Tyralis, H., Pechlivanidis, I. G., Grimaldi, S., and Volpi, E. (2022). Massive feature extraction for explaining and foretelling hydroclimatic time series forecastability at the global scale. Geoscience Frontiers, 13(3):101349.

Ponce-Flores, M., Frausto-Solís, J., Santamaría-Bonfil, G., Pérez-Ortega, J., and González-Barbosa, J. J. (2020). Time series complexities and their relationship to forecasting performance. Entropy, 22(1):89.

Postels, J., Blum, H., Cadena, C., Siegwart, R., Van Gool, L., and Tombari, F. (2020). Quantifying aleatoric and epistemic uncertainty using density estimation in latent space. arXiv preprint arXiv:2012.03082, 1.

Quinlan, J. R. (1996). Learning decision tree classifiers. ACM Computing Surveys (CSUR), 28(1):71–72.

Reynolds, D. A. et al. (2009). Gaussian mixture models. Encyclopedia of biometrics, 741(659-663):3.

Rojat, T., Puget, R., Filliat, D., Del Ser, J., Gelin, R., and Díaz-Rodríguez, N. (2021). Explainable artificial intelligence (xai) on timeseries data: A survey. arXiv preprint arXiv:2104.00950.

Rongali, S., Rose, A. J., McManus, D. D., Bajracharya, A. S., Kapoor, A., Granillo, E., and Yu, H. (2020). Learning latent space representations to predict patient outcomes: Model development and validation. Journal of Medical Internet Research, 22(3):e16374.

Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. The Annals of Mathematical Statistics, pages 832–837.

Song, X., Deng, L., Wang, H., Zhang, Y., He, Y., and Cao, W. (2024). Deep learning-based time series forecasting. Artificial Intelligence Review, 58(1):23.

Spearman, C. (1961). The proof and measurement of association between two things.

Van Amersfoort, J., Smith, L., Teh, Y. W., and Gal, Y. (2020). Uncertainty estimation using a single deep deterministic neural network. In International conference on machine learning, pages 9690–9700. PMLR.

Weisberg, S. (2001). Yeo-johnson power transformations. Department of Applied Statistics, University of Minnesota. Retrieved June, 1(2003):8.

Yunita, A., Pratama, M. I., Almuzakki, M. Z., Ramadhan, H., Akhir, E. A. P., Mansur, A. B. F., and Basori, A. H. (2025). Performance analysis of neural network architectures for time series forecasting: A comparative study of rnn, lstm, gru, and hybrid models. MethodsX, 15:103462.

Zhou, H., Van Landeghem, J., Popordanoska, T., and Blaschko, M. B. (2024). A novel characterization of the population area under the risk coverage curve (aurc) and rates of finite sample estimators. arXiv preprint arXiv:2410.15361.