Um estudo experimental sobre estratégias de coleta de dados em pipelines meteorológicos
Resumo
Dados meteorológicos são essenciais para diferentes tipos de aplicações, tornando a eficiência na coleta um fator crítico em pipelines de dados. Este trabalho apresenta um estudo comparativo entre três estratégias de acesso aos dados do ECMWF: a Web API oficial, a coleta via HTTP e o acesso por objetos compatíveis com S3. O objetivo é identificar a abordagem mais eficiente em cenários de atualizações frequentes. Foram conduzidos experimentos controlados com o Apache JMeter, para avaliar métricas de tempo, throughput e consumo de recursos sob carga concorrente. Os resultados visam orientar a escolha de arquiteturas de coleta que otimizem o desempenho e a confiabilidade de pipelines meteorológicos operacionais.
Palavras-chave:
Pipeline de dados, Coleta de dados, Dados meteorológicos, Estudo comparativo, ECMWF
Referências
Apache Software Foundation (2026). Apache JMeter. [link]. Acessado em: 16 de março de 2026.
Bauer, P., Thorpe, A., and Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature, 525(7567):47–55.
Bocchi, E., Mellia, M., and Sarni, S. (2014). Cloud storage service benchmarking: Methodologies and experimentations. In Proc. 3rd IEEE International Conference on Cloud Networking (CloudNet), pages 395–400, Luxembourg.
ECMWF (2024a). Ecmwf web api documentation. [link]. Acessado em: 14 de março de 2026.
ECMWF (2024b). Grib: Edition 2 - ecmwf parameter database. Acessado em: 13 mar. 2026.
ECMWF (2026a). About ecmwf. Acesso em: 10 fev. 2026.
ECMWF (2026b). Documentation and support: Changes to the ECMWF forecasting system. Acessado em: 23 mar. 2026.
ECMWF (2026c). Ecmwf open data. Acessado em: 13 mar. 2026.
ECMWF (2026d). S2S: Sub-seasonal to Seasonal Prediction Project. Acedido em: 17 mar. 2026.
ECMWF (2026e). TIGGE: Thorpex Interactive Grand Global Ensemble. Acedido em: 17 mar. 2026.
Fielding, R., Nottingham, M., and Reschke, J. (2022). HTTP semantics. RFC 9110. Acessado em: Acesso em: 10 jun. 2026.
Gallagher, J., Habermann, T., and Lee, J. (2024). Web accessible apis in the cloud trade study (task 28). Technical report, OPeNDAP / NASA.
Gowan, J., Habermann, T., and Jelenak, A. (2022a). Advancing cloud-native access to meteorological data. Journal of Open Source Software, 7(72).
Gowan, T. A., Horel, J. D., Jacques, A. A., et al. (2022b). Using cloud computing to analyze model output archived in Zarr format. Journal of Atmospheric and Oceanic Technology, 39(4):449–462.
Heise, E. (2019). GRIB2 for DUMMIES. COSMO Consortium General Meeting. Disponível em: [link]. Acesso em: 10 jun. 2026.
Khemka, A. and Raj, G. (2025). Unstructured data ingestion: Best practices for acquiring, storing, and processing data from 200+ external sources. International Journal of Research and Analytical Reviews (IJRAR), 12(1).
Kleppmann, M. (2017). Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. O’Reilly Media.
Pargaonkar, S. (2023). A comprehensive review of performance testing methodologies and best practices: Software quality engineering. International Journal of Science and Research (IJSR), 12(8):2008–2014.
Rao, N. S. V., Liu, Q., Liu, Z., Kettimuthu, R., and Foster, I. (2019). Throughput analytics of data transfer infrastructures. In Testbeds and Research Infrastructures for the Development of Networks and Communities, volume 270 of Lecture Notes in Computer Science, pages 20–40. Springer.
Saeedizade, E., Zhang, B., and Arslan, E. (2023). Demystifying the performance of data transfers in High-Performance research networks. In 2023 IEEE 25th International Conference on High Performance Computing, Data, and Analytics. IEEE.
Schwitalla, T., Warrach-Sagi, K., Wulfmeyer, V., and Bauer, H.-S. (2017). Continuous high-resolution midlatitude belt simulations for july-august 2013 with the wrf model. Geoscientific Model Development, 10(5):2031–2055.
WMO (2019). Manual on Codes: International Codes, Volume I.2, Part B – Binary Codes. World Meteorological Organization, Geneva, Switzerland. WMO-No. 306.
Bauer, P., Thorpe, A., and Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature, 525(7567):47–55.
Bocchi, E., Mellia, M., and Sarni, S. (2014). Cloud storage service benchmarking: Methodologies and experimentations. In Proc. 3rd IEEE International Conference on Cloud Networking (CloudNet), pages 395–400, Luxembourg.
ECMWF (2024a). Ecmwf web api documentation. [link]. Acessado em: 14 de março de 2026.
ECMWF (2024b). Grib: Edition 2 - ecmwf parameter database. Acessado em: 13 mar. 2026.
ECMWF (2026a). About ecmwf. Acesso em: 10 fev. 2026.
ECMWF (2026b). Documentation and support: Changes to the ECMWF forecasting system. Acessado em: 23 mar. 2026.
ECMWF (2026c). Ecmwf open data. Acessado em: 13 mar. 2026.
ECMWF (2026d). S2S: Sub-seasonal to Seasonal Prediction Project. Acedido em: 17 mar. 2026.
ECMWF (2026e). TIGGE: Thorpex Interactive Grand Global Ensemble. Acedido em: 17 mar. 2026.
Fielding, R., Nottingham, M., and Reschke, J. (2022). HTTP semantics. RFC 9110. Acessado em: Acesso em: 10 jun. 2026.
Gallagher, J., Habermann, T., and Lee, J. (2024). Web accessible apis in the cloud trade study (task 28). Technical report, OPeNDAP / NASA.
Gowan, J., Habermann, T., and Jelenak, A. (2022a). Advancing cloud-native access to meteorological data. Journal of Open Source Software, 7(72).
Gowan, T. A., Horel, J. D., Jacques, A. A., et al. (2022b). Using cloud computing to analyze model output archived in Zarr format. Journal of Atmospheric and Oceanic Technology, 39(4):449–462.
Heise, E. (2019). GRIB2 for DUMMIES. COSMO Consortium General Meeting. Disponível em: [link]. Acesso em: 10 jun. 2026.
Khemka, A. and Raj, G. (2025). Unstructured data ingestion: Best practices for acquiring, storing, and processing data from 200+ external sources. International Journal of Research and Analytical Reviews (IJRAR), 12(1).
Kleppmann, M. (2017). Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. O’Reilly Media.
Pargaonkar, S. (2023). A comprehensive review of performance testing methodologies and best practices: Software quality engineering. International Journal of Science and Research (IJSR), 12(8):2008–2014.
Rao, N. S. V., Liu, Q., Liu, Z., Kettimuthu, R., and Foster, I. (2019). Throughput analytics of data transfer infrastructures. In Testbeds and Research Infrastructures for the Development of Networks and Communities, volume 270 of Lecture Notes in Computer Science, pages 20–40. Springer.
Saeedizade, E., Zhang, B., and Arslan, E. (2023). Demystifying the performance of data transfers in High-Performance research networks. In 2023 IEEE 25th International Conference on High Performance Computing, Data, and Analytics. IEEE.
Schwitalla, T., Warrach-Sagi, K., Wulfmeyer, V., and Bauer, H.-S. (2017). Continuous high-resolution midlatitude belt simulations for july-august 2013 with the wrf model. Geoscientific Model Development, 10(5):2031–2055.
WMO (2019). Manual on Codes: International Codes, Volume I.2, Part B – Binary Codes. World Meteorological Organization, Geneva, Switzerland. WMO-No. 306.
Publicado
08/09/2026
Como Citar
GONÇALVES, Hiara F.; SILVA, Gabriel C. R. T.; VIANA, Vinícius; B. DA SILVA MOTA, Flávio; SOUZA, Vanessa C. O.; V. DE PAULA, Melise Maria.
Um estudo experimental sobre estratégias de coleta de dados em pipelines meteorológicos. In: SIMPÓSIO BRASILEIRO DE BANCO DE DADOS (SBBD), 41. , 2026, São Carlos/SP.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 536-549.
ISSN 2763-8979.
DOI: https://doi.org/10.5753/sbbd.2026.249253.
