Cloud Segmentation in Multispectral Images from the Sentinel-2 Satellite: A Comparative Study of Deep Learning Approaches
Abstract
To enhance the processing of multispectral images under adverse conditions, such as the presence of clouds, this study compared the neural networks U-Net, DeepLabV3+, and SegFormer using images from the Sentinel-2 satellite, evaluating their performance through metrics such as Intersection over Union (IoU) and Dice coefficient. The results indicate that SegFormer achieved the highest accuracy in cloud detection but with a longer inference time, while U-Net demonstrated a balance between accuracy and efficiency, and DeepLabV3+ stood out for its shorter processing time but lower performance. The study highlights the relevance of neural networks in remote sensing and indicates the need for model optimization.
References
Azad, R., Heidary, M., Yilmaz, K., Hüttemann, M., Karimijafarbigloo, S., Wu, Y., Schmeink, A., and Merhof, D. (2023). Loss functions in the era of semantic segmentation: A survey and outlook. arXiv preprint arXiv:2312.05391.
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818.
Choi, J., Seo, D., Jung, J., Han, Y., Oh, J., and Lee, C. (2024). Cloud detection using a unet3+ model with a hybrid swin transformer and efficientnet (unet3+ ste) for very-high-resolution satellite imagery. Remote Sensing, 16(20):3880.
Domnich, M., Sünter, I., Trofimov, H., Wold, O., Harun, F., Kostiukhin, A., Järveoja, M., Veske, M., Tamm, T., Voormansik, K., et al. (2021). Kappamask: Ai-based cloudmask processor for sentinel-2. Remote Sensing, 13(20):4100.
Farnaz, Nuthammachot, N., Shabbir, R., and Iqbal, B. (2025). Pixel and region-oriented classification of sentinel-2 imagery to assess lulc dynamics and their climate impact in nowshera, pakistan. Open Geosciences, 17(1):20220745.
Francis, A., Mrziglod, J., Sidiropoulos, P., and Muller, J.-P. (2020). Sentinel-2 cloud mask catalogue.
Gui, S., Song, S., Qin, R., and Tang, Y. (2024). Remote sensing object detection in the deep learning era—a review. Remote Sensing, 16(2).
He, M., Zhang, J., He, Y., Zuo, X., and Gao, Z. (2024). Annotated dataset for training cloud segmentation neural networks using high-resolution satellite remote sensing imagery. Remote Sensing, 16(19):3682.
Janga, B., Asamani, G. P., Sun, Z., and Cristea, N. (2023). A review of practical ai for remote sensing in earth sciences. Remote Sensing, 15(16).
Koonce, B. (2021). Convolutional Neural Networks with Swift for Tensorflow. Springer.
Langhorst, T., Andreadis, K. M., and Allen, G. H. (2024). Global cloud biases in optical satellite remote sensing of rivers. Geophysical Research Letters, 51(16):e2024GL110085.
Li, J., Wu, Z., Hu, Z., Jian, C., Luo, S., Mou, L., Zhu, X. X., and Molinier, M. (2021). A lightweight deep learning-based cloud detection method for sentinel-2a imagery fusing multiscale spectral and spatial features. IEEE Transactions on Geoscience and Remote Sensing, 60:1–19.
Liu, C.-C., Zhang, Y.-C., Chen, P.-Y., Lai, C.-C., Chen, Y.-H., Cheng, J.-H., and Ko, M.-H. (2019). Clouds classification from sentinel-2 imagery with deep residual learning and semantic image segmentation. Remote Sensing, 11(2):119.
Liu, Y., Zhang, Y., Wang, Y., Hou, F., Yuan, J., Tian, J., Zhang, Y., Shi, Z., Fan, J., and He, Z. (2023). A survey of visual transformers. IEEE Transactions on Neural Networks and Learning Systems.
Luotamo, M., Metsämäki, S., and Klami, A. (2020). Multiscale cloud detection in remote sensing images using a dual convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 59(6):4972–4983.
Misra, G., Cawkwell, F., and Wingler, A. (2020). Status of phenological research using sentinel-2 data: A review. Remote Sensing, 12(17).
Ozdogan, M., Wang, S., Ghose, D., Fraga, E. P., Fernandes, A. M., and Varela, G. (2024). Field-scale rice area and yield mapping in sri lanka with optical remote sensing and limited training data. Available at SSRN 4940849.
Robinson, T. R., Rosser, N., and Walters, R. J. (2019). The spatial and temporal influence of cloud cover on satellite-based emergency mapping of earthquake disasters. Scientific reports, 9(1):12455.
Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer.
Saifullah, S., Dreżewski, R., and Yudhana, A. (2025). Advanced brain tumor segmentation using deeplabv3plus with xception encoder on a multi-class mr image dataset. Multimedia Tools and Applications, pages 1–22.
Santos, M., Paula, M., and Souza, V. (2024). Extração de séries espaço-temporais a partir de um cubo de dados geoespacial em benefício da cafeicultura mineira. In Anais do XV Workshop de Computação Aplicada à Gestão do Meio Ambiente e Recursos Naturais, pages 211–214, Porto Alegre, RS, Brasil. SBC.
Sourget, T., Hasany, S. N., Mériaudeau, F., and Petitjean, C. (2023). Can segformer be a true competitor to u-net for medical image segmentation? In Annual Conference on Medical Image Understanding and Analysis, pages 111–118. Springer.
Uchegbulam, O., Ameloko, A., and Omo-Irabor, O. (2021). Effect of cloud cover on land use land cover dynamics using remotely sensed data of western niger delta, nigeria. Journal of Applied Sciences and Environmental Management, 25(5):799–804.
Ventura, R., Musis, C., Ventura, T., and II, A. C. (2023). Rsdd: um conjunto de dados para modelagem de desmatamentos. In Anais do XIV Workshop de Computação Aplicada à Gestão do Meio Ambiente e Recursos Naturais, pages 159–162, Porto Alegre, RS, Brasil. SBC.
Wang, Z., Zhao, L., Meng, J., Han, Y., Li, X., Jiang, R., Chen, J., and Li, H. (2024). Deep learning-based cloud detection for optical remote sensing images: A survey. Remote Sensing, 16(23):4583.
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., and Luo, P. (2021). Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:12077–12090.
Xu, K., Wang, W., Deng, X., Wang, A., Wu, B., and Jia, Z. (2024). Transga-net: Integration transformer with gradient-aware feature aggregation for accurate cloud detection in remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 21:1–5.
Xu, X., He, W., Xia, Y., Zhang, H., Wu, Y., Jiang, Z., and Hu, T. (2025). Tanet: Thin cloud-aware network for cloud detection in optical remote sensing image. IEEE Transactions on Geoscience and Remote Sensing, 63:1–16.
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818.
Choi, J., Seo, D., Jung, J., Han, Y., Oh, J., and Lee, C. (2024). Cloud detection using a unet3+ model with a hybrid swin transformer and efficientnet (unet3+ ste) for very-high-resolution satellite imagery. Remote Sensing, 16(20):3880.
Domnich, M., Sünter, I., Trofimov, H., Wold, O., Harun, F., Kostiukhin, A., Järveoja, M., Veske, M., Tamm, T., Voormansik, K., et al. (2021). Kappamask: Ai-based cloudmask processor for sentinel-2. Remote Sensing, 13(20):4100.
Farnaz, Nuthammachot, N., Shabbir, R., and Iqbal, B. (2025). Pixel and region-oriented classification of sentinel-2 imagery to assess lulc dynamics and their climate impact in nowshera, pakistan. Open Geosciences, 17(1):20220745.
Francis, A., Mrziglod, J., Sidiropoulos, P., and Muller, J.-P. (2020). Sentinel-2 cloud mask catalogue.
Gui, S., Song, S., Qin, R., and Tang, Y. (2024). Remote sensing object detection in the deep learning era—a review. Remote Sensing, 16(2).
He, M., Zhang, J., He, Y., Zuo, X., and Gao, Z. (2024). Annotated dataset for training cloud segmentation neural networks using high-resolution satellite remote sensing imagery. Remote Sensing, 16(19):3682.
Janga, B., Asamani, G. P., Sun, Z., and Cristea, N. (2023). A review of practical ai for remote sensing in earth sciences. Remote Sensing, 15(16).
Koonce, B. (2021). Convolutional Neural Networks with Swift for Tensorflow. Springer.
Langhorst, T., Andreadis, K. M., and Allen, G. H. (2024). Global cloud biases in optical satellite remote sensing of rivers. Geophysical Research Letters, 51(16):e2024GL110085.
Li, J., Wu, Z., Hu, Z., Jian, C., Luo, S., Mou, L., Zhu, X. X., and Molinier, M. (2021). A lightweight deep learning-based cloud detection method for sentinel-2a imagery fusing multiscale spectral and spatial features. IEEE Transactions on Geoscience and Remote Sensing, 60:1–19.
Liu, C.-C., Zhang, Y.-C., Chen, P.-Y., Lai, C.-C., Chen, Y.-H., Cheng, J.-H., and Ko, M.-H. (2019). Clouds classification from sentinel-2 imagery with deep residual learning and semantic image segmentation. Remote Sensing, 11(2):119.
Liu, Y., Zhang, Y., Wang, Y., Hou, F., Yuan, J., Tian, J., Zhang, Y., Shi, Z., Fan, J., and He, Z. (2023). A survey of visual transformers. IEEE Transactions on Neural Networks and Learning Systems.
Luotamo, M., Metsämäki, S., and Klami, A. (2020). Multiscale cloud detection in remote sensing images using a dual convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 59(6):4972–4983.
Misra, G., Cawkwell, F., and Wingler, A. (2020). Status of phenological research using sentinel-2 data: A review. Remote Sensing, 12(17).
Ozdogan, M., Wang, S., Ghose, D., Fraga, E. P., Fernandes, A. M., and Varela, G. (2024). Field-scale rice area and yield mapping in sri lanka with optical remote sensing and limited training data. Available at SSRN 4940849.
Robinson, T. R., Rosser, N., and Walters, R. J. (2019). The spatial and temporal influence of cloud cover on satellite-based emergency mapping of earthquake disasters. Scientific reports, 9(1):12455.
Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer.
Saifullah, S., Dreżewski, R., and Yudhana, A. (2025). Advanced brain tumor segmentation using deeplabv3plus with xception encoder on a multi-class mr image dataset. Multimedia Tools and Applications, pages 1–22.
Santos, M., Paula, M., and Souza, V. (2024). Extração de séries espaço-temporais a partir de um cubo de dados geoespacial em benefício da cafeicultura mineira. In Anais do XV Workshop de Computação Aplicada à Gestão do Meio Ambiente e Recursos Naturais, pages 211–214, Porto Alegre, RS, Brasil. SBC.
Sourget, T., Hasany, S. N., Mériaudeau, F., and Petitjean, C. (2023). Can segformer be a true competitor to u-net for medical image segmentation? In Annual Conference on Medical Image Understanding and Analysis, pages 111–118. Springer.
Uchegbulam, O., Ameloko, A., and Omo-Irabor, O. (2021). Effect of cloud cover on land use land cover dynamics using remotely sensed data of western niger delta, nigeria. Journal of Applied Sciences and Environmental Management, 25(5):799–804.
Ventura, R., Musis, C., Ventura, T., and II, A. C. (2023). Rsdd: um conjunto de dados para modelagem de desmatamentos. In Anais do XIV Workshop de Computação Aplicada à Gestão do Meio Ambiente e Recursos Naturais, pages 159–162, Porto Alegre, RS, Brasil. SBC.
Wang, Z., Zhao, L., Meng, J., Han, Y., Li, X., Jiang, R., Chen, J., and Li, H. (2024). Deep learning-based cloud detection for optical remote sensing images: A survey. Remote Sensing, 16(23):4583.
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., and Luo, P. (2021). Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:12077–12090.
Xu, K., Wang, W., Deng, X., Wang, A., Wu, B., and Jia, Z. (2024). Transga-net: Integration transformer with gradient-aware feature aggregation for accurate cloud detection in remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 21:1–5.
Xu, X., He, W., Xia, Y., Zhang, H., Wu, Y., Jiang, Z., and Hu, T. (2025). Tanet: Thin cloud-aware network for cloud detection in optical remote sensing image. IEEE Transactions on Geoscience and Remote Sensing, 63:1–16.
Published
2025-07-20
How to Cite
DIAS, Jean A. C. et al.
Cloud Segmentation in Multispectral Images from the Sentinel-2 Satellite: A Comparative Study of Deep Learning Approaches. In: WORKSHOP ON COMPUTING APPLIED TO THE MANAGEMENT OF THE ENVIRONMENT AND NATURAL RESOURCES (WCAMA), 16. , 2025, Maceió/AL.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 59-68.
ISSN 2595-6124.
DOI: https://doi.org/10.5753/wcama.2025.7754.
