Mapping the Unseen: Exploiting Super-Resolution for Semantic Segmentation in Low-Resolution Images

• Matheus B. Pereira UFMG
• Jefersson Alex dos Santos UFMG

DOI: https://doi.org/10.5753/sibgrapi.est.2020.12987

Resumo

High-resolution aerial images are usually not accessible or affordable. On the other hand, low-resolution remote sensing data is easily found in public open repositories. The problem is that the low-resolution representation can compromise pattern recognition algorithms, especially semantic segmentation. In this M.Sc. dissertation1 , we design two frameworks in order to evaluate the effectiveness of super-resolution in the semantic segmentation of low-resolution remote sensing images. We carried out an extensive set of experiments on different remote sensing datasets. The results show that super-resolution is effective to improve semantic segmentation performance on low-resolution aerial imagery, outperforming unsupervised interpolation and achieving semantic segmentation results comparable to highresolution data.

Referências

D. Pouliot, R. Latifovic, J. Pasher, and J. Duffe, "Landsat super- resolution enhancement using convolution neural networks and sentinel-2 for training," Remote Sensing, vol. 10, no. 3, 2018.

D. Dai, Y. Wang, Y. Chen, and L. Van Gool, "Is image super-resolution helpful for other vision tasks?" in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 2016, pp. 1–9.

V. Badrinarayanan, A. Kendall, and R. Cipolla, "Segnet: A deep convolutional encoder-decoder architecture for image segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 12, 2017.

M. Haris, G. Shakhnarovich, and N. Ukita, "Task-driven super resolution: Object detection in low-resolution images," arXiv preprint arXiv:1803.11316, 2018.

——, "Deep backprojection networks for super-resolution," in Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2018.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "Ssd: Single shot multibox detector," in Computer Vision – ECCV 2016. Springer International Publishing, 2016, pp. 21–37.

S. N. Ferdous, M. Mostofa, and N. M. Nasrabadi, "Super resolution- assisted deep aerial vehicle detection ," in Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, T. Pham, Ed., vol. 11006, International Society for Optics and Photonics. SPIE, 2019, pp. 432 – 443.

J. Shermeyer and A. Van Etten, "The effects of super-resolution on object detection performance in satellite imagery," arXiv preprint arXiv:1812.04098, 2018.

C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. P. Aitken, A. Tejani, J. Totz, Z. Wang et al., "Photo-realistic single image super-resolution using a generative adversarial network." in CVPR, vol. 2, no. 3, 2017, p. 4.

J. Kim, J. Kwon Lee, and K. Mu Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1646–1654.

Z. Guo, G. Wu, X. Song, W. Yuan, Q. Chen, H. Zhang, X. Shi, M. Xu, Y. Xu, R. Shibasaki, and X. Shao, "Super-resolution integrated building semantic segmentation for multi-source remote sensing imagery," IEEE Access, vol. 7, pp. 99 381–99 397, 2019.

W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-time single image and video super- resolution using an efficient sub-pixel convolutional neural network," in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.

O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Springer International Publishing, 2015, pp. 234–241.

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.

O. A. Penatti, K. Nogueira, and J. A. Dos Santos, "Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?" in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2015, pp. 44–51.

K. Nogueira, M. Dalla Mura, J. Chanussot, W. R. Schwartz, and J. A. dos Santos, "Dynamic multicontext segmentation of remote sensing images based on convolutional networks," IEEE Transactions on Geoscience and Remote Sensing, pp. 1–18, 2019.

M. B. Pereira and J. A. dos Santos, "How effective is super-resolution to improve dense labelling of coarse resolution imagery?" in 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2019, pp. 202–209.

M. B. Pereira and J. A. d. Santos, "An end-to-end framework for low-resolution remote sensing semantic segmentation," arXiv preprint arXiv:2003.07955, 2020.