How Effective Is Super-Resolution to Improve Dense Labelling of Coarse Resolution Imagery?
Resumo
Coarse resolution remote sensing images, such as LANDSAT and MODIS are easily found in public open repositories and, therefore, are widely used in many studies. But their use for automatic creation of thematic maps is very restrict since most of deep-based semantic segmentation (a.k.a dense labelling) approaches are only suitable for subdecimeter data. In this paper, we design a straightforward framework in order to evaluate the effectiveness of deep-based super-resolution in the semantic segmentation of low-resolution remote sensing images. We carried out an extensive set of experiments on three remote sensing datasets with distinct nature/properties. The results show that super-resolution is effective to improve semantic segmentation performance on low-resolution aerial imagery. It not only outperforms unsupervised interpolation but also achieves semantic segmentation results comparable to high-resolution data.
Palavras-chave:
super resolution, semantic segmentation, remote sensing
Referências
V. V. Klemas "Coastal and environmental remote sensing from unmanned aerial vehicles: An overview" Journal of Coastal Research vol. 31 no. 5 pp. 1260-12015.
R. A. Schowengerdt Remote sensing: models and methods for image processing Elsevier 2006.
D. Pouliot R. Latifovic J. Pasher J. Duffe "Landsat super-resolution enhancement using convolution neural networks and sentinel-2 for training" Remote Sensing vol. 10 no. 3 2018.
D. Dai Y. Wang Y. Chen L. van Gool "Is image super-resolution helpful for other vision tasks?" 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) pp. 1-9 2016.
V. Badrinarayanan A. Kendall R. Cipolla "Segnet: A deep convolutional encoder-decoder architecture for image segmentation" IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 39 no. 12 2017.
M. Haris G. Shakhnarovich N. Ukita Task-driven super resolution: Object detection in low-resolution images 2018.
J. Shermeyer A. van Etten The effects of super-resolution on object detection performance in satellite imagery 2018.
C. Dong C. C. Loy K. He X. Tang "Image super-resolution using deep convolutional networks" IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 38 no. 2 pp. 295-2016.
J. Kim J. Kwon Lee K. Mu Lee "Accurate image super-resolution using very deep convolutional networks" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 1646-12016.
B. Lim S. Son H. Kim S. Nah K. M. Lee "Enhanced deep residual networks for single image super-resolution" The IEEE conference on computer vision and pattern recognition (CVPR) workshops vol. 1 no. 2 pp. 4 2017.
M. Haris G. Shakhnarovich N. Ukita "Deep backprojection networks for super-resolution" Conference on Computer Vision and Pattern Recognition (CVPR) 2018.
J. Yu Y. Fan J. Yang N. Xu Z. Wang X. Wang T. Huang Wide activation for efficient and accurate image super-resolution 2018.
C. Lanaras J. Bioucas-Dias S. Galliani E. Baltsavias K. Schindler Super-resolution of sentinel-2 images: Learning a globally applicable deep neural network 2018.
K. Jiang Z. Wang P. Yi J. Jiang J. Xiao Y. Yao "Deep distillation recursive network for remote sensing imagery super-resolution" Remote Sensing vol. 10 no. 11 2018.
J. Long E. Shelhamer T. Darrell "Fully convolutional networks for semantic segmentation" The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015.
O. Ronneberger P. Fischer T. Brox "U-net: Convolutional networks for biomedical image segmentation" in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 Springer International Publishing pp. 234-2015.
J. Sherrah Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery 2016.
N. Audebert B. Le Saux S. Lefèvre "Semantic segmentation of earth observation data using multimodal and multi-scale deep networks" in Computer Vision - ACCV 2016 International Publishing pp. 180-2017.
D. Marmanis K. Schindler J. Wegner S. Galliani M. Datcu U. Stilla "Classification with an edge: Improving semantic image segmentation with boundary detection" ISPRS Journal of Photogrammetry and Remote Sensing vol. pp. 158-2018.
W. Liu D. Anguelov D. Erhan C. Szegedy S. Reed C.-Y. Fu A. C. Berg "Ssd: Single shot multibox detector" in Computer Vision - ECCV 2016 Springer International Publishing pp. 21-37 2016.
K. Simonyan A. Zisserman Very deep convolutional networks for large-scale image recognition 2014.
K. Nogueira M. Dalla Mura J. Chanussot W. R. Schwartz J. A. dos Santos "Dynamic multicontext segmentation of remote sensing images based on convolutional networks" IEEE Transactions on Geoscience and Remote Sensing pp. 1-18 2019.
E. Maggiori Y. Tarabalka G. Charpiat P. Alliez "High-resolution aerial image labeling with convolutional neural networks" IEEE Transactions on Geoscience and Remote Sensing vol. 55 no. 12 pp. 7092-72017.
R. A. Schowengerdt Remote sensing: models and methods for image processing Elsevier 2006.
D. Pouliot R. Latifovic J. Pasher J. Duffe "Landsat super-resolution enhancement using convolution neural networks and sentinel-2 for training" Remote Sensing vol. 10 no. 3 2018.
D. Dai Y. Wang Y. Chen L. van Gool "Is image super-resolution helpful for other vision tasks?" 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) pp. 1-9 2016.
V. Badrinarayanan A. Kendall R. Cipolla "Segnet: A deep convolutional encoder-decoder architecture for image segmentation" IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 39 no. 12 2017.
M. Haris G. Shakhnarovich N. Ukita Task-driven super resolution: Object detection in low-resolution images 2018.
J. Shermeyer A. van Etten The effects of super-resolution on object detection performance in satellite imagery 2018.
C. Dong C. C. Loy K. He X. Tang "Image super-resolution using deep convolutional networks" IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 38 no. 2 pp. 295-2016.
J. Kim J. Kwon Lee K. Mu Lee "Accurate image super-resolution using very deep convolutional networks" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 1646-12016.
B. Lim S. Son H. Kim S. Nah K. M. Lee "Enhanced deep residual networks for single image super-resolution" The IEEE conference on computer vision and pattern recognition (CVPR) workshops vol. 1 no. 2 pp. 4 2017.
M. Haris G. Shakhnarovich N. Ukita "Deep backprojection networks for super-resolution" Conference on Computer Vision and Pattern Recognition (CVPR) 2018.
J. Yu Y. Fan J. Yang N. Xu Z. Wang X. Wang T. Huang Wide activation for efficient and accurate image super-resolution 2018.
C. Lanaras J. Bioucas-Dias S. Galliani E. Baltsavias K. Schindler Super-resolution of sentinel-2 images: Learning a globally applicable deep neural network 2018.
K. Jiang Z. Wang P. Yi J. Jiang J. Xiao Y. Yao "Deep distillation recursive network for remote sensing imagery super-resolution" Remote Sensing vol. 10 no. 11 2018.
J. Long E. Shelhamer T. Darrell "Fully convolutional networks for semantic segmentation" The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015.
O. Ronneberger P. Fischer T. Brox "U-net: Convolutional networks for biomedical image segmentation" in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 Springer International Publishing pp. 234-2015.
J. Sherrah Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery 2016.
N. Audebert B. Le Saux S. Lefèvre "Semantic segmentation of earth observation data using multimodal and multi-scale deep networks" in Computer Vision - ACCV 2016 International Publishing pp. 180-2017.
D. Marmanis K. Schindler J. Wegner S. Galliani M. Datcu U. Stilla "Classification with an edge: Improving semantic image segmentation with boundary detection" ISPRS Journal of Photogrammetry and Remote Sensing vol. pp. 158-2018.
W. Liu D. Anguelov D. Erhan C. Szegedy S. Reed C.-Y. Fu A. C. Berg "Ssd: Single shot multibox detector" in Computer Vision - ECCV 2016 Springer International Publishing pp. 21-37 2016.
K. Simonyan A. Zisserman Very deep convolutional networks for large-scale image recognition 2014.
K. Nogueira M. Dalla Mura J. Chanussot W. R. Schwartz J. A. dos Santos "Dynamic multicontext segmentation of remote sensing images based on convolutional networks" IEEE Transactions on Geoscience and Remote Sensing pp. 1-18 2019.
E. Maggiori Y. Tarabalka G. Charpiat P. Alliez "High-resolution aerial image labeling with convolutional neural networks" IEEE Transactions on Geoscience and Remote Sensing vol. 55 no. 12 pp. 7092-72017.
Publicado
28/10/2019
Como Citar
PEREIRA, Matheus B. ; DOS SANTOS, Jefersson A. .
How Effective Is Super-Resolution to Improve Dense Labelling of Coarse Resolution Imagery?. In: CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 32. , 2019, Rio de Janeiro.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2019
.
DOI: https://doi.org/10.5753/sibgrapi.2019.9802.
