Open Set Semantic Segmentation of Remote Sensing Images
Resumo
The development of computational vision approaches that exploit satellite imagery is relatively recent, mainly due to the limited availability of this type of image. In the area of remote sensing, applications that employ computational vision techniques are modeled for classification in closed set scenarios. However, the world is not purely closed set, many scenarios present classes that are not previously known by the algorithm, an open set scenario. Thus, the main objective of this paper is the study and development of semantic segmentation techniques considering the open set scenario applied to remote sensing images. Focusing on this problem, this is the first work to study and develop semantic segmentation techniques for open set scenarios applied to remote sensing images. The main contributions of this paper are: 1) a discussion of related works in open set semantic segmentation, showing evidence that these techniques can be adapted for open set remote sensing tasks; 2) the development and evaluation of four novel approaches for open set semantic segmentation. Our methods yielded competitive results when compared to closed set methods for the same dataset
Palavras-chave:
Open Set, Deep Learning, Semantic Segmentation, Remote Sensing
Referências
W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult, "Toward open set recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1757–1772, 2013.
K. Nogueira, M. Dalla Mura, J. Chanussot, W. R. Schwartz, and J. A. dos Santos, "Learning to semantically segment high-resolution remote sensing images," in 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2016, pp. 3566–3571.
A. Bendale and T. E. Boult, "Towards open set deep networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1563–1572.
A. Bendale and T. Boult, "Towards open world recognition," in Pro- the IEEE Conference on Computer Vision and Pattern ceedings of Recognition, 2015.
P. R. Mendes, R. M. de Souza, R. d. O. Werneck, B. V. Stein, D. V. Pazinato, W. R. de Almeida, O. A. Penatti, R. d. S. Torres, and A. Rocha, "Nearest neighbors distance ratio open-set classifier," Machine Learning, vol. 106, no. 3, pp. 359–386, 2017.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural infor- mation processing systems, 2012, pp. 1097–1105.
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in CVPR, 2016, pp. 770–778.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in CVPR, 2017, pp. 4700–4708.
R. C. Gonzalez and P. Wintz, Digital image processing(Book), 1977, no. 13.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. S¨usstrunk, "Slic superpixels compared to state-of-the-art superpixel methods," IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
K. Nogueira, M. Dalla Mura, J. Chanussot, W. R. Schwartz, and J. A. Dos Santos, "Dynamic multicontext segmentation of remote sensing images based on convolutional networks," TGRS, vol. 57, no. 10, pp. 7503–7520, 2019.
R. G. Congalton and K. Green, Assessing the accuracy of remotely sensed data: principles and practices. CRC press, 2008.
J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
C. C. da Silva, K. Nogueira, H. N. Oliveira, and J. A. d. Santos, images," arXiv "Towards open-set semantic segmentation of aerial preprint arXiv:2001.10063, 2020.
H. Oliveira, C. Silva, G. L. Machado, K. Nogueira, and J. A. d. Santos, "Fully convolutional open set segmentation," arXiv preprint arXiv:2006.14673, 2020.
K. Nogueira, M. Dalla Mura, J. Chanussot, W. R. Schwartz, and J. A. dos Santos, "Learning to semantically segment high-resolution remote sensing images," in 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2016, pp. 3566–3571.
A. Bendale and T. E. Boult, "Towards open set deep networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1563–1572.
A. Bendale and T. Boult, "Towards open world recognition," in Pro- the IEEE Conference on Computer Vision and Pattern ceedings of Recognition, 2015.
P. R. Mendes, R. M. de Souza, R. d. O. Werneck, B. V. Stein, D. V. Pazinato, W. R. de Almeida, O. A. Penatti, R. d. S. Torres, and A. Rocha, "Nearest neighbors distance ratio open-set classifier," Machine Learning, vol. 106, no. 3, pp. 359–386, 2017.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural infor- mation processing systems, 2012, pp. 1097–1105.
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in CVPR, 2016, pp. 770–778.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in CVPR, 2017, pp. 4700–4708.
R. C. Gonzalez and P. Wintz, Digital image processing(Book), 1977, no. 13.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. S¨usstrunk, "Slic superpixels compared to state-of-the-art superpixel methods," IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
K. Nogueira, M. Dalla Mura, J. Chanussot, W. R. Schwartz, and J. A. Dos Santos, "Dynamic multicontext segmentation of remote sensing images based on convolutional networks," TGRS, vol. 57, no. 10, pp. 7503–7520, 2019.
R. G. Congalton and K. Green, Assessing the accuracy of remotely sensed data: principles and practices. CRC press, 2008.
J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
C. C. da Silva, K. Nogueira, H. N. Oliveira, and J. A. d. Santos, images," arXiv "Towards open-set semantic segmentation of aerial preprint arXiv:2001.10063, 2020.
H. Oliveira, C. Silva, G. L. Machado, K. Nogueira, and J. A. d. Santos, "Fully convolutional open set segmentation," arXiv preprint arXiv:2006.14673, 2020.
Publicado
07/11/2020
Como Citar
DA SILVA, Caio Cesar Viana; DOS SANTOS, Jefersson Alex.
Open Set Semantic Segmentation of Remote Sensing Images. In: WORKSHOP DE TESES E DISSERTAÇÕES - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 33. , 2020, Evento Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2020
.
p. 63-69.
DOI: https://doi.org/10.5753/sibgrapi.est.2020.12985.
