Weaklier Supervised: Semi-automatic Scribble Generation Applied to Semantic Segmentation
Resumo
With many applications regarding semantic segmentation arising, along with the advent of the Deep Semantic Segmentation Networks, the need for large labeled datasets has also largely increased. But labeling thousands of images can be very expensive and time-consuming. Approaches such as weak and semi supervision try do deal with this problem, but the first cannot deal with large datasets and the latter is hard to deal with semantic segmentation. Therefore, in this work we propose a combination of both to create a novel pipeline of weak supervision, with focus in satellite imagery, capable of dealing with large datasets. We propose a pipeline to automatically generate scribbles in images, requiring that the user only label 10% of the images in a given dataset, while a classifier deal with the remaining images. Along with that, we also propose a simple semantic segmentation pipeline, that uses only images with scribbles to train a network. Results show that performance is lower, but similar to a fully supervised pipeline.
Referências
A. Ratner, S. H. Bach, H. Ehrenberg, J. Fries, S. Wu, and C. Ré, "Snorkel: Rapid training data creation with weak supervision," Proceedings of the VLDB Endowment, vol. 11, no. 3, pp. 269-282, 2017.
Z. H. Zhou, "A brief introduction to weakly supervised learning," National Science Review, vol. 5, no. 1, pp. 44-53, 2018.
D. Lin, J. Dai, J. Jia, K. He, and J. Sun, "ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016, pp. 3159-3167.
Y. Hua, D. Marcos, L. Mou, X. X. Zhu, and D. Tuia, "Semantic Segmentation of Remote Sensing Images with Sparse Annotations," IEEE Geoscience and Remote Sensing Letters, vol. 19, 2022.
S. Wang, W. Chen, S. M. Xie, G. Azzari, and D. B. Lobell, "Weakly supervised deep learning for segmentation of remote sensing imagery," Remote Sensing, vol. 12, no. 2, 2020.
J. Wang, C. H. Ding, S. Chen, C. He, and B. Luo, "Semi-supervised remote sensing image semantic segmentation via consistency regularization and average update of pseudo-label," Remote Sensing, vol. 12, no. 21, pp. 1-16, 2020.
X. Ding, Y. Xu, L. Deng, and X. Yang, "Colorization using quaternion algebra with automatic scribble generation," Lecture Notes in Computer Science, vol. 7131 LNCS, pp. 103-114, 2012.
D. Batra, A. Kowdle, D. Parikh, J. Luo, and T. Chen, "Interactively co-segmentating topically related images with intelligent scribble guidance," International Journal of Computer Vision, vol. 93, no. 3, pp. 273-292, 2011.
M. Hamilton, Z. Zhang, B. Hariharan, N. Snavely, and W. T. Freeman, "Unsupervised Semantic Segmentation by Distilling Feature Correspondences," in ICLR 2022, April 2022.
B. A. Jaimes, J. P. K. Ferreira, and C. L. Castro, "Unsupervised Semantic Segmentation of Aerial Images with Application to UAV Localization," IEEE Geoscience and Remote Sensing Letters, vol. 19, 2022.
B. Sirmacek and C. Unsalan, "A probabilistic framework to detect buildings in aerial and satellite images," IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 1, pp. 211-221, 2011.
L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 801-818.
A. Boguszewski, D. Batorski, N. Ziemba-Jankowska, T. Dziedzic, and A. Zambrzycka, "Landcover.ai: Dataset for automatic mapping of buildings, woodlands, water and roads from aerial imagery," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2021, pp. 1102-1110.
