Self-Supervised Temporal Consistency applied to Domain Adaptation in Semantic Segmentation of Urban Scenes

• Felipe M. Barbosa USP
• Fernando S. Osório USP

Resumo

Deep Learning-based Semantic Segmentation has reached remarkable levels of accuracy throughout the years. Nonetheless, applying pretrained models to new domains—even though related—leads to considerable drops in performance due to domain shift. In addition to that, semantic labels are costly to obtain for new application scenarios, what makes the process of transfer learning even harder. Finally, since the majority of works in semantic segmentation rely on single-frame predictions, they miss a huge opportunity offered by reasoning on the temporal properties of video data. In light of that, this work addresses all previous problems by implementing a domain adaptation strategy based on self-supervised learning of temporal consistency in videos. The results show that our strategy greatly improves model precision and stability on a given unlabeled target domain.

Palavras-chave: semantic segmentation, self-supervised learning, transfer learning, domain adaptation, temporal consistency