In Defense of Multimodal and Temporal Data for Unsupervised Domain Adaptation

• Felipe Manfio Barbosa USP
• Fernando Santos Osório USP

Abstract

In the presence of a Domain Shift, pretrained models tend to deliver degraded performance. This poses a major challenge in tasks such as visual perception in urban environments for autonomous vehicles, where a high level of accuracy is of utmost importance. In this sense, Unsupervised Domain Adaptation has recently gained momentum due to its practical relevance in real-world applications. It provides means for enhancing the performance of pretrained models in new tasks and application scenarios, without the need for laborious annotation processes. Several promising contributions have been made in recent years, including adversarial, self-training, and contrastive learning approaches. Nonetheless, multimodal and temporal data remain underexplored, despite their clear advantages. In this work, we provide a compilation of the State-of-the-Art in Unsupervised Domain Adaptation for Semantic Segmentation of urban scenes. We present the main techniques, their pros and cons, and provide a fruitful discussion on the advantages of multimodal (Depth) and temporal data in this scenario. Current challenges and future research directions are also presented, so that both new and experienced researchers can benefit from this work.

Keywords: Adaptation models, Accuracy, Correlation, Annotations, Semantic segmentation, Urban areas, Contrastive learning, Robots, Autonomous vehicles, Visual perception, unsupervised domain adaptation, depth, multimodal, temporal, semantic segmentation