Analysis of Virtual Environments for the Evaluation of LiDAR-Based Object Detection Models: A Study Using the CARLA Simulator - Ohata, Daniel; Cavalcanti, Vinicius; Netto, Roberto Silva (Anais do Simpósio de Realidade Virtual e Aumentada (SVR); 2025: Anais do XXVII Simpósio de Realidade Virtual e Aumentada, 2025)
Virtual environments are increasingly used for testing LiDAR-based object detection models, offering advantages in cost, safety, and reproducibility. This paper presents an evaluation of the CARLA simulator as a virtual testbed to assess cross-domain performance of a detection model trained on the real-world nuScenes dataset. Using matching sensor parameters and urban layouts, we generated synthetic LiDAR point clouds in CARLA and tested the model under various simulated scenarios, including changes in lighting and object density. Results show that the mean Average Precision (mAP) in synthetic data was 5.2 percentage points lower than in the real domain, with performance degradation concentrated in classes affected by occlusion and sparse geometry. Despite this, certain object categories (e.g., vehicles and static infrastructure) maintained comparable detection rates across domains. The originality of this work lies in the controlled analysis of transferability using real-to-sim evaluation under matched configurations. Our findings support the use of simulation for early-stage validation, particularly in tasks requiring high spatial accuracy and reproducibility. This study highlights the practical trade-offs and domain limitations of using virtual environments as a reliable complement to physical testing in LiDAR-based 3D perception pipelines.