CAN you Feel it? An Immersive and Tangible Car Simulation Experience

• Carlos Gabriel Pereira UFPE
• David Ohara UFPE
• Amadeo Neto UFPE
• Fatima Nunes USP
• João Marcelo Teixeira UFPE / USP

DOI: https://doi.org/10.5753/svr_estendido.2025.15753

Resumo

We present a Work-in-Progress (WiP) Virtual Reality (VR) system that turns a real vehicle into an input controller for its own 3D twin. Using a commodity On-Board Diagnostics II (OBD-II) Bluetooth adapter attached to a BYD Dolphin Plus, we extract real-time steering wheel angle, longitudinal acceleration, and brake status and forward them to a Meta Quest 3 headset. In the current prototype, a PC acts as a bridge between the OBD-II link and the Head Mounted Display (HMD); the VR application, implemented in Unity, renders a full-scale digital model registered to the real car via spatial anchors, allowing users to physically touch the real vehicle while seeing and interacting with its virtual counterpart. We report feasibility results of Bluetooth protocol inspection for the BYD Dolphin Plus, low-latency telemetry forwarding, and anchor stability indoors. We outline the remaining steps to remove the PC bridge and connect the OBD-II adapter directly to the Quest 3, enabling standalone operation. Our findings indicate that real-vehicle telemetry can drive a collocated VR twin with low end-to-end latency, supporting the broader vision of using the physical car as a tangible interface in training and demonstration scenarios.

Palavras-chave: Virtual Reality, OBD-II, Spatial Anchors, Digital Twin

Referências

Benz, T. M., Riedl, B., and Chuang, L. L. (2019). Projection displays induce less simulator sickness than head-mounted displays in a real vehicle driving simulator. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pages 379–387.

Blissing, B., Bruzelius, F., and Eriksson, O. (2022). The effects on driving behavior when using a head-mounted display in a dynamic driving simulator. ACM Transactions on Applied Perception (TAP), 19(1):1–18.

Borges, L. F. M. R., Miranda, A. S., Ferreira, J. V. L., Lazarini, T. P., Neto, A. D. A., Novais, M. G., Schimidt, M. Q., Andreão, R. V., and Mestria, M. (2025). Virtual reality system for inspection and training: A case study in wheel-loader operations. Journal on Interactive Systems, 16(1):288–301.

CSS Electronics (n.d.). Uds protocol tutorial - unified diagnostic services. [link]. Accessed: 2025-08-01.

ELM Electronics (n.d.). Elm327 datasheet. [link]. Accessed: 2025-08-01.

Gabbard, J. L., Smith, M., Tanous, K., Kim, H., and Jonas, B. (2019). Ar drivesim: An immersive driving simulator for augmented reality head-up display research. Frontiers in Robotics and AI, Volume 6 - 2019.

Himmels, C., Andreev, V., Syed, A. A., Lindner, J., Denk, F., and Riener, A. (2023). Are head-mounted displays really not suitable for driving simulation? a comparison with a screen-based simulator. In 2023 IEEE intelligent vehicles symposium (IV), pages 1–6. IEEE.

Meta (2024). Spatial anchors overview (unity). [link]. Accessed 2025-08-01.

Microsoft (2025). Spatial anchors—mixed reality design guidelines. [link]. Accessed 2025-08-01.

Scargill, T., Chen, J., and Gorlatova, M. (2021). Here to stay: Measuring hologram stability in markerless smartphone augmented reality.

Serrano, S. M., Izquierdo, R., Daza, I. G., Sotelo, M. A., and Llorca, D. F. (2023). Digital twin in virtual reality for human-vehicle interactions in the context of autonomous driving. In 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), pages 590–595. IEEE.

Weiss, E. and Gerdes, J. C. (2023). High speed emulation in a vehicle-in-the-loop driving simulator. IEEE Transactions on Intelligent Vehicles, 8(2):1826–1836.

ZoyncTech (2025). Room scan exporter for meta quest 3. [link]. Accessed: 2025-08-01.