Exploring multimodal implicit behavior learning for vehicle navigation in simulated cities

• Eric Aislan Antonelo UFSC
• Gustavo Claudio Karl Couto UFSC
• Christian Möller Åbo Akademi University

DOI: https://doi.org/10.5753/eniac.2025.14289

Resumo

Standard Behavior Cloning (BC) fails to learn multimodal driving decisions, where multiple valid actions exist for the same scenario. We explore Implicit Behavioral Cloning (IBC) with Energy-Based Models (EBMs) to better capture this multimodality. We propose Data-Augmented IBC (DA-IBC), which improves learning by perturbing expert actions to form the counterexamples of IBC training and using better initialization for derivative-free inference. Experiments in the CARLA simulator with Bird’s-Eye View inputs demonstrate that DA-IBC outperforms standard IBC in urban driving tasks designed to evaluate multimodal behavior learning in a test environment. The learned energy landscapes are able to represent multimodal action distributions, which BC fails to achieve.

Referências

Antonelo, E. A., Couto, G. C. K., Möller, C., and Fernandes, P. H. (2024). Investigating behavior cloning from few demonstrations for autonomous driving based on bird’s-eye view in simulated cities. In Brazilian Conference on Intelligent Systems, pages 155–168. Springer.

Balesni, M., Tampuu, A., and Matiisen, T. (2023). Controlling steering with energy-based models. arXiv preprint arXiv:2301.12264.

Bansal, M., Krizhevsky, A., and Ogale, A. (2018). Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. arXiv preprint arXiv:1812.03079.

Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016). End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316.

Chen, D., Zhou, B., Koltun, V., and Krähenbühl, P. (2020). Learning by cheating. In Conference on robot learning, pages 66–75. PMLR.

Codevilla, F., Müller, M., Dosovitskiy, A., López, A. M., and Koltun, V. (2018). End-to-end driving via conditional imitation learning. IEEE Int. Conf. on Robotics and Automation (ICRA), pages 1–9.

Codevilla, F., Santana, E., Lopez, A., and Gaidon, A. (2019). Exploring the limitations of behavior cloning for autonomous driving. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9328–9337.

Couto, G. C. K. and Antonelo, E. A. (2024). Hierarchical generative adversarial imitation learning with mid-level input generation for autonomous driving on urban environments. IEEE Transactions on Intelligent Vehicles, pages 1–14.

Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017). CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, pages 1–16.

Du, Y. and Mordatch, I. (2019). Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems, 32.

Florence, P., Lynch, C., Zeng, A., Ramirez, O. A., Wahid, A., Downs, L., Wong, A., Lee, J., Mordatch, I., and Tompson, J. (2022). Implicit behavioral cloning. In Conference on Robot Learning, pages 158–168. PMLR.

Oord, A. v. d., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.

Pang, B., Zhao, T., Xie, X., and Wu, Y. N. (2021). Trajectory prediction with latent belief energy-based model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11814–11824.

Petrazzini, I. G. and Antonelo, E. A. (2021). Proximal policy optimization with continuous bounded action space via the beta distribution. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1–8. IEEE.

Pomerleau, D. A. (1991). Efficient training of artificial neural networks for autonomous navigation. Neural Computation, 3(1):88–97.

Rhinehart, N., McAllister, R., and Levine, S. (2018). Deep imitative models for flexible inference, planning, and control. arXiv preprint arXiv:1810.06544.

Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.

Singh, S., Tu, S., and Sindhwani, V. (2023). Revisiting energy based models as policies: Ranking noise contrastive estimation and interpolating energy models. arXiv preprint arXiv:2309.05803.

Xu, H., Gao, Y., Yu, F., and Darrell, T. (2017). End-to-end learning of driving models from large-scale video datasets. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2174–2182.