Farneback Optical Flow Application for Automotive Onboard Vibration Monitoring

• Leonardo Pezenatto da Silva Pollux, Part of Accenture http://orcid.org/0009-0005-8543-8873
• Luiz Fernando Martins Pastuch Stellantis N.V. https://orcid.org/0000-0002-9151-0940

DOI: https://doi.org/10.5753/ssv.2024.32624

Resumo

This research explores a vision-based vibration monitoring algorithm that may be used to detect and measure vibrations on a vehicle with an onboard camera. To achieve this, we explored Optical Flow algorithms techniques such as Lucas-Kanade, Horn-Schunck and Farneback to analyze pixel motion between frames and relate it to vibrations. For testing and validation, CARLA simulator was used to acquire the front camera image and IMU data in scenarios where the vehicle drives over road bumps and potholes. The visual vibration estimation was compared with the z-axis acceleration data of an IMU and the results shows that it is possible to use optical flow methods to detect vibration on the vehicle.

Palavras-chave: optical flow, vehicle vibration, camera, computer vision

Referências

Q. Yu, A. Yin, Q. Zhang, and S. Ma, "Optical flow tracking method for vibration identification of out-of-plane vision," Journal of Vibroengineering, Vol. 19, No. 4, pp. 2363–2374, June 2017. DOI: 10.21595/jve.2017.17771

Nie G-Y, Bodda S. S., Sandhu H. K., Han K., Gupta A., "Computer-Vision-Based Vibration Tracking Using a Digital Camera: A Sparse-Optical-Flow-Based Target Tracking Method," Sensors, Vol. 22, No. 18, 2022, pp. 6869. DOI: 10.3390/s22186869

Xiu C., Weng Y., Shi W., "Vision and Vibration Data Fusion-Based Structural Dynamic Displacement Measurement with Test Validation," Sensors, Vol. 23, No. 9, 2023, pp. 4547. DOI: 10.3390/s23094547

P. M. Harikrishnan and V. P. Gopi, "Vehicle Vibration Signal Processing for Road Surface Monitoring," IEEE Sensors Journal, Vol. 17, No. 16, pp. 5192-5197, August 15, 2017. DOI: 10.1109/JSEN.2017.2719865

R. Rojas, "Lucas-Kanade Optical Flow Tutorial," Freie Universität Berlin, 2017. Available: [link]

J. T. J. Chao, P. W. F. Liu, and C. K. K. Lee, "Optical Flow Estimation: A Comprehensive Review," Sensors, Vol. 22, No. 13, 2022, pp. 5017. DOI: 10.3390/s22135017

G. Farnebäck, "Two-frame motion estimation based on polynomial expansion," Computer Vision Laboratory, Linköping University, SE-581 83 Linköping, Sweden. [Online]. Available: [link]

N. A. Nemade and V. V. Gohokar, "Comparative performance analysis of optical flow algorithms for anomaly detection." [Online]. Available: [link]

J. de Boer and M. Kalksma, "Choosing between optical flow algorithms for UAV position change measurement." [Online]. Available: [link]

R. Radhakrishnan, D. Sharma, and V. Murthy, "A review on particle image velocimetry and optical flow methods in riverine environment," in Proceedings of the 1st International Conference on Recent Trends in Engineering & Technology, Pune, India, February 2017.

NVIDIA, "Vehicles," NVIDIA Omniverse PhysX Documentation, v5.1.0. [Online]. Available: [link] [Accessed: 13-Oct-2024].