Visual Odometry through Optical Flow Integrated with a Multi-sensor Localization System Applied to Mining Vehicles

• Gabriel Gomes UFMG
• Gilmar Cruz UFMG
• Jacó Domingues Instituto Tecnológico Vale
• Gustavo Pessin Instituto Tecnológico Vale
• Armando Neto UFMG
• Gustavo M. Freitas UFMG

Abstract

This paper presents a method for estimating vehicles’ odometry in mining environments, where GNSS-based localization systems often fail. It relies on visual odometry from a ground-facing camera. Captured images are used to calculate the optical flow, which describes the movement of pixels in a sequence of images. This movement is combined with data from an IMU to determine the camera’s movement (Visual-Inertial Odometry). The optical flow is computed using Gunnar Farneback’s GPU processing method to accurately and efficiently estimate the camera motion. An analysis of four localization implementations is conducted: (1) Wheel Odometry, (2) Visual-Inertial Odometry, (3) fusion of Wheel Odometry and IMU using Extended Kalman Filter (EKF), and (4) fusion using EKF of Visual-Inertial Odometry, Wheel Odometry, and IMU. Results obtained in the simulation illustrate that the proposed multi-sensor localization system can effectively assist in localizing mining vehicles in field tasks.

Keywords: Location awareness, Visualization, Three-dimensional displays, Wheels, Graphics processing units, Cameras, Odometry, Data mining, Optical flow, Visual odometry, Visual Odometry, Optical Flow, Localization