Real-time localization measure and perception detection using multi-sensor fusion for Automated Guided Vehicles

Automated Guided Vehicles (AGVs) need to localize themselves reliably and perceive environment accurately to perform their tasks efficiently. To that end, they rely on noisy sensor measurements that potentially provide erroneous estimates if they are used directly. To prevent this issue, measurements from different kinds of sensors are generally used together. This paper presents a hybrid multi-sensor fusion pipeline configuration that can function with asynchronous measurements from camera, LiDAR, odometry and Inertial Measurement Units (IMUs). The hybrid multi-sensor fusion algorithm consists of two parts that run in parallel, one of them is the combination of camera and LiDAR with a Fully Convolutional Neural Network (FCNx) architecture for performing classification and segmentation, the other one is to deal with the task of detecting objects and tracking states by using Kalman filter to fusion odometry and IMUs sensors. The developed algorithm was tested with an open-source multi-sensor navigation dataset and real-time experiments with an AGV. It was found that using sensor fusion resulted in a smaller deviation from the actual trajectory compared to using only a laser scanner. Furthermore, in each experiment, using sensor fusion decreased the localization error in the periods where the laser was unavailable, although the amount of improvement depended on the duration of unavailability and motion characteristics.