High-Precision Vision Localization System for Autonomous Guided Vehicles in Dusty Industrial Environments

Overview of the experimental AGV navigation system; the length and width of the AGV are 5 and 1.8 m, respectively. The camera located before the AGV and the LED target should be 3.5~10 m away from the camera. An infrared penetrating filter is mounted on the camera lens. Distributed LED lights are placed on the target; they emit infrared light of 850 nm. The camera will track the LED target through rotations. Thus, the pose of the AGV with full view of the LED target can be computed.

Structure of localization system: A navigation personal computer (PC) was connected with sensors and AGV to run the positioning system. The rotating platform and camera formed a feedback servo tracking system that maintained the LED target in the middle of the camera view. The camera transferred the LED target image to the PC to compute the vision pose. An IMU was installed in the vision localization cabinet to improve vision results. The AGV then sends the odometer data to the PC and receives control instructions.

Flowchart of the localization system: The camera searches the LED target after system initialization and continues to track the target. When the camera obtains a new valid image, including the full target, a vision computation thread is used to handle the image and optimize results. Then, the vision observation results are sent to the Kalman filter process with the IMU and odometer data. The filtering process estimates the final AGV pose for navigation. If there is no new image in the data fusion loops, the odometer and IMU are used to predict the pose.

LED light target (left) and its image captured by the camera (right); the LED lights fixed on the bracket are non-coplanar. There are five ball probe seats on the target that can hold the laser tracker target ball while the camera is set to dark mode and has an infrared penetrating filter. Thus, we can only see the white LED light blobs.

The camera captures the LED light target and recognizes the white blobs in its image. The blobs are sorted by height to match the real LED lights. (u₀, v₀) is the 2D point coordinate of the first blob in the image plane. (x₀, y₀, z₀) is the 3D point coordinate of the first LED light in the world coordinate system defined by the laser tracker. The camera pose in the world frame is the transformation (R_cam, t_cam), which can be computed through the ePnP algorithm.

This is an illustration of a calibration experiment that transfers the camera pose to the rotating center. Two calibrated LED targets are fixed on the ground while the AGV remains still in the middle. We rotate the platform to ensure that the camera views each target and computes the vision results.

This is a depiction of a calibration experiment that transfers the camera rotating center to the AGV control center. Calibrated LED targets are fixed to the ground and the AGV stays at different stations.

Position transformation between the camera rotating center and AGV control center

LED blob image when the stationary camera is watching the target; the LED blob has pixel-level vibrations, which generate vision noise at a position and yaw angle of 2 mm and 0.02°, respectively.

(a) Top view of the camera and LED target in which d_cam is the distance between them and α_cam is their intersection angle; (b) Purple dots denote the vision pose results of the camera rotating center. The dots approximately obey the Gaussian distribution and can be simplified as the red ellipse. Q_x is the semi-major, Q_y is the semi-minor, and β_cam is the rotating angle.

This is the structure of data fusion. The vision result of the camera rotating center p_i = [x_i, y_i, θ_i] is sent to the optimization process, VLPO. The odometer and IMU provide the transformation constraint p̂ = [∅x_i, ∅y_i, Δθ_i] to improve p_i to obtain a more accurate result p_opt. The square root central difference Kalman filter (SR-CDKF; Nørgaard et al., 2000) runs at 10 Hz to estimate the current AGV pose z_k. The SR-CDKF receives the measurement vector y_k (AGV control center pose), control input vector u_k = [v,w], and pose z_k−1 in the last frame. The pose will be sent to navigation block for AGV trajectory tracking control.

(a) These ball probe seats are fixed on the AGV for pose measurement; one is at the AGV control center and the other is at the front. Two seats form a direction that is the same as the AGV yaw angle. (b) depicts the localization experiment. The laser tracker measures the target ball attached on the ball probe seat to give the ground truth.

This is the position of static vision localization. Red asterisks denote the vision results of the AGV pose. Blue circles denote the ground truth poses measured by the laser tracker.