A Vehicle Tracking System Using Thermal and Lidar Data

Abstract

Object detection is important for autonomous vehicles. While regular cameras can be easily affected by low light environments or high brightness objects, thermal cameras can still get sharp images in those conditions. In this project, an object detection system is developed with thermal images and LiDAR data to achieve vehicle detection and status estimation in extreme lighting conditions. A convolutional neural network that is trained for this project can detect objects in thermal images, and then a tracking algorithm developed in the project can track the same objects between images from different time frames. LiDAR data can be projected to the thermal image plane after calibrations, and once the bounding boxes of the detected objects have been made by the neural network, the LiDAR points within the bounding boxes can be associated to the objects. The system can use the bounding boxes and their associated LiDAR data to estimate the status of the objects, such as location and velocity.