Multiple object tracking combining camera and radar

As the demand for accurate real-time multiple object tracking methods increases, the idea of developing a device combining multimodal sensors is proposed. This master thesis proposes an implementation of a pedestrian tracking algorithm based on a camera and a radar and explores the benefits and drawbacks of this type of setup. The intent behind this fusion is to combine the advantages of each of the sensors in order to gain accuracy in situations deemed challenging for methods making use of only one of the two sensors. The final solution uses a state-of-the-art 2D pedestrian detector to extract target positions from images taken by the camera and signal processing to extract observations from the radar. A particle filter is used to combine both information streams by primarily combining the azimuth angle as computed from camera detections with probable ranges observed by the radar. Its accuracy and precision are tested through a series of challenging tests. From the results, it is concluded that, while the camera offers accurate detection, angular localisation and appearance recognition it struggles with too large distance variations, which is why, in these situations, the use of a radar can be highly beneficial. Further improvements to the current method are also suggested for future works