Object Tracking with a Fusion of Event-Based Camera and Frame-Based Camera

Sun, Haixin; Frémont, Vincent

doi:10.1007/978-3-031-16078-3_15

Cited by 1 publication

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For data fusion, the authors [6]- [9] carried out fusion on the feature level from a perspective with ego-motion. Furthermore, [9] performed early fusion, and [10] performed late fusion. Nevertheless, a fusion between event-based and RGB cameras is a developing field [9]; more knowledge about this topic in the area of stationary sensors in ITS is desirable.…”

Section: Event-based Camera Rgb Cameramentioning

confidence: 99%

See 1 more Smart Citation

TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras

Creß,

Zimmer,

Purschke

et al. 2024

IEEE Trans. Intell. Veh.

View full text Add to dashboard Cite

Event-based cameras are predestined for Intelligent Transportation Systems (ITS). They provide very high temporal resolution and dynamic range, which can eliminate motion blur and improve detection performance at night. However, eventbased images lack color and texture compared to images from a conventional RGB camera. Considering that, data fusion between event-based and conventional cameras can combine the strengths of both modalities. For this purpose, extrinsic calibration is necessary. To the best of our knowledge, no targetless calibration between event-based and RGB cameras can handle multiple moving objects, nor does data fusion optimized for the domain of roadside ITS exist. Furthermore, synchronized event-based and RGB camera datasets considering roadside perspective are not yet published. To fill these research gaps, based on our previous work, we extended our targetless calibration approach with clustering methods to handle multiple moving objects. Furthermore, we developed an early fusion, simple late fusion, and a novel spatiotemporal late fusion method. Lastly, we published the TUMTraf Event Dataset, which contains more than 4,111 synchronized event-based and RGB images with 50,496 labeled 2D boxes. During our extensive experiments, we verified the effectiveness of our calibration method with multiple moving objects. Furthermore, compared to a single RGB camera, we increased the detection performance of up to +9% mAP in the day and up to +13% mAP during the challenging night with our presented event-based sensor fusion methods. The TUMTraf Event Dataset is available at https://innovationmobility.com/tumtraf-dataset.

show abstract

Section: Event-based Camera Rgb Cameramentioning

confidence: 99%

“…They also performed an early fusion and detection by combining the RGB and event voxels. Furthermore, a late fusion between event-based and RGB cameras was developed by [10]. The authors used DBSCAN [37] clustering as a detector for the event-based camera and RetinaNet [54] for the RGB camera.…”

Section: Fusionmentioning

confidence: 99%