Multiple Future Prediction Leveraging Synthetic Trajectories

Berlincioni, Lorenzo; Becattini, Federico; Seidenari, Lorenzo; Bimbo, Alberto Del

doi:10.1109/icpr48806.2021.9412158

Cited by 10 publications

(9 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Preliminarily, the authors relied on a face alignment tool [62] for RGB images and an open event camera simulator (ESIM) [63] to extract cropped facial data by training a YOLOv3 [64] object detector. Another recent work on facial expression recognition was presented in [53]. In [53], the authors present the NEFER dataset, which consists of paired RGB and event videos containing human faces annotated with face bounding boxes and facial landmarks as well as labeled with the corresponding emotions.…”

Section: Related Workmentioning

confidence: 99%

“…Another recent work on facial expression recognition was presented in [53]. In [53], the authors present the NEFER dataset, which consists of paired RGB and event videos containing human faces annotated with face bounding boxes and facial landmarks as well as labeled with the corresponding emotions. The face detector was obtained using the same approach as in [52], except that it was trained on a YOLOv2 [65] object detection model.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Faces in Event Streams (FES): An Annotated Face Dataset for Event Cameras

Bissarinova,

Rakhimzhanova,

Kenzhebalin

et al. 2024

Sensors

View full text Add to dashboard Cite

The use of event-based cameras in computer vision is a growing research direction. However, despite the existing research on face detection using the event camera, a substantial gap persists in the availability of a large dataset featuring annotations for faces and facial landmarks on event streams, thus hampering the development of applications in this direction. In this work, we address this issue by publishing the first large and varied dataset (Faces in Event Streams) with a duration of 689 min for face and facial landmark detection in direct event-based camera outputs. In addition, this article presents 12 models trained on our dataset to predict bounding box and facial landmark coordinates with an mAP50 score of more than 90%. We also performed a demonstration of real-time detection with an event-based camera using our models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Faces in Event Streams (FES): An Annotated Face Dataset for Event Cameras

Bissarinova,

Rakhimzhanova,

Kenzhebalin

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…There the task is to forecast future positions of moving agents such as cars and pedestrians. Compared to head motion prediction, where predictions are guided by content and user attitude, trajectory forecasting is a more constrained task due to social behavioral rules [3,25,31,36,55,72], inertia of moving agents and environmental constraints [7,9,36,42,61]. Nonetheless, the ability to forecast a multimodal prediction is of fundamental importance for planning secure trajectories for autonomous vehicles.…”

Section: Multiple Trajectory Prediction In Roboticsmentioning

confidence: 99%

“…An extension of such loss, the multimodality Loss, has been introduced by Berlincioni et al [7], where the authors rely on synthetic data to generate multiple ground truth futures and directly optimize the model to output multiple adequate predictions. This approach requires the ability to generate synthetic samples but replaces the exploration step with an explicit supervision signal.…”

Section: Multiple Trajectory Prediction In Roboticsmentioning

confidence: 99%

See 1 more Smart Citation

Deep variational learning for multiple trajectory prediction of 360° head movements

Guimard

Sassatelli

Marchetti

et al. 2022

Proceedings of the 13th ACM Multimedia Systems Conference

Self Cite

View full text Add to dashboard Cite

Prediction of head movements in immersive media is key to design efficient streaming systems able to focus the bandwidth budget on visible areas of the content. Numerous proposals have therefore been made in the recent years to predict 360°images and videos. However, the performance of these models is limited by a main characteristic of the head motion data: its intrinsic uncertainty. In this article, we present an approach to generate multiple plausible futures of head motion in 360°videos, given a common past trajectory. Our method provides likelihood estimates of every predicted trajectory, enabling direct integration in streaming optimization. To the best of our knowledge, this is the first work that considers the problem of multiple head motion prediction for 360°video streaming. We first quantify this uncertainty from the data. We then introduce our discrete variational multiple sequence (DVMS) learning framework, which builds on deep latent variable models. We design a training procedure to obtain a flexible and lightweight stochastic prediction model compatible with sequence-to-sequence recurrent neural architectures. Experimental results on 3 different datasets show that our method DVMS outperforms competitors adapted from the selfdriving domain by up to 37% on prediction horizons up to 5 sec., at lower computational and memory costs. Finally, we design a method to estimate the respective likelihoods of the multiple predicted trajectories, by exploiting the stationarity of the distribution of the prediction error over the latent space. Experimental results on 3 datasets show the quality of these estimates, and how they depend on the video category. CCS CONCEPTS• Human-centered computing → Virtual reality; • Information systems → Multimedia streaming; • Computing methodologies → Neural networks.

show abstract

Explaining autonomous driving with visual attention and end-to-end trainable region proposals

Cultrera

Becattini

Seidenari

et al. 2023

J Ambient Intell Human Comput

View full text Add to dashboard Cite

Autonomous driving is advancing at a fast pace, with driving algorithms becoming more and more accurate and reliable. Despite this, it is of utter importance to develop models that can offer a certain degree of explainability in order to be trusted, understood and accepted by researchers and, especially, society. In this work we present a conditional imitation learning agent based on a visual attention mechanism in order to provide visually explainable decisions by design. We propose different variations of the method, relying on end-to-end trainable regions proposal functions, generating regions of interest to be weighed by an attention module. We show that visual attention can improve driving capabilities and provide at the same time explainable decisions.

show abstract

Multiple Future Prediction Leveraging Synthetic Trajectories

Cited by 10 publications

References 27 publications

Faces in Event Streams (FES): An Annotated Face Dataset for Event Cameras

Faces in Event Streams (FES): An Annotated Face Dataset for Event Cameras

Deep variational learning for multiple trajectory prediction of 360° head movements

Explaining autonomous driving with visual attention and end-to-end trainable region proposals

Contact Info

Product

Resources

About