Image Sequence Based Cyclist Action Recognition Using Multi-Stream 3D Convolution

Zernetsch, Stefan; Schreck, Steven; Kress, Viktor; Doll, Konrad; Sick, Bernhard

doi:10.1109/icpr48806.2021.9413233

Cited by 5 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The network, referred to as I3D (Inflated 3D), outperforms the other architectures on the Kinetics human action detection video dataset [10] and achieves 98.0 % accuracy on the UCF101 Human Action Classes dataset [11]. The I3D architecture is extended to cyclist action in [12] and is used as a feature extractor in this article for cyclist trajectory forecasts. While we chose I3D due to its great performance regarding cyclist action recognition, the feature extractor used in the method is interchangeable.…”

Section: B Related Workmentioning

confidence: 99%

“…In this section, we describe the test site where the dataset used for trajectory forecast is created, and provide a description of the dataset itself. Except for the ground truth data, the dataset is the same as the dataset used in [12]. The dataset was collected at an intersection close to the University of Applied Sciences Aschaffenburg using a wideangle stereo camera system (Fig.…”

Section: Test Site and Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Cyclist Trajectory Forecasts by Incorporation of Multi-View Video Information

Zernetsch¹,

Trupp²,

Kress³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This article presents a novel approach to incorporate visual cues from video-data from a wide-angle stereo camera system mounted at an urban intersection into the forecast of cyclist trajectories. We extract features from image and optical flow (OF) sequences using 3D convolutional neural networks (3D-ConvNet) and combine them with features extracted from the cyclist's past trajectory to forecast future cyclist positions. By the use of additional information, we are able to improve positional accuracy by about 7.5 % for our test dataset and by up to 22 % for specific motion types compared to a method solely based on past trajectories. Furthermore, we compare the use of image sequences to the use of OF sequences as additional information, showing that OF alone leads to significant improvements in positional accuracy. By training and testing our methods using a real-world dataset recorded at a heavily frequented public intersection and evaluating the methods' runtimes, we demonstrate the applicability in real traffic scenarios. Our code and parts of our dataset are made publicly available.

show abstract

Section: B Related Workmentioning

confidence: 99%

Section: Test Site and Datasetmentioning

confidence: 99%

Cyclist Trajectory Forecasts by Incorporation of Multi-View Video Information

Zernetsch¹,

Trupp²,

Kress³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence as a Basis for Automated Driving

Zernetsch,

Kress,

Bieshaar

et al. 2024

Cooperatively Interacting Vehicles

View full text Add to dashboard Cite

The project Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence as a Basis for Automated Driving (DeCoInt$$^2$$ 2 ) focuses on detecting the intentions of vulnerable road users (VRUs) in automated driving using cooperative technologies. Especially in urban areas, VRUs, e.g., pedestrians and cyclists, will continue to play an essential role in mixed traffic. For an accident-free and highly efficient traffic flow with automated vehicles, it is vital to perceive VRUs and their intentions and analyze them similarly to humans when driving and forecasting VRU trajectories. Doing this reliably and robustly with a multimodal sensor system (e.g., cameras, LiDARs, accelerometers, and gyroscopes in mobile devices) in real-time is a big challenge. We follow a holistic, cooperative approach to recognize humans’ movements and forecast their trajectories. Heterogeneous open sets of agents, i.e., collaboratively interacting vehicles, infrastructure, and VRUs equipped with mobile devices, exchange information to determine individual models of their surrounding environment, allowing an accurate and reliable forecast of VRU basic movements and trajectories. The collective intelligence of cooperating agents resolves occlusions, implausibilities, and inconsistencies. We developed new methods by considering and combining novel signal processing and modeling techniques with machine learning-based pattern recognition approaches. The cooperation between agents happens on several levels: the VRU perception level, the level of recognized trajectories, or the level of already detected intentions.

show abstract