The 6th AI City Challenge

Naphade, Milind; Wang, Shuo; Anastasiu, David C.; Tang, Zheng; Chang, Ming-Ching; Ya, Yao; Zheng, Liang; Rahman, Mohammed Shaiqur; Venkatachalapathy, Archana; Sharma, Anuj; Feng, Qianjin; Ablavsky, Vitaly; Sclaroff, Stan; Chakraborty, Pranamesh; Li, Alice; Li, Shangru; Chellappa, Rama

doi:10.1109/cvprw56347.2022.00378

Cited by 39 publications

(1 citation statement)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The 2023 NVIDIA AI City Challenge [28] has released naturalistic driving dataset [29] consisting of 210 video clips, of around 10-min. length each with a resolution of 1920×1080 pixels.…”

Section: Track3 Datasetmentioning

confidence: 99%

Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition

Akdag,

Zhu,

Bondarev

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Classification and localization of driving actions over time is important for advanced driver-assistance systems and naturalistic driving studies. Temporal localization is challenging because it requires robustness, reliability, and accuracy. In this study, we aim to improve the temporal localization and classification accuracy performance by adapting video action recognition and 2D human-pose estimation networks to one model. Therefore, we design a transformer-based fusion architecture to effectively combine 2D-pose features and spatio-temporal features. The model uses 2D-pose features as the positional embedding of the transformer architecture and spatio-temporal features as the main input to the encoder of the transformer. The proposed solution is generic and independent of the camera numbers and positions, giving frame-based class probabilities as output. Finally, the post-processing step combines information from different camera views to obtain final predictions and eliminate false positives. The model performs well on the A2 test set of the 2023 NVIDIA AI City Challenge for naturalistic driving action recognition, achieving the overlap score of the organizer-defined distracted driver behaviour metric of 0.5079.

show abstract

“…The 2023 NVIDIA AI City Challenge [28] has released naturalistic driving dataset [29] consisting of 210 video clips, of around 10-min. length each with a resolution of 1920×1080 pixels.…”

Section: Track3 Datasetmentioning

confidence: 99%

Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition

Akdag,

Zhu,

Bondarev

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

show abstract

Score-based matching for city-scale multi-target multi-camera vehicle tracking

Yang,

Liu,

Chen

et al. 2024

Multimed Tools Appl

View full text Add to dashboard Cite

DAKRS: Domain Adaptive Knowledge-Based Retrieval System for Natural Language-Based Vehicle Retrieval

Nguyen

Chung

2023

IEEE Access

View full text Add to dashboard Cite

Given Natural Language (NL) text descriptions, NL-based vehicle retrieval aims to extract target vehicles from a multi-view multi-camera traffic video pool. Due to inherent distinctions between textual and visual data, this is a challenging multi-modal retrieval task that requires robust feature extractors (e.g. neural network) to well-align the abstract representations of texts and images in the same domain. However, solutions to the problem have been challenged by the high data complexities of not only the multiview, multi-camera attributes of visual data and the diverse range of textual descriptions but also a lack of high-volume datasets in this relatively new field, alongside a prominently large domain gap between training and test sets. Many existing approaches have developed computationally expensive models to separately extract the subspaces of language and vision before blending into the same shared representation space while only focusing on single-modal information and ignoring much of the multi-modal information to deal with the aforementioned issues. Hence, we propose a Domain Adaptive Knowledge-based Retrieval System (DAKRS) to effectively and efficiently align multi-modal knowledge in a setting of limited labels. Our contributions are threefold: (i) An efficient extension of Contrastive Language-Image Pre-training (CLIP)'s transfer learning into a baseline text-to-image multi-modular vehicle retrieval framework; (ii) A data enhancement module to create pseudo-vehicle tracks from the traffic video pool by leveraging the robustness of baseline retrieval model combine with background subtraction; and (iii) A SSDA (SSDA) scheme to engineer pseudo-labels for adapting model parameters to the target domain distribution. Experimental results are benchmarked on the Cityflow-NL dataset, illustrating our competitiveness against state-ofthe-art performances in terms of effectiveness and efficiency without needing further post-processing or ensembling.

show abstract

The 6th AI City Challenge

Cited by 39 publications

References 36 publications

Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition

Transformer-based Fusion of 2D-pose and Spatio-temporal Embeddings for Distracted Driver Action Recognition

Score-based matching for city-scale multi-target multi-camera vehicle tracking

DAKRS: Domain Adaptive Knowledge-Based Retrieval System for Natural Language-Based Vehicle Retrieval

Contact Info

Product

Resources

About