Multitask Network for Joint Object Detection, Semantic Segmentation and Human Pose Estimation in Vehicle Occupancy Monitoring

Nikolas, Ebert,; Mangat, Patrick; Wasenmüller, Oliver

doi:10.1109/iv51971.2022.9827088

Cited by 5 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we perform comprehensive experiments on the benchmarks ImageNet-1K [ 21 ] for image classification, COCO [ 22 ] for object detection and instance segmentation, and ADE20K [ 41 ] for semantic segmentation. Domains such as autonomous driving [ 42 , 43 ] and medical technology [ 44 , 45 ] are some of the most important areas for the application of computer vision tasks. For this reason we also investigate the effectiveness of our network in these domains using the two datasets BDD100K [ 46 ] and AGAR [ 45 ].…”

Section: Discussionmentioning

confidence: 99%

PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention

Nikolas

Stricker²,

Wasenmüller

2023

Sensors

Self Cite

View full text Add to dashboard Cite

Recently, transformer architectures have shown superior performance compared to their CNN counterparts in many computer vision tasks. The self-attention mechanism enables transformer networks to connect visual dependencies over short as well as long distances, thus generating a large, sometimes even a global receptive field. In this paper, we propose our Parallel Local-Global Vision Transformer (PLG-ViT), a general backbone model that fuses local window self-attention with global self-attention. By merging these local and global features, short- and long-range spatial interactions can be effectively and efficiently represented without the need for costly computational operations such as shifted windows. In a comprehensive evaluation, we demonstrate that our PLG-ViT outperforms CNN-based as well as state-of-the-art transformer-based architectures in image classification and in complex downstream tasks such as object detection, instance segmentation, and semantic segmentation. In particular, our PLG-ViT models outperformed similarly sized networks like ConvNeXt and Swin Transformer, achieving Top-1 accuracy values of 83.4%, 84.0%, and 84.5% on ImageNet-1K with 27M, 52M, and 91M parameters, respectively.

show abstract

Section: Discussionmentioning

confidence: 99%

PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention

Nikolas

Stricker²,

Wasenmüller

2023

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The work of [45] argued that the use of multi-task networks with a shared backbone and branches for segmentation, detection and pose estimation is advantageous in the in-car monitoring task. DaCruz et al [47] addressed the issue of domain adaptation from synthetic to real data.…”

Section: A Ticam Benchmarkmentioning

confidence: 99%

“…Summary of reported experimental results on VIZTA TICaM dataset for in-car cabin monitoring. †[45] uses a custom training/validation split on TICaM.…”

mentioning

confidence: 99%

Time-of-Flight Depth Sensing for Automotive Safety and Smart Building Applications: The VIZTA Project

Rambach,

Mirbach,

Anisimov

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Time-of-Flight (ToF) can be an advantageous sensing modality for several indoor applications, used alone or in combination with other sensors such as RGB cameras. As part of the research project VIZTA (Vision, Identification, with Z-sensing Technologies and key Applications), we developed methods using Machine Learning algorithms with ToF depth measurements as inputs to address two key areas of applications, in-car cabin monitoring (person detection and segmentation) and smart building monitoring ( person counting and anomaly detection). In this article, we discuss the entire research approach followed in VIZTA, from setting up the experimental environments for collecting data and creating the VIZTA public datasets, to developing Deep Learning algorithms tailored to ToF data, used either in 2D depth map or 3D point cloud format. We discuss the advantages and challenges of using ToF-data, as well as the lessons learned during the evaluation and benchmarking of our methods.

show abstract