SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation

Liu, Zechen; Wu, Zizhang; Tóth, Roland

doi:10.1109/cvprw50498.2020.00506

Cited by 247 publications

(132 citation statements)

References 34 publications

Supporting

Mentioning

132

Contrasting

Order By: Relevance

“…In SMOKE [24], a CNN is trained end-to-end in a single stage by means of a unified loss function. The parameters of 3D bounding boxes are regressed directly without using 2D detection, thanks to a network made of two branches: a classification branch where an object is encoded by its 3D center (projected on the image plane), and a 3D parameter regression branch to construct 3D bounding box around each center.…”

Section: D Object Detection From Monocular Imagesmentioning

confidence: 99%

Lightweight convolutional neural network for real-time 3D object detection in road and railway environments

Mauri

Khemmar

Decoux

et al. 2022

J Real-Time Image Proc

View full text Add to dashboard Cite

For smart mobility, and autonomous vehicles (AV), it is necessary to have a very precise perception of the environment to guarantee reliable decision-making, and to be able to extend the results obtained for the road sector to other areas such as rail. To this end, we introduce a new single-stage monocular real-time 3D object detection convolutional neural network (CNN) based on YOLOv5, dedicated to smart mobility applications for both road and rail environments. To perform the 3D parameter regression, we replace YOLOv5’s anchor boxes with our hybrid anchor boxes. Our method is available in different model sizes such as YOLOv5: small, medium, and large. The new model that we propose is optimized for real-time embedded constraints (lightweight, speed, and accuracy) that takes advantage of the improvement brought by split attention (SA) convolutions called small split attention model (Small-SA). To validate our CNN model, we also introduce a new virtual dataset for both road and rail environments by leveraging the video game Grand Theft Auto V (GTAV). We provide extensive results of our different models on both KITTI and our own GTAV datasets. Through our results, we show that our method is the fastest available 3D object detection with accuracy results close to state-of-the-art methods on the KITTI road dataset. We further demonstrate that the pre-training process on our GTAV virtual dataset improves the accuracy on real datasets such as KITTI, thus allowing our method to obtain an even greater accuracy than state-of-the-art approaches with 16.16% 3D average precision on hard car detection with inference time of 11.1 ms/image on an RTX 3080 GPU.

show abstract

Section: D Object Detection From Monocular Imagesmentioning

confidence: 99%

Lightweight convolutional neural network for real-time 3D object detection in road and railway environments

Mauri

Khemmar

Decoux

et al. 2022

J Real-Time Image Proc

View full text Add to dashboard Cite

show abstract

“…MonoPSR [64] evaluates pedestrians from monocular RGB images, leveraging point clouds at training time to learn local shapes of objects. MonoDIS [65] proposes to disentangle the contribution of each loss component, while SMOKE [66] combines a single keypoint estimate with regressed 3D variables. Kundegorski and Breckon [67] achieve reasonable performances combining infrared imagery and real-time photogrammetry.…”

Section: A Monocular 3d Visionmentioning

confidence: 99%

“…• SMOKE [66] is a single-stage monocular 3D object detection method which is based on projecting 3D points onto the image plane. The authors have shared their quantitative evaluation.…”

Section: ) Other Baselinesmentioning

confidence: 99%

Perceiving Humans: From Monocular 3D Localization to Social Distancing

Bertoni

Kreiss

Alahi

2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Perceiving humans in the context of Intelligent Transportation Systems (ITS) often relies on multiple cameras or expensive LiDAR sensors. In this work, we present a new cost-effective vision-based method that perceives humans' locations in 3D and their body orientation from a single image. We address the challenges related to the ill-posed monocular 3D tasks by proposing a neural network architecture that predicts confidence intervals in contrast to point estimates. Our neural network estimates human 3D body locations and their orientation with a measure of uncertainty. Our proposed solution (i) is privacy-safe, (ii) works with any fixed or moving cameras, and (iii) does not rely on ground plane estimation. We demonstrate the performance of our method with respect to three applications: locating humans in 3D, detecting social interactions, and verifying the compliance of recent safety measures due to the COVID-19 outbreak. We show that it is possible to rethink the concept of "social distancing" as a form of social interaction in contrast to a simple location-based rule. We publicly share the source code towards an open science mission.

show abstract

“…Another approach in this direction is Liu, Wu & T'oth (2020), where the authors extended the CenterNet-based detector by adding depth and orientation estimation branches. They proposed a multi-step disentangling loss to handle different kinds of loss functions in 3D detection tasks.…”

Section: Monocular 3d Object Detection Based On 2d Detectormentioning

confidence: 99%

GAC3D: improving monocular 3D object detection with ground-guide model and adaptive convolution

Bui

Ngo

Pham

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

Monocular 3D object detection has recently become prevalent in autonomous driving and navigation applications due to its cost-efficiency and easy-to-embed to existent vehicles. The most challenging task in monocular vision is to estimate a reliable object’s location cause of the lack of depth information in RGB images. Many methods tackle this ill-posed problem by directly regressing the object’s depth or take the depth map as a supplement input to enhance the model’s results. However, the performance relies heavily on the estimated depth map quality, which is bias to the training data. In this work, we propose depth-adaptive convolution to replace the traditional 2D convolution to deal with the divergent context of the image’s features. This lead to significant improvement in both training convergence and testing accuracy. Second, we propose a ground plane model that utilizes geometric constraints in the pose estimation process. With the new method, named GAC3D, we achieve better detection results. We demonstrate our approach on the KITTI 3D Object Detection benchmark, which outperforms existing monocular methods.

show abstract

SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation

Cited by 247 publications

References 34 publications

Lightweight convolutional neural network for real-time 3D object detection in road and railway environments

Lightweight convolutional neural network for real-time 3D object detection in road and railway environments

Perceiving Humans: From Monocular 3D Localization to Social Distancing

GAC3D: improving monocular 3D object detection with ground-guide model and adaptive convolution

Contact Info

Product

Resources

About