YOLOP: You Only Look Once for Panoptic Driving Perception

Abstract

A panoptic driving perception system is an essential part of autonomous driving. A high-precision and real-time perception system can assist the vehicle in making reasonable decisions while driving. We present a panoptic driving perception network (you only look once for panoptic (YOLOP)) to perform traffic object detection, drivable area segmentation, and lane detection simultaneously. It is composed of one encoder for feature extraction and three decoders to handle the specific tasks. Our model performs extremely well on the challenging BDD100K dataset, achieving state-of-the-art on all three tasks in terms of accuracy and speed. Besides, we verify the effectiveness of our multi-task learning model for joint training via ablative studies. To our best knowledge, this is the first work that can process these three visual perception tasks simultaneously in real-time on an embedded device Jetson TX2(23 FPS), and maintain excellent accuracy. To facilitate further research, the source codes and pre-trained models are released at https://github.com/hustvl/YOLOP.

An object detection method for the work of an unmanned sweeper in a noisy environment on an improved YOLO algorithm

“…based on YOLO-v1, the overall performance is still not satisfactory. YOLO-9000 [17], , YOLO-v4 [19], YOLO-v5 [20], YOLOX [21], YOLOP [22], YOLO-v7 [23], YOLO-v5 achieves a good balance between accuracy and real-time performance, and it is widely used in industry, so there are a lot of available interfaces, which are convenient for maintenance and upgrades. Therefore, this paper chooses to improve based on the YOLO-v5 algorithm to realize object detection task for tra c participants of unmanned sweeper.…”

FR-DETR: End-to-End Flowchart Recognition With Precision and Robustness

“…Hu et al [26] jointly learned multiple tasks across different domains with a unified Transformer. Wu et al [21] introduced a multi-task network that can jointly handle object detection, drivable area segmentation, and lane detection in autonomous driving.…”

Improving Worst Case Visual Localization Coverage via Place-Specific Sub-Selection in Multi-Camera Systems

“…This includes adversarial training of domain-invariant features [24], learning weather-specific priors [25], multi-scale feature learning per domain [26], image-level feature alignment for single stage detectors [27], image enhancement before object detection [28], [29] which includes approaches specific to hazy [30]- [32] and low light conditions [2], [33] as well as using multiple differentiable image pre-processing units in sequence [34] or parallel [35]. These research efforts have been further fostered by the availability of annotated datasets for adverse conditions (e.g., BDD100K [36]), leading to robust supervised methods such as YOLOP [37]. Despite training on adverse conditions, this problem still remains challenging.…”

“…We propose to use an off-the-shelf object detection method to extract a large set of potential candidate vehicles in the images observed by the autonomous vehicle. In our experiments, we use the recent vehicle detection network YOLOP [37]. YOLOP is a variant of YOLO that has been trained on BDD100K [36] to perform the tasks of vehicle detection, lane line segmentation and driveable area segmentation.…”

“…We use dense semantic segmentation network, Panoptic-Deeplab-v3 [65], to obtain vehicle-labelled pixels. In order to decouple the effect of perception noise (leading to false positive detections) from localization performance of the proposed method, we combine the semantic segmentation output with an object detector network, YOLO-P [66] to emulate a high-precision vehicle detection system of an AV. Using YOLO-P, we extract bounding boxes for all objects observed in the image that are classified as vehicles.…”