Fast Panoptic Segmentation Network

Geus, Daan de; Meletis, Panagiotis; Dubbelman, Gijs

doi:10.1109/lra.2020.2969919

Cited by 38 publications

(27 citation statements)

References 27 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lately, oneshot detectors, such as RetinaNet [17] or FCOS [13], have achieved great progress and even surpassed two-stage detectors on public benchmarks. Motivated by their benefits, FPSNet [10] builds upon RetinaNet [17] and achieves a fast inference speed. DenseBox [11] leverages the idea of reusing discarded dense object proposals by matching them with final object bounding boxes.…”

Section: Related Workmentioning

confidence: 99%

“…Most recent approaches [6,7] focus on achieving high accuracy on public benchmarks by designing novel semantic segmentation heads on top of Mask R-CNN [8] instance segmentation network or by designing clever merging heuristics within the network [9]. However, significant progress has been achieved by fast networks, starting from [10], which builds upon a one-stage detector, to [11], where dense bounding boxes are clustered into instance masks using semantic segmentation and finally, the proposal-free method [12], which predicts instance centers and regresses instance center offsets.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast Panoptic Segmentation with Soft Attention Embeddings

Petrovai

Nedevschi

2022

Sensors

View full text Add to dashboard Cite

Panoptic segmentation provides a rich 2D environment representation by unifying semantic and instance segmentation. Most current state-of-the-art panoptic segmentation methods are built upon two-stage detectors and are not suitable for real-time applications, such as automated driving, due to their high computational complexity. In this work, we introduce a novel, fast and accurate single-stage panoptic segmentation network that employs a shared feature extraction backbone and three network heads for object detection, semantic segmentation, instance-level attention masks. Guided by object detections, our new panoptic segmentation head learns instance specific soft attention masks based on spatial embeddings. The semantic masks for stuff classes and soft instance masks for things classes are pixel-wise coherent and can be easily integrated in a panoptic output. The training and inference pipelines are simplified and no post-processing of the panoptic output is necessary. Benefiting from fast inference speed, the network can be deployed in automated vehicles or robotic applications. We perform extensive experiments on COCO and Cityscapes datasets and obtain competitive results in both accuracy and time. On the Cityscapes dataset we achieve 59.7 panoptic quality with an inference speed of more than 10 FPS on high resolution 1024 × 2048 images.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fast Panoptic Segmentation with Soft Attention Embeddings

Petrovai

Nedevschi

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The ultimate objective for the entire system is to take measurements from agent's sensors, such as cameras, LiDARs, or RaDARs, and estimate a world model, its perception system. As is typical in 3D environment sensing, designing a comprehensive "hand-crafted" model is not feasible; instead, the model is learned from training data, using machine learning, such as semantic segmentation (Meletis and Dubbelman, 2018), part-level panoptic segmentation (de Geus et al, 2020), human pose estimation (Cao et al, 2021), action recognition (Li Z. et al, 2018), and depth and flow (Zou et al, 2018).…”

Section: Ai Based Techniquesmentioning

confidence: 99%

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

Vermesan

John

Pype

et al. 2021

Front. Future Transp.

View full text Add to dashboard Cite

The automotive sector digitalization accelerates the technology convergence of perception, computing processing, connectivity, propulsion, and data fusion for electric connected autonomous and shared (ECAS) vehicles. This brings cutting-edge computing paradigms with embedded cognitive capabilities into vehicle domains and data infrastructure to provide holistic intrinsic and extrinsic intelligence for new mobility applications. Digital technologies are a significant enabler in achieving the sustainability goals of the green transformation of the mobility and transportation sectors. Innovation occurs predominantly in ECAS vehicles’ architecture, operations, intelligent functions, and automotive digital infrastructure. The traditional ownership model is moving toward multimodal and shared mobility services. The ECAS vehicle’s technology allows for the development of virtual automotive functions that run on shared hardware platforms with data unlocking value, and for introducing new, shared computing-based automotive features. Facilitating vehicle automation, vehicle electrification, vehicle-to-everything (V2X) communication is accomplished by the convergence of artificial intelligence (AI), cellular/wireless connectivity, edge computing, the Internet of things (IoT), the Internet of intelligent things (IoIT), digital twins (DTs), virtual/augmented reality (VR/AR) and distributed ledger technologies (DLTs). Vehicles become more intelligent, connected, functioning as edge micro servers on wheels, powered by sensors/actuators, hardware (HW), software (SW) and smart virtual functions that are integrated into the digital infrastructure. Electrification, automation, connectivity, digitalization, decarbonization, decentralization, and standardization are the main drivers that unlock intelligent vehicles' potential for sustainable green mobility applications. ECAS vehicles act as autonomous agents using swarm intelligence to communicate and exchange information, either directly or indirectly, with each other and the infrastructure, accessing independent services such as energy, high-definition maps, routes, infrastructure information, traffic lights, tolls, parking (micropayments), and finding emergent/intelligent solutions. The article gives an overview of the advances in AI technologies and applications to realize intelligent functions and optimize vehicle performance, control, and decision-making for future ECAS vehicles to support the acceleration of deployment in various mobility scenarios. ECAS vehicles, systems, sub-systems, and components are subjected to stringent regulatory frameworks, which set rigorous requirements for autonomous vehicles. An in-depth assessment of existing standards, regulations, and laws, including a thorough gap analysis, is required. Global guidelines must be provided on how to fulfill the requirements. ECAS vehicle technology trustworthiness, including AI-based HW/SW and algorithms, is necessary for developing ECAS systems across the entire automotive ecosystem. The safety and transparency of AI-based technology and the explainability of the purpose, use, benefits, and limitations of AI systems are critical for fulfilling trustworthiness requirements. The article presents ECAS vehicles’ evolution toward domain controller, zonal vehicle, and federated vehicle/edge/cloud-centric based on distributed intelligence in the vehicle and infrastructure level architectures and the role of AI techniques and methods to implement the different autonomous driving and optimization functions for sustainable green mobility.

show abstract

“…And as can be seen in table III, using a smaller images increases inference speed by 2× at the expense of 3 Panoptic Quality points. PQ, SQ and RQ correspond to the Panoptic Quality, Architecture PQ(%) Time(ms) JSISNet [17] 17.6 n/a AUNet [5] 59.0 n/a Panoptic FPN [3] 58.1 n/a Single Network PS [18] 42.9 590 DeeperLab [8] 56.53 308 Panoptic Deeplab [9] 63.0 175 AdapIS [10] 62.0 n/a FPSNet [19] 55.1 114 Real-Time PS [20] 58…”

Section: A Image Size and Inference Timementioning

confidence: 99%

YOLO-based Panoptic Segmentation Network

Diaz-Zapata

Erkent

Laugier

2021

2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC)

View full text Add to dashboard Cite

Autonomous vehicles need information about their surroundings to safely navigate them. For this, the task of Panoptic Segmentation is proposed as a method of fully parsing the scene by assigning each pixel a label and instance id. Given the constraints of autonomous driving, this process needs to be done in a fast manner. In this paper, we propose the first panoptic segmentation network based on the YOLOv3 real-time object detection network by adding a semantic and instance segmentation branches. YOLO-panoptic is able to do real-time inference and achieves a performance similar to the state of the art methods in some metrics.

show abstract

Fast Panoptic Segmentation Network

Cited by 38 publications

References 27 publications

Fast Panoptic Segmentation with Soft Attention Embeddings

Fast Panoptic Segmentation with Soft Attention Embeddings

Automotive Intelligence Embedded in Electric Connected Autonomous and Shared Vehicles Technology for Sustainable Green Mobility

YOLO-based Panoptic Segmentation Network

Contact Info

Product

Resources

About