Real-time Instance Segmentation with Discriminative Orientation Maps

Abstract

Although instance segmentation has made considerable advancement over recent years, it's still a challenge to design high accuracy algorithms with real-time performance. In this paper, we propose a real-time instance segmentation framework termed OrienMask. Upon the one-stage object detector YOLOv3, a mask head is added to predict some discriminative orientation maps, which are explicitly defined as spatial offset vectors for both foreground and background pixels. Thanks to the discrimination ability of orientation maps, masks can be recovered without the need for extra foreground segmentation. All instances that match with the same anchor size share a common orientation map. This special sharing strategy reduces the amortized memory utilization for mask predictions but without loss of mask granularity. Given the surviving box predictions after NMS, instance masks can be concurrently constructed from the corresponding orientation maps with low complexity. Owing to the concise design for mask representation and its effective integration with the anchorbased object detector, our method is qualified under realtime conditions while maintaining competitive accuracy. Experiments on COCO benchmark show that OrienMask achieves 34.8 mask AP at the speed of 42.7 fps evaluated with a single RTX 2080 Ti. The code is available at https://github.com/duwt/OrienMask.

Sparse Instance Activation for Real-Time Instance Segmentation

“…We adopt ResNet-50 [16] to reach higher inference speed and its variant ResNet-d [17] to achieve better accuracy but with higher latency and aim for providing a stronger baseline for realtime instance segmentation. Additionally, we adopt a simple random crop and larger weight decay (0.05) to better compare with OrienMask [12] and YOLACT [2]. Table 1 shows that our SparseInst is superior to most realtime methods with better performance and faster inference speed.…”

CompleteInst: An Efficient Instance Segmentation Network for Missed Detection Scene of Autonomous Driving

“…Chen et al proposed BlendMask [25], which includes a blender module, consisting of a bottom module and a top module where the bottom module uses backbone features to predict a set of bases containing low-level semantic information and the top module predicts attention at the instance-level that contains high-level instance information. Du et al proposed Orient Mask [26], adding an additional OrienHead base on YOLOv3 [27] to predict orientation maps, and combining the obtained bounding boxes and orientation maps for constructing masks. In general, the mask generation method of the single-stage method corresponds to one mask for one position, which is also the reason for the missed detection problem.…”

Monitoring social distancing with single image depth estimation

“…State-of-the-art solutions [30], [54], [55], [56], [57] employ complex architectures, running at few FPS on modern GPUs. Nonetheless, compact and fast models have been recently proposed [58], [59], [60], [61], [62]. In this work, we leverage the instance segmentation task using [61] to identify people, assign them a unique label and filter control points occluded by them.…”