Towards Light-weight and Real-time Line Segment Detection

Abstract

Previous deep learning-based line segment detection (LSD) suffer from the immense model size and high computational cost for line prediction. This constrains them from real-time inference on computationally restricted environments. In this paper, we propose a real-time and lightweight line segment detector for resource-constrained environments named Mobile LSD (M-LSD). We design an extremely efficient LSD architecture by minimizing the backbone network and removing the typical multi-module process for line prediction in previous methods. To maintain competitive performance with such a light-weight network, we present novel training schemes: Segments of Line segment (SoL) augmentation and geometric learning scheme. SoL augmentation splits a line segment into multiple subparts, which are used to provide auxiliary line data during the training process. Moreover, the geometric learning scheme allows a model to capture additional geometric cues from matching loss, junction and line segmentation, length and degree regression. Compared with TP-LSD-Lite, previously the best real-time LSD method, our model (M-LSDtiny) achieves competitive performance with 2.5% of model size and an increase of 130.5% in inference speed on GPU when evaluated with Wireframe and YorkUrban datasets. Furthermore, our model runs at 56.8 FPS and 48.6 FPS on Android and iPhone mobile devices, respectively. To the best of our knowledge, this is the first real-time deep LSD method available on mobile devices.

Pose Refinement with Joint Optimization of Visual Points and Lines

“…[16] integrates AFMbased reparameterization scheme [15] for line segments, and significantly improves both accuracy and efficiency. Recently, [17]- [20] directly encodes a line segment into several parameters predicted directly from the networks, without heuristics-driven line proposal generation, and further simplify the architecture.…”

“…Wireframe [12] and YorkUrban [41] are typical line segment detection datasets. The following prevalent metrics are used in previous wireframe parser tasks [9], [12], [15]- [17], [20], and we evaluate our model with the structural Average Precision (sAP ) [13], [42]. We conduct the experiments separately with the native Hourglass Network as the backbone and the one substituted with the Hybrid Block, which are abbreviated as VLSE HG and VLSE HG HB respectively.…”

“…Moreover, they also struggle to distinguish high-level local textures and structural elements. With the rapid advance of deep learning, the Convolutional Neural Networks (CNN) based line segment detection models have achieved superior performances over the traditional methods [12]- [20]. Supported by a largescale dataset with manual wireframe annotations [12], we succeed in training a deep model to capture structured and informative line segments (e.g., building contours, stairs), while ignoring line segments associated with texture (e.g., carpet patterns), which aims to lift the subsequent mapping and localization accuracy.…”

LSDNet: Trainable Modification of LSD Algorithm for Real-Time Line Segment Detection

“…The troubling problem of formulating an explicit criterion that matches the human expectation of what exactly constitutes a "salient line segment" can be avoided by manually annotating images and training a CNN, which then learns an implicit algorithm from data samples. This is an approach that yields the best accuracy in line segment detection task today [14]- [20]. Skipping ahead, let us note that such annotation is not a simple task either -existing datasets on line segment detection have numerous and sometimes extreme VOLUME 4, 2016 FIGURE 1: Overview of the proposed approach.…”

“…While object detectors encondings -after many iterations of refinementhave become fast and elegant, we believe that intermediate representations of most line detector used today are still either imprecise, slow or unintuitive. So whether it is the complexity of the interpreter or the sheer weight of the CNN backbone, CNN-based detectors that outperform the traditional ones in accuracy are also computationally harder [20]. Their complexity limits the scope of application of such algorithms in cases where speed, energy consumption, or hardware price are critical.…”