Hi-Fi: Hierarchical Feature Integration for Skeleton Detection

Abstract

In natural images, the scales (thickness) of object skeletons may dramatically vary among objects and object parts, making object skeleton detection a challenging problem. We present a new convolutional neural network (CNN) architecture by introducing a novel hierarchical feature integration mechanism, named Hi-Fi, to address the skeleton detection problem. The proposed CNNbased approach has a powerful multi-scale feature integration ability that intrinsically captures highlevel semantics from deeper layers as well as lowlevel details from shallower layers. By hierarchically integrating different CNN feature levels with bidirectional guidance, our approach (1) enables mutual refinement across features of different levels, and (2) possesses the strong ability to capture both rich object context and high-resolution details. Experimental results show that our method significantly outperforms the state-of-the-art methods in terms of effectively fusing features from very different scales, as evidenced by a considerable performance improvement on several benchmarks.

Res2Net: A New Multi-Scale Backbone Architecture

“…Multi-scale feature representations of CNNs are of great importance to a number of vision tasks including object detection [43], face analysis [4], [41], edge detection [37], semantic segmentation [6], salient object detection [34], [65], and skeleton detection [67], boosting the model performance of those fields.…”

DeepFlux for Skeletons in the Wild

“…Deep learning-based methods: With the popularization of CNNs, deep learning-based methods [35,34,17,24,51,22] igure 2. The DeepFlux pipeline.…”

“…[24] develop a two-stream network that combines image and segmentation cues to capture complementary information for skeleton localization. In [51], the authors introduce a hierarchical feature integration (Hi-Fi) mechanism. By hierarchically integrating multi-scale features with bidirectional guidance, high-level semantics and low-level details can benefit from each other.…”

“…Though the method we propose in the present paper ben-efits from CNN-based learning, it differs from the methods in [35,34,17,24,51,22] in a fundamental way, due to its different learning objective. Instead of treating object skeleton detection in natural images as a binary classification problem, DeepFlux focuses on learning the context flux of skeletons, and as such includes more informative nonlocal cues, such as the relative position of skeleton points to image points in their vicinity, and thus also, implicitly, the associated object boundaries.…”

Dense Fusion Classmate Network for Land Cover Classification

“…From global to local, semantic segmentation needs rich representations that span levels from low to high for more information aggregation. Some works have focused on the feature fusion, such as [22,19] . The spatial pyramid pooling [8] (SSP) generates a fixed-length representation regardless of image size/scale, which helps to merge multi scale information to get more information for segmentation at a deeper stage of the network hierarchy.…”