MIFNet: A lightweight multiscale information fusion network

Abstract

Semantic segmentation technique plays a crucial role in Internet of Things applications, such as industrial robotics and self‐driving. Recently deep learning approaches have boosted semantic segmentation accuracy greatly. However, their comprehensive performance in terms of accuracy and efficiency is still far from satisfactory. We observe that (1) accuracy‐oriented methods rely on numerous convolution layers and sophisticated architectures, which result in heavy computational complexity and usually take a long time for inference; (2) efficiency‐oriented methods fail to capture the multiscale context information for discriminative representations during the feature fusion process, thus leading to suboptimal performance. Previous semantic segmentation approaches fail to address these two challenges simultaneously. To tackle the dilemma of precise segmentation and efficient inference, we propose a novel lightweight Multiscale Information Fusion Network (MIFNet). Specifically, the proposed MIFNet mainly consists of two core components, that is, Pyramid Refinement Connection Module (PRCM) and Lightweight Information Fusion Module (LIFM). The PRCM exploits skip learning to establish dependency between different stages. Meanwhile, the pyramid attention mechanism (PAM) in PRCM, which adjusts the weight of hybrid pyramid attention vector to refine spatial features of low‐level, is developed to alleviate the semantic gap. Moreover, the LIFM is designed to detect objects at multiple scales from the global‐local perspective. In LIFM, the proposed multiscale dense concatenation (MDC) adopts various dilated convolution to extract multiscale local context information. Extensive experimental results on benchmarks data sets demonstrate the significantly better performance of the proposed MIFNet compared with most existing state‐of‐the‐art methods.

A Survey on Image Semantic Segmentation Using Deep Learning Techniques

“…In recent years, the work related to multi-scale information fusion has emerged one after another. Some works [34,35] constructed a long jump connection branch, which connects shallow features to deep features to reduce the loss of spatial information. Huang et al [36] proposed the feature-aligned pyramid network (FaPN) for dense image prediction.…”

2D Semantic Segmentation: Recent Developments and Future Directions

“…MIFNet (proposed in Ref. [113]) is a lightweight neural network architecture that reduces network parameters and computational complexity while maintaining model accuracy and improving real-time performance on mobile devices. MIFNet adopts multiscale information fusion technology to extract information from feature maps of different scales.…”

A Multi-Watermarking Algorithm for Medical Images Using Inception V3燼nd燚CT

“…In recent years, the research of deep learning has become a hot spot again, and its applications are all over the fields of computer vision, natural language processing, speech recognition, and so on. Based on the research on deep learning, Cheng et al [8] proposed a lightweight multiscale information fusion network (MIFNet), which solved the two problems of accurate segmentation and efficient reasoning and improved the performance of semantic segmentation technology. Among the typical deep learning networks, deep learning models such as convolutional neural networks (CNN) [9], deep residual networks (DRN) [10], generative adversarial networks (GAN) [11], and U-Net [12] have achieved outstanding results in various fields.…”