Feature pyramid network with multi-scale prediction fusion for real-time semantic segmentation

Quyen, Toan V.; Kim, Min Young

doi:10.1016/j.neucom.2022.11.062

Cited by 17 publications

(19 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Refs. 24 and 25, after connecting to multiscale resolution images, the image information is fused by a fusion module with an attention mechanism. The experiments all demonstrate the importance of a multiscale resolution feature fusion module with an attention mechanism for image feature fusion.…”

Section: Related Workmentioning

confidence: 99%

“…However, the absence of multiscale image input limited the author to attain only a Dice score of 0.9421. Van Quyen and Kim 54 proposed a dual prediction method to effectively capture both thin and large objects within complex street scenes. Employing a pyramid-like approach, this method fell short in achieving a high score, as the target environment did not involve medical images, resulting in a final score of only 0.9233.…”

Section: Experiments Data and Preprocessingmentioning

confidence: 99%

See 1 more Smart Citation

PS5-Net: a medical image segmentation network with multiscale resolution

Li,

Liu,

et al. 2024

J. Med. Imag.

View full text Add to dashboard Cite

.PurposeIn recent years, the continuous advancement of convolutional neural networks (CNNs) has led to the widespread integration of deep neural networks as a mainstream approach in clinical diagnostic support. Particularly, the utilization of CNN-based medical image segmentation has delivered favorable outcomes for aiding clinical diagnosis. Within this realm, network architectures based on the U-shaped structure and incorporating skip connections, along with their diverse derivatives, have gained extensive utilization across various medical image segmentation tasks. Nonetheless, two primary challenges persist. First, certain organs or tissues present considerable complexity, substantial morphological variations, and size discrepancies, posing significant challenges for achieving highly accurate segmentation. Second, the predominant focus of current deep neural networks on single-resolution feature extraction limits the effective extraction of feature information from complex medical images, thereby contributing to information loss via continuous pooling operations and contextual information interaction constraints within the U-shaped structure.ApproachWe proposed a five-layer pyramid segmentation network (PS5-Net), a multiscale segmentation network with diverse resolutions that is founded on the U-Net architecture. Initially, this network effectively leverages the distinct features of images at varying resolutions across different dimensions, departing from prior single-resolution feature extraction methods to adapt to intricate and variable segmentation scenarios. Subsequently, to comprehensively integrate feature information from diverse resolutions, a kernel selection module is proposed to assign weights to features across different dimensions, enhancing the fusion of feature information from various resolutions. Within the feature extraction network denoted as PS-UNet, we preserve the classical structure of the traditional U-Net while enhancing it through the incorporation of dilated convolutions.ResultsPS5-Net attains a Dice score of 0.9613 for liver segmentation on the CHLISC dataset and 0.8587 on the ISIC2018 dataset for skin lesion segmentation. Comparative analysis with diverse medical image segmentation methodologies in recent years reveals that PS5-Net has achieved the highest scores and substantial advancements.ConclusionsPS5-Net effectively harnesses the rich semantic information available at different resolutions, facilitating a comprehensive and nuanced understanding of the input medical images. By capitalizing on global contextual connections, the network adeptly captures the intricate interplay of features and dependencies across the entire image, resulting in more accurate and robust segmentation outcomes. The experimental validation of PS5-Net underscores its superior performance in medical image segmentation tasks, offering promising prospects for enhancing diagnostic and analytical processes within clinical settings. These results highlight the potential of PS5-Net to significantly c...

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Experiments Data and Preprocessingmentioning

confidence: 99%

PS5-Net: a medical image segmentation network with multiscale resolution

Li,

Liu,

et al. 2024

J. Med. Imag.

View full text Add to dashboard Cite

show abstract

“…Some algorithms determine the location region by the contextual information [ 45 ] of targets [ 46 ]. Lin et al [ 47 ] introduced a multi-scale fusion strategy of a feature pyramid network (FPN) [ 48 ] to extract and fuse features at different scales, and obtained deep semantic information and shallow position information. Chen et al [ 49 ] considered different feature extraction methods based on depth and shallow features to improve the detection effect of small targets.…”

Section: Related Workmentioning

confidence: 99%

“…To deal with that, multi-scale learning is used, as a strategy to effectively integrate two types of feature information and perform better semantic representation. According to [ 43 , 47 ], it is appropriate to use separate groups of features to model distinct factors. One concern is that the shallow feature information required by small target detection can be easily diluted in the extraction process.…”

Section: St-centernetmentioning

confidence: 99%

ST-CenterNet: Small Target Detection Algorithm with Adaptive Data Enhancement

Guo

2023

Entropy

View full text Add to dashboard Cite

General target detection with deep learning has made tremendous strides in the past few years. However, small target detection sometimes is associated with insufficient sample size and difficulty in extracting complete feature information. For safety during autonomous driving, remote signs and pedestrians need to be detected from driving scenes photographed by car cameras. In the early period of a medical lesion, because of the small area of the lesion, target detection is of great significance to detect masses and tumors for accurate diagnosis and treatment. To deal with these problems, we propose a novel deep learning model, named CenterNet for small targets (ST-CenterNet). First of all, due to the lack of visual information on small targets in the dataset, we extracted less discriminative features. To overcome this shortcoming, the proposed selective small target replication algorithm (SSTRA) was used to realize increasing numbers of small targets by selectively oversampling them. In addition, the difficulty of extracting shallow semantic information for small targets results in incomplete target feature information. Consequently, we developed a target adaptation feature extraction module (TAFEM), which was used to conduct bottom-up and top-down bidirectional feature extraction by combining ResNet with the adaptive feature pyramid network (AFPN). The improved new network model, AFPN, was added to solve the problem of the original feature extraction module, which can only extract the last layer of the feature information. The experimental results demonstrate that the proposed method can accurately detect the small-scale image of distributed targets and simultaneously, at the pixel level, classify whether a subject is wearing a safety helmet. Compared with the detection effect of the original algorithm on the safety helmet wearing dataset (SHWD), we achieved mean average precision (mAP) of 89.06% and frames per second (FPS) of 28.96, an improvement of 18.08% mAP over the previous method.

show abstract

“…Several intrinsic difficulties hinder the accurate segmentation of tiny objects, including information loss, noisy feature representation, inadequate samples and sensitive to perturbation [35]. To overcome these limitations, extensive efforts have been made from various perspectives [36], such as data augmentation [37], soft label assignment [38], scalespecific segmentation [39], feature reassembly [40], attentionbased segmentation [41], similarity-aware learning [42], superresolution-based segmentation [43], context-aware modeling [44], focus-aware segmentation [16], etc. The strategies shown here are not used separately, but in most cases, they are combined together to achieve better performance.…”

Section: Related Workmentioning

confidence: 99%

Laplacian salience-gated feature pyramid network for accurate liver vessel segmentation

Gao¹,

Zong²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Liver vessels generated from computed tomography are usually pretty small, which poses big challenges for satisfactory vessel segmentation, including 1) the scarcity of high-quality and large-volume vessel masks, 2) the difficulty in capturing vessel-specific features, and 3) the heavily imbalanced distribution of vessels and liver tissues. To advance, a sophisticated model and an elaborated dataset have been built. The model has a newly conceived Laplacian salience filter, highlighting vessel-like regions and suppressing other liver regions, to shape the vessel-specific feature learning and to balance vessels against others. It is further coupled with a pyramid deep learning architecture to capture various levels of features, hence the enhancement of feature formulation. Experiments show that this model markedly outperforms the state-of-the-art approaches, achieving a relative improvement of Dice score by at least 1.63% compared to the existing best model on available datasets. More promisingly, the averaged Dice score produced by existing models on the newly con- structed dataset is as high as 0.728 ± 0.067, which is at least 19.1% higher than that obtained from the existing best dataset under the same settings. These observations suggest that the proposed Laplacian salience, along with the elaborated dataset, can be helpful for liver vessel seg- mentation. </p>

show abstract

Feature pyramid network with multi-scale prediction fusion for real-time semantic segmentation

Cited by 17 publications

References 19 publications

PS5-Net: a medical image segmentation network with multiscale resolution

PS5-Net: a medical image segmentation network with multiscale resolution

ST-CenterNet: Small Target Detection Algorithm with Adaptive Data Enhancement

Laplacian salience-gated feature pyramid network for accurate liver vessel segmentation

Contact Info

Product

Resources

About