FourierNet: Compact Mask Representation for Instance Segmentation Using Differentiable Shape Decoders

Riaz, Hamd ul Moqeet; Benbarka, Nuri; Zell, Andreas

doi:10.1109/icpr48806.2021.9413048

Cited by 11 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2: Overview of the FCSN architecture (left) and differentiable spatial to numerical transform (DSNT) (right) transforms for segmentation. In [26], the authors used DNN to learn Fourier coefficients of sampled points on boundary curves for instance segmentation. However, they regarded the x and y coordinates of boundary points as two sequences of real numbers and applied Fourier transforms independently.…”

Section: B Segmentation Via Shapementioning

confidence: 99%

FCSN: Global Context Aware Segmentation by Learning the Fourier Coefficients of Objects in Medical Images

Jeon

Yang

Feng

2024

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

The encoder-decoder model is a commonly used Deep Neural Network (DNN) model for medical image segmentation. Conventional encoder-decoder models make pixel-wise predictions focusing heavily on local patterns around the pixel. This makes it challenging to give segmentation that preserves the object's shape and topology, which often requires an understanding of the global context. In this work, we propose a Fourier Coefficient Segmentation Network (FCSN)-a novel global context-aware DNN model that segments an object by learning the complex Fourier coefficients of the object's masks. The Fourier coefficients are calculated by integrating over the whole contour. Therefore, for our model to make a precise estimation of the coefficients, the model is motivated to incorporate the global context of the object, leading to a more accurate segmentation of the object's shape. This global context awareness also makes our model robust to unseen local perturbations during inference, such as additive noise or motion blur that are prevalent in medical images. We compare FCSN with other state-of-the-art global context-aware models (UNet++, DeepLabV3+, UNETR) on 5 medical image segmentation tasks, of which 3 are camera imaging datasets (ISIC_2018, RIM_CUP, RIM_DISC) and 2 are medical imaging datasets (PROSTATE, FE-TAL). When FCSN is compared with UNETR, FCSN attains significantly lower Hausdorff scores with 19.14 (6%), 17.42 (6%), 9.16 (14%), 11.18 (22%), and 5.98 (6%) for ISIC_2018, RIM_CUP, RIM_DISC, PROSTATE, and FETAL tasks respectively. Moreover, FCSN is lightweight by discarding the decoder module, which incurs significant computational overhead. FCSN only requires 29.7M parameters which are 75.6M and 9.9M fewer parameters than UNETR and DeepLabV3+, respectively. FCSN attains inference and training speeds of 1.6ms/img and 6.3ms/img, which is 8× and 3× faster than UNet and UNETR. The code for FCSN is made publicly available at https://github.com/nus-mornin-lab/FCSN.

show abstract

Section: B Segmentation Via Shapementioning

confidence: 99%

FCSN: Global Context Aware Segmentation by Learning the Fourier Coefficients of Objects in Medical Images

Jeon

Yang

Feng

2024

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

show abstract

“…For onestage methods, PolarMask [23] converts contour coordinates into polar space and adds a prediction head to generate polar coordinates. FourierNet [24] converts coordinates into Fourier space. While one-stage methods predict contour coordinates directly, two-stages methods produce target contour iteratively.…”

Section: A Instance Segmentationmentioning

confidence: 99%

DCTC: Fast and Accurate Contour-Based Instance Segmentation With DCT Encoding for High-Resolution Remote Sensing Images

Chen,

Liu,

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Instance segmentation in Remote Sensing Images (RSI) poses significant challenges due to the diverse scales of targets, scene complexity, and a high number of targets, making most methods struggle with suboptimal performance and time-consuming computations. To solve those problems, a fast and accurate RSI instance segmentation model (named DCTC) is designed in this paper. DCTC transforms classification problem into regression problem to improve the reference speed. DCTC contains two parallel branches. The contour branch performs iterative regression on contours, extracting precise contour information to improve boundary accuracy. Meanwhile, the DCT branch refines mask predictions and supplements instance context information, which particularly benefits the segmentation of small targets. DCT encoding is employed in the DCT branch to convert the mask representation into DCT format, aligning the outputs of the contour and DCT branches. Three innovative modules are introduced in the DCT branch: the Coarse Result Generation module (CRG), Iteratively Deform and Regress module (IDR), and Contour and DCT Fusion module (CDF). The CRG module generates coarse DCT vectors and contour coordinates, facilitating information exchange between the contour and DCT branches. The IDR module iteratively refines DCT vectors, enabling DCTC to focus more on small targets and instance details. The CDF module merges DCT vectors and contour coordinates, ensuring effective interaction between boundary and context information, thereby enhancing performance. Extensive experiments demonstrate the superiority of DCTC, which achieves 67.7, 36.3, 67.4, 55.1AP on NWPU VHR-10, iSAID, SAR Ship Detection Dataset (SSDD), and High-Resolution SAR Images Dataset (HRSID), and ranks first among state-of-the-art methods while maintaining real-time processing capability. Furthermore, DCTC exhibits strong performance on both optical and SAR images, and the designed DCT branch can be simply plug into any contour-based method to improve the network performance.

show abstract

“…Similar to FourierNet [ 25 ], we devise a novel differentiable shape decoder named the Bézier Differentiable Shape Decoder, abbreviated as BDSD. FourierNet uses Inverse Fast Fourier Transformation as the shape decoder to convert the coefficients of the Fourier series into contour points, whereas we apply the parametric equation of Bézier curves to map the control points to contour points.…”

Section: Proposed Approachmentioning

confidence: 99%

BézierSeg: Parametric Shape Representation for Fast Object Segmentation in Medical Images

Haichou

Deng

et al. 2023

Life

View full text Add to dashboard Cite

Background: Delineating the lesion area is an important task in image-based diagnosis. Pixel-wise classification is a popular approach to segmenting the region of interest. However, at fuzzy boundaries, such methods usually result in glitches, discontinuity or disconnection, inconsistent with the fact that lesions are solid and smooth. Methods: To overcome these problems and to provide an efficient, accurate, robust and concise solution that simplifies the whole segmentation pipeline in AI-assisted applications, we propose the BézierSeg model which outputs Bézier curves encompassing the region of interest. Results: Directly modeling the contour with analytic equations ensures that the segmentation is connected and continuous, and that the boundary is smooth. In addition, it offers sub-pixel accuracy. Without loss of precision, the Bézier contour can be resampled and overlaid with images of any resolution. Moreover, clinicians can conveniently adjust the curve’s control points to refine the result. Conclusions: Our experiments show that the proposed method runs in real time and achieves accuracy competitive with pixel-wise segmentation models.

show abstract

FourierNet: Compact Mask Representation for Instance Segmentation Using Differentiable Shape Decoders

Cited by 11 publications

References 14 publications

FCSN: Global Context Aware Segmentation by Learning the Fourier Coefficients of Objects in Medical Images

FCSN: Global Context Aware Segmentation by Learning the Fourier Coefficients of Objects in Medical Images

DCTC: Fast and Accurate Contour-Based Instance Segmentation With DCT Encoding for High-Resolution Remote Sensing Images

BézierSeg: Parametric Shape Representation for Fast Object Segmentation in Medical Images

Contact Info

Product

Resources

About