Highly Accurate Dichotomous Image Segmentation

Qin, Xuebin; Dai, Hang; Hu, Xiaobin; Fan, Deng-Ping; Shao, Ling; Gool, Luc Van

doi:10.1007/978-3-031-19797-0_3

Cited by 28 publications

(37 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Random and irregular anomalous regions (Fig. 4, P ) are first obtained from the Perlin [30] noise and then multiplied by the object foreground [33,56] (Fig. 4, F ) of the normal sample to obtain the ground truth mask (Fig.…”

Section: Trainingmentioning

confidence: 99%

DiffusionAD: Denoising Diffusion for Anomaly Detection

Zhang¹,

Wang²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Anomaly detection is widely applied due to its remarkable effectiveness and efficiency in meeting the needs of realworld industrial manufacturing. We introduce a new pipeline, DiffusionAD, to anomaly detection. We frame anomaly detection as a "noise-to-norm" paradigm, in which anomalies are identified as inconsistencies between a query image and its flawless approximation. Our pipeline achieves this by restoring the anomalous regions from the noisy corrupted query image while keeping the normal regions unchanged. DiffusionAD includes a denoising sub-network and a segmentation sub-network, which work together to provide intuitive anomaly detection and localization in an end-to-end manner, without the need for complicated post-processing steps. Remarkably, during inference, this framework delivers satisfactory performance with just one diffusion reverse process step, which is tens to hundreds of times faster than general diffusion methods. Extensive evaluations on standard and challenging benchmarks including VisA and DAGM show that DiffusionAD outperforms current state-of-the-art paradigms, demonstrating the effectiveness and generalizability of the proposed pipeline.

show abstract

Section: Trainingmentioning

confidence: 99%

DiffusionAD: Denoising Diffusion for Anomaly Detection

Zhang¹,

Wang²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This work focuses on the intrinsic attributes of Q instead of this redundant information. Hence we remove the redundant information using IS‐Net [QDH * 22], a segmentation network with pretrained weights.…”

Section: Wytiwyr Prototype Systemmentioning

confidence: 99%

WYTIWYR: A User Intent‐Aware Framework with Multi‐modal Inputs for Visualization Retrieval

Xiao

Hou

Cheng

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example‐based chart retrieval methods are built upon non‐decoupled and low‐level visual features that are hard to interpret, while definition‐based ones are constrained to pre‐defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What‐You‐Think‐Is‐What‐You‐Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language‐image pre‐training (CLIP) model to achieve zero‐shot classification as well as multi‐modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.

show abstract

“…Consequently, we compose a new dataset, called HQSeg-44K, which contains 44K extremely fine-grained image mask annotations. HQSeg-44K is constructed by merging six existing image datasets [34,29,26,37,8,45] with highly accurate mask annotations, covering over 1,000 diverse semantic classes. Thanks to the smaller-scale dataset and our minimal integrated architecture, HQ-SAM can be trained in only 4 hours on 8 RTX 3090 GPUs.…”

Section: Hq-sam Predictionmentioning

confidence: 99%

“…We note that the released SA-1B dataset only contains automatically generated mask labels, missing very accurate manual annotation on objects with complex structures. Due to the annotation difficulty, HQSeg-44K leverages a collection of six existing image datasets including DIS [34] (train set), ThinObject-5K [29] (train set), FSS-1000 [26], ECSSD [37], MSRA-10K [8], DUT-OMRON [45] with extremely fine-grained mask labeling, where each of them contains 7.4K mask labels on average. To make HQ-SAM robust and generalizable to new data, HQSeg-44K contains diverse semantic classes of more than 1,000.…”

Section: Training and Inference Of Hq-sammentioning

confidence: 99%

“…Model DIS [34] COIFT [29] HRSOD [50] ThinObject [ Datasets For training we use the compiled HQSeg-44K, described in Section 3.3. For a comprehensive evaluation of the segmentation performance of HQ-SAM, we perform experiments on a wide range of datasets, including four extremely fine-grained segmentation datasets: DIS [34] (validation set), ThinObject-5K [29] (test set), COIFT [29] and HR-SOD [50].…”

Section: Sam Vs Hq-sam On Training and Inferencementioning

confidence: 99%

See 1 more Smart Citation

Mask Transfiner for High-Quality Instance Segmentation

Danelljan²,

Li³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

The recent Segment Anything Model (SAM) represents a big leap in scaling up segmentation models, allowing for powerful zero-shot capabilities and flexible prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction quality falls short in many cases, particularly when dealing with objects that have intricate structures. We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We design a learnable High-Quality Output Token, which is injected into SAM's mask decoder and is responsible for predicting the high-quality mask. Instead of only applying it on mask-decoder features, we first fuse them with early and final ViT features for improved mask details. To train our introduced learnable parameters, we compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is only trained on the introduced detaset of 44k masks, which takes only 4 hours on 8 GPUs. We show the efficacy of HQ-SAM in a suite of 9 diverse segmentation datasets across different downstream tasks, where 7 out of them are evaluated in a zero-shot transfer protocol. Our code and models will be released at https://github.com/SysCV/SAM-HQ.

show abstract

Highly Accurate Dichotomous Image Segmentation

Cited by 28 publications

References 72 publications

DiffusionAD: Denoising Diffusion for Anomaly Detection

DiffusionAD: Denoising Diffusion for Anomaly Detection

WYTIWYR: A User Intent‐Aware Framework with Multi‐modal Inputs for Visualization Retrieval

Mask Transfiner for High-Quality Instance Segmentation

Contact Info

Product

Resources

About