Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation

Li, Feng; Zhang, Hao; xu, Huaizhe; Liu, Shilong; Zhang, Lei; Ni, Lionel M.; Shum, Heung-Yeung

doi:10.48550/arxiv.2206.02777

Cited by 27 publications

(34 citation statements)

References 23 publications

(72 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recurring theme in all three modes of ML is the high complexity of their models. This could be caused by a high-dimensional input size like classifying a high-resolution image database [72], or a complex problem like image segmentation [73]. A commonlyused -but known to be inaccurate [74,75] -measure of complexity is the parametercount of an ML model.…”

Section: Exponential Growth Of Practical Machine Learning Modelsmentioning

confidence: 99%

Quantum Machine Learning: from physics to software engineering

Melnikov¹,

Kordzanganeh²,

Alodjants³

et al. 2023

Preprint

View full text Add to dashboard Cite

Quantum machine learning (QML) is a new, rapidly growing, and fascinating area of research where quantum information science and quantum technologies meet novel machine learning and artificial intelligent facilities. A comprehensive analysis of the main directions of current QML methods and approaches is performed in this review. The aim of our work is twofold. First, we show how classical machine learning approach can help improve the facilities of quantum computers and simulators available today. It is most important due to the modern noisy intermediate-scale quantum (NISQ) era of quantum technologies. In particular, the classical machine learning approach allows optimizing quantum hardware for achieving desired quantum states by implementing quantum devices. Second, we discuss how quantum algorithms and quantum computers may be useful for solving keystone classical machine learning tasks. Currently, quantum-inspired algorithms, which use a quantum approach to classical information processing, represent a powerful tool in software engineering for improving classical computation capacities. In this work, we discuss various quantum neural network capabilities that can be implemented in quantum-classical training algorithms for variational circuits. It is expected that quantum computers will be involved in routine machine learning procedures. In this sense, we are showing how it is essential to elucidate the speedup problem for random walks on arbitrary graphs, which are used in both classical and quantum algorithms. Quantum technologies enhanced by machine learning in fundamental and applied quantum physics, as well as quantum tomography and photonic quantum computing, are also covered.

show abstract

Section: Exponential Growth Of Practical Machine Learning Modelsmentioning

confidence: 99%

Quantum Machine Learning: from physics to software engineering

Melnikov¹,

Kordzanganeh²,

Alodjants³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…With multiple stages involved, learning is often not end-toend. Recent work has proposed end-to-end approaches with Transformer based architectures [16,17,33,35,58,67,68], for which the model directly predicts segmentation masks and optimizes based on a bipartite graph matching loss. Nevertheless, they still require customized architectures (e.g., per instance mask generation, and mask fusion module).…”

Section: Related Workmentioning

confidence: 99%

“…merging multiple predictions [14,30,34,40,65,69]. Recently, end-to-end methods [16,17,33,35,58,67,68] have been proposed, based on a differentiable bipartite graph matching [7]; this effectively converts a one-to-many mapping into a one-to-one mapping based on the identified matching. However, such methods still require customized architectures and specialized loss functions with built-in inductive bias for the panoptic segmentation task.…”

Section: Introductionmentioning

confidence: 99%

A Generalist Framework for Panoptic Segmentation of Images and Videos

Chen¹,

Li²,

Saxena³

et al. 2022

Preprint

View full text Add to dashboard Cite

Panoptic segmentation assigns semantic and instance ID labels to every pixel of an image. As permutations of instance IDs are also valid solutions, the task requires learning of high-dimensional one-to-many mapping. As a result, state-of-the-art approaches use customized architectures and task-specific loss functions. We formulate panoptic segmentation as a discrete data generation problem, without relying on inductive bias of the task. A diffusion model based on analog bits [12] is used to model panoptic masks, with a simple, generic architecture and loss function. By simply adding past predictions as a conditioning signal, our method is capable of modeling video (in a streaming setting) and thereby learns to track object instances automatically. With extensive experiments, we demonstrate that our generalist approach can perform competitively to stateof-the-art specialist methods in similar settings.

show abstract

“…Transformer-based networks were successfully applied in various computer vision tasks and held impressive results. Mask DINO [11] extends DINO [12] by adding a new branch to perform mask prediction for panoptic, instance and semantic segmentation. Content query embeddings from DINO [12] are used to perform mask classification for all segmentation tasks.…”

Section: Semantic Instance Segmentationmentioning

confidence: 99%

Speeding Up Semantic Instance Segmentation by Using Motion Information

2022

View full text Add to dashboard Cite

Environment perception and understanding represent critical aspects in most computer vision systems and/or applications. State-of-the-art techniques to solve this vision task (e.g., semantic instance segmentation) require either dedicated hardware resources to run or a longer execution time. Generally, the main efforts were to improve the accuracy of these methods rather than make them faster. This paper presents a novel solution to speed up the semantic instance segmentation task. The solution combines two state-of-the-art methods from semantic instance segmentation and optical flow fields. To reduce the inference time, the proposed framework (i) runs the inference on every 5th frame, and (ii) for the remaining four frames, it uses the motion map computed by optical flow to warp the instance segmentation output. Using this strategy, the execution time is strongly reduced while preserving the accuracy at state-of-the-art levels. We evaluate our solution on two datasets using available benchmarks. Then, we conclude on the results obtained, highlighting the accuracy of the solution and the real-time operation capability.

show abstract

Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation

Cited by 27 publications

References 23 publications

Quantum Machine Learning: from physics to software engineering

Quantum Machine Learning: from physics to software engineering

A Generalist Framework for Panoptic Segmentation of Images and Videos

Speeding Up Semantic Instance Segmentation by Using Motion Information

Contact Info

Product

Resources

About