Hierarchical Image Classification using Entailment Cone Embeddings

Dhall, Ankit; Makarova, Anastasia; Ganea, Octavian; Pavllo, Dario; Greeff, Michael; Krause, Andreas

doi:10.1109/cvprw50498.2020.00426

Cited by 32 publications

(21 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CIFAR‐10 dataset [23] contains of 10 classes of 32 × 32 RGB images. The 60,000 images are divided into training and test sets, which have 50,000 and 10,000 images, respectively.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

Hierarchical bilinear convolutional neural network for image classification

Zhang

Tang

Luo

et al. 2021

IET Computer Vision

View full text Add to dashboard Cite

Image classification is one of the mainstream tasks of computer vision. However, the most existing methods use labels of the same granularity level for training. This leads to ignoring the hierarchy that may help to differentiate different visual objects better. Embedding hierarchical information into the convolutional neural networks (CNNs) can effectively regulate the semantic space and thus reduce the ambiguity of prediction. To this end, a multi-task learning framework, named as Hierarchical Bilinear Convolutional Neural Network (HB-CNN), is developed by seamlessly integrating CNNs with multitask learning over the hierarchical visual concept structures. Specifically, the labels with a tree structure are used as the supervision to hierarchically train multiple branch networks. In this way, the model can not only learn additional information (e.g. context information) as the coarse-level category features, but also focus the learned fine-level category features on the object properties. To smoothly pass hierarchical conceptual information and encourage feature reuse, a connectivity pattern is proposed to connect features at different levels. Furthermore, a bilinear module is embedded to generalise various orderless texture feature descriptors so that our model can capture more discriminative features. The proposed method is extensively evaluated on the CIFAR-10, CIFAR-100, and 'Orchid' Plant image sets. The experimental results show the effectiveness and superiority of our method. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…The CIFAR‐10 dataset [23] contains of 10 classes of 32 × 32 RGB images. The 60,000 images are divided into training and test sets, which have 50,000 and 10,000 images, respectively.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

Hierarchical bilinear convolutional neural network for image classification

Zhang

Tang

Luo

et al. 2021

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…In [9], various label relations (e.g., mutual exclusion and subsumption) were encoded as conditional random field to pair-wisely capture the correlation between classes. More recently, Bertinetto et al [12] proposed a hierarchical loss that could be viewed as a weighted version of the plain cross entropy loss, and the weights were defined according to the depth of the label tree. Our LHT model introduces a novel loss that combines the traditional cross entropy loss with the proposed confusion loss.…”

Section: Related Workmentioning

confidence: 99%

“…In hierarchical classification, Bertinetto et al [3] proposed a soft-label-based method to soften the one-hot labels according to the metric defined by the label tree. Dhall et al [12] employed entailment cones to learn order preserving embedding, which can be embedded in both Euclidean and hyperbolic geometry enforces transitivity among class hierarchies. Although label embedding is a potential strategy to capture the correlation across class hierarchies, in this paper, we attempt to develop a unified deep classification framework for hierarchical classification.…”

Section: Related Workmentioning

confidence: 99%

Label Hierarchy Transition: Modeling Class Hierarchies to Enhance Deep Classifiers

Wang¹,

Cai²,

Xiao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Hierarchical classification aims to sort the object into a hierarchy of categories. For example, a bird can be categorized according to a three-level hierarchy of order, family, and species. Existing methods commonly address hierarchical classification by decoupling it into several multi-class classification tasks. However, such a multi-task learning strategy fails to fully exploit the correlation among various categories across different hierarchies. In this paper, we propose Label Hierarchy Transition, a unified probabilistic framework based on deep learning, to address hierarchical classification. Specifically, we explicitly learn the label hierarchy transition matrices, whose column vectors represent the conditional label distributions of classes between two adjacent hierarchies and could be capable of encoding the correlation embedded in class hierarchies. We further propose a confusion loss, which encourages the classification network to learn the correlation across different label hierarchies during training. The proposed framework can be adapted to any existing deep network with only minor modifications. We experiment with three public benchmark datasets with various class hierarchies, and the results demonstrate the superiority of our approach beyond the prior arts. Source code will be made publicly available.

show abstract

“…The success of our proposed framework is affected by two key observations in most existing long-tailed classification works. First, incorporating the hierarchy in the model would help improve generalization on classes especially for those minority classes, by leveraging shared features among hierarchically-related classes [7]. Second, the model learning from the original distribution obtains a better representation, i.e., convolutional layers and the model learning from a re-balanced distribution shows a fairer and more unbiased classification for all categories [40,22].…”

Section: Co-occurrencementioning

confidence: 99%

“…Also, Given a sample from unknown classes, a trained model may have no ability to handle the challenge, e.g., giving a hypertension fundus to a DR classification model. Such approaches only assume mutually exclusive, unstructured labels [7].…”

Section: Pre-training With Hierarchical Informationmentioning

confidence: 99%

Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition

Ju¹,

Wang²,

Yu³

et al. 2021

Preprint

View full text Add to dashboard Cite

In the real world, medical datasets often exhibit a longtailed data distribution (i.e., a few classes occupy most of the data, while most classes have rarely few samples), which results in a challenging imbalance learning scenario. For example, there are estimated more than 40 different kinds of retinal diseases with variable morbidity, however with more than 30+ conditions are very rare from the global patient cohorts, which results in a typical long-tailed learning problem for deep learning-based screening models. Moreover, there may exist more than one kind of disease on the retina, which results in a multi-label scenario and bring label co-occurrence issue for re-sampling strategy. In this work, we propose a novel framework that leverages the prior knowledge in retinal diseases for training a more robust representation of the model under a hierarchy-sensible constraint. Then, an instance-wise class-balanced sampling strategy and hybrid knowledge distillation manner are firstly introduced to learn from the long-tailed multi-label distribution. Our experiments training on the retinal dataset of more than one million samples demonstrate the superiority of our proposed methods which outperform all competitors and significantly improve the recognition accuracy of most diseases especially those rare diseases.

show abstract

Hierarchical Image Classification using Entailment Cone Embeddings

Cited by 32 publications

References 9 publications

Hierarchical bilinear convolutional neural network for image classification

Hierarchical bilinear convolutional neural network for image classification

Label Hierarchy Transition: Modeling Class Hierarchies to Enhance Deep Classifiers

Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition

Contact Info

Product

Resources

About