Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition

Abstract

Recognizing multiple labels of images is a practical and challenging task, and significant progress has been made by searching semantic-aware regions and modeling label dependency. However, current methods cannot locate the semantic regions accurately due to the lack of part-level supervision or semantic guidance. Moreover, they cannot fully explore the mutual interactions among the semantic regions and do not explicitly model the label co-occurrence.

Attention-Driven Dynamic Graph Convolutional Network for Multi-label Image Recognition

“…Recently, Graph Convolutional Network (GCN) [16] achieves great success in modeling relationship among vertices of a graph. Current state-of-the art methods [2,4] build a complete graph to model the label correlations between [2,3]. (b) shows our motivation that different image has its own graph that can describe the relations of co-occurred categories in the image.…”

Behavior Recognition Based on Category Subspace in Crowded Videos

“…Then, according to formula (2), we construct subspace for each category. Finally, we retrain the model SSGRL [6] by using the generated subspace of per category and obtain classification result. It is obvious in Table 4 that most of categories adopting subspace have better performance than the original model.…”

“…In this connection, it stimulates research for coming up with approaches to capture and explore the label correlations in various ways. Some approaches, based on graph representation learning [6], [10], are proposed to capture the label correlations for multi-label recognition. Besides, a novel approach multi-instance multi-label fast learning (MIMLfast) is proposed in literature [5] to utilize the relations among multiple labels.…”

“…These categories with insufficient samples cannot achieve a good classification effect in the training process in spite of strong feature representation. For instance, experiments [6] on multi-label dataset (VOC 2007 [37]) show that the performance (mean average precision (mAP) is 92%) of categories with fewer samples (the number of samples is less than the average on training set) is inferior to that (mAP is 95.1%) of categories with more samples (the number of samples is larger than the average on training set). Simultaneously, the dataset WWW [1] is also imbalanced.…”

“…Meanwhile, we design a classifier for categories with fewer samples. The classifier is utilized to optimize the current category, which is different from the globally optimized classifiers [3], [5], [6] utilizing the relationship between categories. Specifically, the classifier can weaken the main category and reduce the dependence of the main category on the number of samples on the one hand, and enhance the relationship between the main category and other categories for the indirect classification of the main category on the other hand.…”