ML-MG: Multi-label Learning with Missing Labels Using a Mixed Graph

Diverse Image Annotation

“…As such, image annotation is treated as a multi-label learning problem, where tag correlations play a key role. Most common tag correlations involve tag-level smoothness [30,32] (i.e., the prediction scores of two semantically similar tags should be similar in the same image), image-level smoothness [13,30,32,20] (i.e., visually similar images have similar tags), low rank assumption [2] (i.e., the whole tag space is spanned by a lower-dimensional space), and semantic hierarchy [30,25] (i.e. parent tags in a hierarchy are as probable as their children).…”

“…Note that most of these methods only focus on positive tag correlations, while negative correlations have rarely been explored, such as mutual exclusion [7,3] and diversity. The third direction focuses on designing loss functions that encourage certain types of annotation solutions, such as the (weighted) hamming loss [30,34] or the pairwise ranking loss [1]. The fourth direction handles incomplete tags in training, which has been studied in many recent works [30,34,5,29,31,19].…”

“…It not only presents a simple way to understand the image's content, but also provides useful information for other tasks, such as object detection [12] or caption generation [6][33] [14]. Many existing methods for image annotation are designed to produce a complete list of tags that cover all contents of an image, such as ML-MG [30] and FastTag [5]. We argue that such a complete list Figure 1.…”

“…This image with its complete tag list is extracted from the ESP Game [26] dataset. We also show the annotated tags by ML-MG [30] and our method using 3 and 5 tags, respectively. Obviously our tags are more representative and diverse than the tags of ML-MG.…”

“…However, when comparing this strategy to the top-k (in terms of individual tag prediction score) retrieved tags of automated annotation methods, we observe a serious discrepancy. For example, as shown in Figure 1, the top-3 tags of the recent annotation method ML-MG [30] are quite redundant. Similar to many other methods, ML-MG focuses on predicting highly representative individual tags, while ignoring diversity within the retrieved tag list.…”