WELDON: Weakly Supervised Learning of Deep Convolutional Neural Networks

Abstract

Audio tagging aims to perform multi-label classification on audio chunks and it is a newly proposed task in the Detection and Classification of Acoustic Scenes and Events 2016 (DCASE 2016) challenge. This task encourages research efforts to better analyze and understand the content of the huge amounts of audio data on the web. The difficulty in audio tagging is that it only has a chunk-level label without a frame-level label. This paper presents a weakly supervised method to not only predict the tags but also indicate the temporal locations of the occurred acoustic events. The attention scheme is found to be effective in identifying the important frames while ignoring the unrelated frames. The proposed framework is a deep convolutional recurrent model with two auxiliary modules: an attention module and a localization module. The proposed algorithm was evaluated on the Task 4 of DCASE 2016 challenge. State-of-the-art performance was achieved on the evaluation set with equal error rate (EER) reduced from 0.13 to 0.11, compared with the convolutional recurrent baseline system.

WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation

“…R-CNN [21], or scene understanding [27,25,43,12]. Since this approach is highly inefficient, there have been extensive attempts for using convolutional layers to share feature computation, for image classification [44,13,70], object detection [22,20,52] or image segmentation [8,42]. However, fully connected layers are beneficial in standard deep architectures, e.g.…”

“…The standard max-pooling MIL approach [44] is obtained with only one element, and both top instance model [39], Learning with Label Proportion [65] and global average pooling [70] can be obtained with more. Drawing from negative evidence [47,12,13] we can incorporate minimum scoring regions to support classification and our spatial pooling function can reduce to the kMax+kMin layer of [13].…”

“…Recent alternatives include Global Average Pooling (GAP) [70], soft max in LSE pooling [58], Learning from Label Proportion (LLP) [65,36], and top max scoring [39]. Negative evidence models [47,12,13] explicitly select regions accounting for the absence of the class. In WILDCAT, we propose to incorporate negative evidence insights, but with a differentiate positive and negative contribution process.…”

“…To optimally perform domain adaptation in this context, it becomes necessary to align informative image regions, e.g. by detecting objects [44,29] parts [68,69,70,35] or context [23,13]. Although some works incorporate more precise annotations during training, e.g.…”

“…We propose a new pooling strategy (right of Figure 2) which generalizes several approaches in the literature, including (top) max pooling [44,39], global average pooling [70] or negative evidence models [47,12,13].…”