Semantics reinforcement and fusion learning for multimedia streams

Joshi, Dhiraj; Naphade, Milind; Natsev, Apostol

doi:10.1145/1282280.1282328

Cited by 6 publications

(4 citation statements)

References 27 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous approaches use co-training [3] or ensemble algorithms [14] that train different classifiers on single-medium feature vectors and combine the classifiers through a voting scheme to produce a single classifier with better accuracy. More recent approaches concatenate the single-medium feature vectors into a single cross-media vector, see Magalhães and Rüger [17].…”

Section: Related Workmentioning

confidence: 99%

Web news categorization using a cross-media document graph

Iria

Ciravegna

Magalhães

2009

Proceedings of the ACM International Conference on Image and Video Retrieval

View full text Add to dashboard Cite

In this paper we propose a multimedia categorization framework that is able to exploit information across different parts of a multimedia document (e.g., a Web page, a PDF, a Microsoft Office document). For example, a Web news page is composed by text describing some event (e.g., a car accident) and a picture containing additional information regarding the real extent of the event (e.g., how damaged the car is) or providing evidence corroborating the text part. The framework handles multimedia information by considering not only the document's text and images data but also the layout structure which determines how a given text block is related to a particular image. The novelties and contributions of the proposed framework are: (1) support of heterogeneous types of multimedia documents; (2) a documentgraph representation method; and (3) the computation of crossmedia correlations. Moreover, we applied the framework to the tasks of categorising Web news feed data, and our results show a significant improvement over a single-medium based framework.

show abstract

Section: Related Workmentioning

confidence: 99%

Web news categorization using a cross-media document graph

Iria

Ciravegna

Magalhães

2009

Proceedings of the ACM International Conference on Image and Video Retrieval

View full text Add to dashboard Cite

show abstract

“…Here, concepts refer to very generic semantic categories of objects, events, activities, and people. Most concept detection systems rely on many different classification algorithms and combine them together through boosting or fusion [11,22]. Research in the closely related area of automatic image annotation or tagging has mainly focused on consumer and Web images [3,6,15,19].…”

Section: Related Workmentioning

confidence: 99%

“…In classifier combination literature, confidence-based and naïve fusion are well known. Recent work has also studied semantics reinforcement with systematic fusion learning given an ensemble of classifiers using TRECVID corpus [11]. Confidence based fusion is expected to be good when the classifiers themselves are strong and hence the confidence in them is high.…”

mentioning

confidence: 99%

Inferring generic activities and events from image content and bags of geo-tags

Joshi

Luo

2008

Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval

Self Cite

View full text Add to dashboard Cite

The use of contextual information in building concept detectors for digital media has caught the attention of the multimedia community in the recent years. Generally speaking, any information extracted from image headers or tags, or from large collections of related images and used at classification time, can be considered as contextual. Such information, being discriminative in its own right, when combined with pure contentbased detection systems using pixel information, can improve the overall recognition performance significantly. In this paper, we describe a framework for probabilistically modeling geographical information using a Geographical Information Systems (GIS) database for event and activity recognition in general-purpose consumer images, such as those obtained from Flickr. The proposed framework discriminatively models the statistical saliency of geo-tags in describing an activity or event. Our work leverages the inherent patterns of association between events and their geographical venues. We use descriptions of small local neighborhoods to form bags of geo-tags as our representation. Statistical coherence is observed in such descriptions across a wide range of event classes and across many different users. In order to test our approach, we identify certain classes of activities and events wherein people commonly participate and take pictures. Images and corresponding metadata, for the identified events and activities, are obtained from Flickr. We employ visual detectors obtained from Columbia University (Columbia 374), which perform pure visual event and activity recognition. In our experiments, we present the performance advantage obtained by combining contextual GPS information with pixel-based detection systems.

show abstract

“…analysis, association and retrieval of heterogeneous data is still a non-trivial and expensive process. Theoretical and experimental studies have been being conducted to determine optimal strategies for this purpose [1,2,6]. This paper describes the core algorithm used for the automated association of news content (see § 2), captured from many sources, including TV broadcasts, RSS feeds and user blogs.…”

Section: Introductionmentioning

confidence: 99%