AbstractÐThis paper presents a new compact shape representation for retrieving line-patterns from large databases. The basic idea is to exploit both geometric attributes and structural information to construct a shape histogram. We realize this goal by computing the N-nearest neighbor graph for the lines-segments for each pattern. The edges of the neighborhood graphs are used to gate contributions to a two-dimensional pairwise geometric histogram. Shapes are indexed by searching for the line-pattern that maximizes the cross correlation of the normalized histogram bin-contents. We evaluate the new method on a database containing over 2,500 line-patterns each composed of hundreds of lines.

This paper describes the MeMAD project entry to the WMT Multimodal Machine Translation Shared Task.We propose adapting the Transformer neural machine translation (NMT) architecture to a multi-modal setting. In this paper, we also describe the preliminary experiments with textonly translation systems leading us up to this choice.We have the top scoring system for both English-to-German and English-to-French, according to the automatic metrics for flickr18.Our experiments show that the effect of the visual features in our system is small. Our largest gains come from the quality of the underlying text-only NMT system. We find that appropriate use of additional data is effective.

We present a method combining semantic inferencing and visual analysis for finding automatically media (photos and videos) illustrating events. We report on experiments validating our heuristic for mining media sharing platforms and large event directories in order to mutually enrich the descriptions of the content they host. Our overall goal is to design a web-based environment that allows users to explore and select events, to inspect associated media, and to discover meaningful, surprising or entertaining connections between events, media and people participating in events. We present a large dataset composed of semantic descriptions of events, photos and videos interlinked with the larger Linked Open Data cloud and we show the benefits of using semantic web technologies for integrating multimedia metadata.

We present a method to automatically detect and identify events from social media sharing web sites. Our approach is based on the observation that many photos and videos are taken and shared when events occur. We select 9 venues across the globe that demonstrate a significant activity according to the EventMedia dataset and we thoroughly evaluate our approach against an official ground truth obtained directly by scraping the event venues' web sites. The results show our ability to not only detect events with high accuracy but also mine and identify events that have not been published in popular event directories such as Last.fm, Eventful or Upcoming. In addition to the textual identification of events, we show how we can build visual summaries of past events providing viewers with a more compelling feeling of the event's atmosphere.

Multimedia indexing is about developing techniques allowing people to effectively find media. Content-based methods become necessary when dealing with large databases. Current technology allows exploring the emotional space which is known to carry very interesting semantic information. In this paper we state the need for an integrated method which extracts reliable affective information and attaches this semantic information to the medium itself. We describe SAMMI [1], a framework explicitly designed to fulfill this need and we present a list of possible applications pointing out the advantages that the emotional information can bring about. Finally, different scenarios are considered for the recognition of the emotions which involve different modalities, feature sets, fusion algorithms, and result optimization methods such as temporal averaging or thresholding.