Recipe Popularity Prediction with Deep Visual-Semantic Fusion

Sanjo, Satoshi; Katsurai, Marie

doi:10.1145/3132847.3133137

Cited by 30 publications

(19 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Piotrkowicz et al predicted popularity of news articles using only headline features (Piotrkowicz et al 2017). Sanjo et al proposed a visual-semantic fusion model for online recipe popularity prediction, leveraging image and short text features in recipes (Sanjo and Katsurai 2017). User-guided hierarchical attention network (Zhang et al 2018) learned modalities content and user features for social image popularity prediction.…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, recent works have proved the effectiveness of content features in the popularity prediction, such as short text descriptions, titles and images (Zhang et al 2018;Piotrkowicz et al 2017;Sanjo and Katsurai 2017). However, online articles are usually long texts which are non-trivial to model, and diverse forms of meta-data features further complicate the content feature modeling.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Popularity Prediction on Online Articles with Deep Fusion of Temporal Process and Content Features

Liao

et al. 2019

AAAI

View full text Add to dashboard Cite

Predicting the popularity of online article sheds light to many applications such as recommendation, advertising and information retrieval. However, there are several technical challenges to be addressed for developing the best of predictive capability. (1) The popularity fluctuates under impacts of external factors, which are unpredictable and hard to capture. (2) Content and meta-data features, largely determining the online content popularity, are usually multi-modal and nontrivial to model. (3) Besides, it also needs to figure out how to integrate temporal process and content features modeling for popularity prediction in different lifecycle stages of online articles. In this paper, we propose a Deep Fusion of Temporal process and Content features (DFTC) method to tackle them. For modeling the temporal popularity process, we adopt the recurrent neural network and convolutional neural network. For multi-modal content features, we exploit the hierarchical attention network and embedding technique. Finally, a temporal attention fusion is employed for dynamically integrating all these parts. Using datasets collected from WeChat, we show that the proposed model significantly outperforms state-of-the-art approaches on popularity prediction.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Popularity Prediction on Online Articles with Deep Fusion of Temporal Process and Content Features

Liao

et al. 2019

AAAI

View full text Add to dashboard Cite

show abstract

“…Users' needs give rise to smart cooking-oriented tasks that contribute towards the definition of computational cooking as a research field by itself [1]. Indeed, the research community is very active in investigating issues regarding food-related tasks, such as ingredient identification [8], recipe recommendation [11], or recipe popularity prediction [35]. A first line of work consists in leveraging the semantics behind recipe texts and images using deep learning approaches [34,35].…”

Section: Related Work 21 Computational Cookingmentioning

confidence: 99%

“…Designing powerful tools that support cooking activities has become an attractive research field in recent years due to the growing interest of users to eat home-made food and share recipes on social platforms [35]. These massive amounts of data shared on devoted sites, such as All Recipes 1 , allow gathering food-related data including text recipes, images, videos, and/or user preferences.…”

Section: Introductionmentioning

confidence: 99%

“…These massive amounts of data shared on devoted sites, such as All Recipes 1 , allow gathering food-related data including text recipes, images, videos, and/or user preferences. Consequently, novel applications are rising, such as ingredient classification [7], recipe recognition [39] or recipe recommendation [35]. However, solving these tasks is challenging since it requires taking into consideration 1) the heterogeneity of data in terms of format (text, image, video, ...) or structure (e.g., list of items for ingredients, short verbal sentence for instructions, or verbose text for users' reviews); and 2) the cultural factor behind each recipe since the vocabulary, the quantity measurement, and the flavor perception is culturally intrinsic; preventing the homogeneous semantics of recipes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cross-Modal Retrieval in the Cooking Context

Carvalho

Cadène

Picard

et al. 2018

The 41st International ACM SIGIR Conference on Research &Amp; Development in Information Retrieval

139

View full text Add to dashboard Cite

Designing powerful tools that support cooking activities has rapidly gained popularity due to the massive amounts of available data, as well as recent advances in machine learning that are capable of analyzing them. In this paper, we propose a cross-modal retrieval model aligning visual and textual data (like pictures of dishes and their recipes) in a shared representation space. We describe an effective learning scheme, capable of tackling large-scale problems, and validate it on the Recipe1M dataset containing nearly 1 million picture-recipe pairs. We show the effectiveness of our approach regarding previous state-of-the-art models and present qualitative results over computational cooking use cases. CCS CONCEPTS • Information systems → Multimedia and multimodal retrieval; • Computer systems organization → Neural networks;

show abstract