Menu-Match: Restaurant-Specific Food Logging from Images

Beijbom, Oscar; Joshi, Neel; Morris, Dan; Saponas, Scott; Khullar, Siddharth

doi:10.1109/wacv.2015.117

Cited by 118 publications

(77 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current food image datasets vary in many aspects, e.g., type of cuisine, number of food groups, and total images per food class. For instance, Menu-Match dataset [11] contains 41 food classes and a total of 646 images captured in 3 distinct restaurants while PFID [12] has 61 classes with a total of 1098 pictures captured in fast food restaurants and laboratory. There is no default food image database for general classification purpose since most databases archive specific food type.…”

Section: Related Workmentioning

confidence: 99%

A Mobile-Based Diet Monitoring System for Obesity Management

Silva¹,

Rad²,

Cui³

et al. 2018

J Health Med Informat

View full text Add to dashboard Cite

Personal diet management is key to fighting the obesity epidemic. Recent advances in smartphones and wearable sensor technologies have empowered automated food monitoring through food image processing and eating episode detection, with the goal to conquer drawbacks of traditional food journaling that is labour intensive, inaccurate, and low adherent. In this paper, we present a new interactive mobile system that enables automated food recognition and assessment based on user food images and provides dietary intervention while tracking users’ dietary and physical activities. In addition to using techniques in computer vision and machine learning, one unique feature of this system is the realization of real-time energy balance monitoring through metabolic network simulation. As a proof of concept, we have demonstrated the use of this system through an Android application.

show abstract

Section: Related Workmentioning

confidence: 99%

A Mobile-Based Diet Monitoring System for Obesity Management

Silva¹,

Rad²,

Cui³

et al. 2018

J Health Med Informat

View full text Add to dashboard Cite

show abstract

“…Menu-Match [17] uses a database of restaurants and GPS locations, and attempts to guess what is in the picture, using image features such as color and scale-invariant feature transforms [44]. It has not been made available to the general public.…”

Section: Machine Learning and Image Processingmentioning

confidence: 99%

Calorie Estimation From Pictures of Food: Crowdsourcing Study (Preprint)

Jun¹,

Bell²,

Nusrat³

et al. 2017

Preprint

View full text Add to dashboard Cite

Background: Software to accurately estimate food calories from still images could help users and health professionals more efficiently identify dietary patterns and food choices associated with health and health risks. However, calorie estimation from images is difficult, and no publicly available software can do so accurately while minimizing the burden associated with data collection and analysis. Objective: The aim of this study is to determine the accuracy of crowdsourced annotations of calorie content in food images, and to identify and quantify sources of bias and noise as a function of respondent characteristics and food qualities (e.g., energy density). Methods: We invited adult social media users to provide calorie estimates for 20 food images (for which ground truth calorie data were known) using a custom-built webpage that administers an online quiz. The images were selected to provide a range of food types and energy density. Participants optionally provided age range, gender, and their height and weight. Additionally, five nutrition experts provided annotations for the same data to form a basis of comparison. We examined estimate accuracy on the basis of expertise, demographic data, and food qualities using linear mixed effects models with participant and image index as random variables. We also analyzed the advantage of aggregating nonexpert estimates. Results: 2028 respondents agreed to participate in the study (males: 770 [38%], mean body mass index: 27.5). Average accuracy was 5 out of 20 correct guesses, where "correct" was defined as a number within 20% of the ground truth. Even a small crowd of 10 individuals achieved an accuracy of 7, exceeding the average individual's and expert annotator's accuracy of 5. Women were more accurate than men (P<.001), and younger people were more accurate than older people (P<.001). The calorie content of energy-dense foods was overestimated (P=.024). Participants did not perform better when images contained reference objects (such as credit cards) for scale. Conclusions: Our findings provide new information about the way we process food information, which can inform the design of future calorie-estimation applications.

show abstract

“…In 2015, Beijbom et al constructed Menu-Match, an image collection of restaurant-specific foods [1]. They collected 646 images, with 1, 386 tagged items across 41 categories.…”

Section: Related Workmentioning

confidence: 99%

Cookpad Image Dataset

Harashima¹,

Someya²,

Kikuta³

2017

Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

In food-related services, image information is as important as text information for users. For example, in recipe search services, users find recipes based not only on text but also images. To promote studies on food images, many datasets have recently been published. However, they have the following three limitations: most of the datasets include only thousands of images, they only take account of images after cooking not during the cooking process, and the images are not linked to any recipes. In this study, we construct the Cookpad Image Dataset, a novel collection of food images taken from Cookpad, the largest recipe search service in the world. The dataset includes more than 1.64 million images after cooking, and it is the largest among existing datasets. Additionally, it includes more than 3.10 million images taken during the cooking process. To the best of our knowledge, there are no datasets that include such images. Furthermore, the dataset is designed to link to an existing recipe corpus and thus, a variety of recipe texts, such as the title, description, ingredients, and process, is available for each image. In this paper, we described our dataset's features in detail and compared it with existing datasets.

show abstract

Menu-Match: Restaurant-Specific Food Logging from Images

Cited by 118 publications

References 29 publications

A Mobile-Based Diet Monitoring System for Obesity Management

A Mobile-Based Diet Monitoring System for Obesity Management

Calorie Estimation From Pictures of Food: Crowdsourcing Study (Preprint)

Cookpad Image Dataset

Contact Info

Product

Resources

About