FoodFinder: developing a rapid low-cost crowdsourcing approach for obtaining data on meal size from meal photos

Abstract

Capturing meal images using mobile phone cameras is a promising alternative to traditional dietary assessment methods. Acquiring photos is reasonably simple but nutritional content analysis of images is a major challenge. Automated food identification and portion size assessment is computationally and participant intensive; relying on participant feedback for accuracy (1) . Dietitian analysis of photos is accurate but time-consuming and expensive (2) . Crowdsourcing could offer a rapid low-cost alternative by utilising the life-long experience that all humans have in food identification. Previous crowdsourcing methods include the Eatery app, which produces a simple 11-point 'healthiness' scale for each meal (3) and the PlateMate system, which creates a list of all individual foods with portion sizes, energy and macronutrient content (4) . While the Eatery produces limited and subjective data on meal content, PlateMate represents a complex integrated system of multiple tasks requiring on average 25 workers, costing £2·75 and taking 90 min per image. For feasible data-capture in a large-scale longitudinal studies, crowdsourcing data from meal photos needs to be cheaper and quicker. We aimed to develop a simpler task and tested it's feasibility for crowdsourcing dietary data.FoodFinder, a single task for identifying food groups and portion sizes, developed using Qualtrics (www.qualtrics.com/), and linked to the Prolific Academic (https://prolific.ac/) crowdsourcing platform for recruitment and reimbursement of a UK crowd. Thirty meal photos with measured total meal weight (grams) were analysed by a dietitian and crowds ranging in size from 5 to 50 people. The difference between actual meal weight (the gold-standard) and total meal weight estimated by different sized crowds and ratings by a dietician were compared to each other. To establish group consensus crowd estimates were weighted by majority agreement (5) . Bland-Altman analysis assessed agreement with actual meal weight.A crowd of 5 people underestimated true meal weight by 63 g, equating to 15 % of actual meal weight with limits of agreement (LOA) from −299 to 174 g. In comparison experts overestimated by 28 g equating to 9 % of actual meal weight with LOA −158, 214 g. With a crowd of 5 people, crowdsourcing cost £3·35 and took a mean 2 mins 55 sec (SD 2 min 6 sec) per image. A crowd of 50 had similar accuracy and limits of agreement (−65 g LOA −278, 149 g) but was more expensive. Further development of FoodFinder is required to make rapid low-cost analysis of meal photos via crowdsourcing a feasible method for assessing diet.

The second generation of The Avon Longitudinal Study of Parents and Children (ALSPAC-G2): a cohort profile

“…For each of these photos total meal weight was measured using an Mandometer® device (Mikrodidakt AB, Lund, Sweden) and food groups displayed were reviewed by an expert dietitian. In comparison to the measured weight, crowds underestimated meal weight by an average of 63 g (95% level of agreement -299 to 174 g), whereas the dietitian overestimated by 28 g (95% level of agreement -158 to 214 g) 15 . In further analyses, we found that compared with expert dietician review, crowds varying in size from 5 to 50 people identified food groups in the photos with high specificity (mean 98%) but modest sensitivity (68%) i.e.…”

Future Directions for Integrative Objective Assessment of Eating Using Wearable Sensing Technology

“…Depending on the needs of the particular scenario, and balancing speed, accuracy, and cost, data from the central profile may be sent for further analysis. For example, images of meals may be sent to a crowd-based application (44), or a dietician to refine food item identification and portion sizes (72).…”

“…To be feasible for use in large-scale longitudinal studies or public health interventions, crowdsourcing of food data from photographs needs to be fast and low cost. We developed and piloted a novel system, FoodFinder (44), and found that small (n = 5) untrained crowds could rapidly classify foods and estimate meal weight in 3 min for £3.35 per photo. Crowds underestimated measured meal weight by 15% compared with 9% overestimation by an expert.…”

Future directions for integrative objective assessment of eating using wearable sensing technology

“…Depending on the needs of the particular scenario, and balancing speed, accuracy, and cost, data from the central profile may be sent for further analysis. For example, images of meals may be sent to a crowd-based application 42 , or a dietician to refine food item identification and portion sizes 68 .…”

“…To be feasible for use in large-scale longitudinal studies or public health interventions, crowdsourcing of food data from photographs needs to be fast and low cost. We developed and piloted a novel system, FoodFinder 42 , and found that small (n=5) untrained crowds could rapidly classify foods and estimate meal weight in 3 min for £3.35 per photo. Crowds underestimated measured meal weight by 15% compared with 9% overestimation by an expert.…”