Counting Bites With Bits: Expert Workshop Addressing Calorie and Macronutrient Intake Monitoring

Abstract

BackgroundConventional diet assessment approaches such as the 24-hour self-reported recall are burdensome, suffer from recall bias, and are inaccurate in estimating energy intake. Wearable sensor technology, coupled with advanced algorithms, is increasingly showing promise in its ability to capture behaviors that provide useful information for estimating calorie and macronutrient intake.ObjectiveThis paper aimed to summarize current technological approaches to monitoring energy intake on the basis of expert opinion from a workshop panel and to make recommendations to advance technology and algorithms to improve estimation of energy expenditure.MethodsA 1-day invitational workshop sponsored by the National Science Foundation was held at Northwestern University. A total of 30 participants, including population health researchers, engineers, and intervention developers, from 6 universities and the National Institutes of Health participated in a panel discussing the state of evidence with regard to monitoring calorie intake and eating behaviors.ResultsCalorie monitoring using technological approaches can be characterized into 3 domains: (1) image-based sensing (eg, wearable and smartphone-based cameras combined with machine learning algorithms); (2) eating action unit (EAU) sensors (eg, to measure feeding gesture and chewing rate); and (3) biochemical measures (eg, serum and plasma metabolite concentrations). We discussed how each domain functions, provided examples of promising solutions, and highlighted potential challenges and opportunities in each domain. Image-based sensor research requires improved ground truth (context and known information about the foods), accurate food image segmentation and recognition algorithms, and reliable methods of estimating portion size. EAU-based domain research is limited by the understanding of when their systems (device and inference algorithm) succeed and fail, need for privacy-protecting methods of capturing ground truth, and uncertainty in food categorization. Although an exciting novel technology, the challenges of biochemical sensing range from a lack of adaptability to environmental effects (eg, temperature change) and mechanical impact, instability of wearable sensor performance over time, and single-use design.ConclusionsConventional approaches to calorie monitoring rely predominantly on self-reports. These approaches can gain contextual information from image-based and EAU-based domains that can map automatically captured food images to a food database and detect proxies that correlate with food volume and caloric intake. Although the continued development of advanced machine learning techniques will advance the accuracy of such wearables, biochemical sensing provides an electrochemical analysis of sweat using soft bioelectronics on human skin, enabling noninvasive measures of chemical compounds that provide insight into the digestive and endocrine systems. Future computing-based researchers should focus on reducing the burden of wearable sensors, aligning data across m...

Perspective: A Framework for Addressing Dynamic Food Consumption Processes

“…Food consumption can also be defined based on how food is eaten in terms of quantity, such as speed of eating, or in terms of quality, such as characterizing mindful eating ( 34 ). For example, emerging technologies detecting bites and chews can detect when food is eaten as well as the speed of eating ( 35–37 ), as when seeking to understand how meal timing impacts metabolic health ( 38–40 ). Broader contextual information such as social cues from dining partners can motivate measurement of other qualities, such as mimicry of food consumption ( 41 , 42 ).…”

“…To illustrate with Roma's story, measurements could be quite crude at first, aiming to determine dynamics connecting how food is eaten and GI symptoms. For example, passive sensors detecting hand-to-mouth movements or chewing sounds might be used at first to determine the timing of eating occasions ( 35 , 36 ). Roma's GI symptoms might be captured by self-monitoring [e.g., use of ecological momentary assessment to report on symptoms throughout the day in real time ( 47 , 48 )] and sensing methods [e.g., gastric contractions sensed through a band worn on the abdomen ( 49 )].…”

Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review

“…Our review indicates that there is still a strong reliance on self-report as the method of determining ground-truth eating activity in the field. Although using objective methods such as wearable video cameras 68 to determine when eating activity has occurred can be very costly and time-intensive, maintaining eating and/or activity logs is a process that is very burdensome on the participant and ultimately relies on their memory on when food was eaten. In studies comparing food diaries to the doubly labeled water method, which uses doubly labeled water as a biomarker for energy intake and is considered the "gold-standard" method 69 of dietary assessment, there is evidence that misreporting of dietary intake is common 17 .…”

Counting Bites and Recognizing Consumed Food from Videos for Passive Dietary Monitoring

“…1) EAU-based Systems: Detecting eating action units (EAUs) [4] such as feeding gestures is a straightforward way to assist dietary intake monitoring and assessment. EAU-based systems are mainly based on inertial or acoustic sensing.…”

“…To meet the need for objective and accurate dietary assessments, sensing-based technological approaches are emerging with the aim to transform these conventional subjective assessments. Current technological approaches include imagebased sensing, sensing with eating action unit (EAU), and biochemical sensing [4]. EAU-based sensing approaches are primarily designed for detecting eating actions and some are able to produce reasonable estimates with the use of inertial sensors [5]- [13], but one evident limitation of the EAU-based approaches is that they can hardly recognize food items.…”