Feasibility of identifying eating moments from first-person images leveraging human computation

Abstract

There is widespread agreement in the medical research community that more effective mechanisms for dietary assessment and food journaling are needed to fight back against obesity and other nutrition-related diseases. However, it is presently not possible to automatically capture and objectively assess an individual's eating behavior. Currently used dietary assessment and journaling approaches have several limitations; they pose a significant burden on individuals and are often not detailed or accurate enough. In this paper, we describe an approach where we leverage human computation to identify eating moments in first-person point-of-view images taken with wearable cameras. Recognizing eating moments is a key first step both in terms of automating dietary assessment and building systems that help individuals reflect on their diet. In a feasibility study with 5 participants over 3 days, where 17,575 images were collected in total, our method was able to recognize eating moments with 89.68% accuracy.

Usefulness of Wearable Cameras as a Tool to Enhance Chronic Disease Self-Management: Scoping Review (Preprint)

“…Studies including data analysis were predominantly feasibility or pilot studies (n = 13, 54%) [9][10][11]16,20,[22][23][24][25][26][27][28][29], followed by validation studies (n = 4, 17%) [8,28,30,31] and methodological studies (n = 5, 21%) [32][33][34][35][36]. There was one randomised controlled trial [37] conducted with acquired brain injury patients where camera images formed part of a health intervention.…”

“…There was one randomised controlled trial [37] conducted with acquired brain injury patients where camera images formed part of a health intervention. The majority of the studies were conducted in the United States of America (n = 7, 29%) [22,25,27,29,31,32,36], the United Kingdom (n = 6, 25%) [9][10][11][12][13][14], and Ireland (n = 4, 13%) [24,30,33,35]. Three studies [23,26,28] were international multicentre studies, while the remaining studies were from New Zealand [8,20,34] and Spain [37].…”

“…Feasibility and pilot studies [9][10][11]16,20,[22][23][24][25][26][27][28][29] focussed predominantly on assessing the feasibility of using wearable cameras to capture information on dietary intake/food consumption, physical activity and sedentary behaviour. Studies classified as validation [8,28,30,31] compared the validity of wearable cameras either against, or in combination with other measurement approaches, including self-reported diaries, questionnaires, as well as objective measurement techniques such as accelerometers and doubly labelled water.…”

“…Eight had between 20 -50 participants [8,9,13,20,24,[29][30][31][32][33]36,37] and only three studies (13%) [26][27][28] had more than 50 participants (maximum = 84). Participants were predominantly healthy adults (n= 15, 65%) [8,[10][11][12][13]20,22,23,[25][26][27][28]33,35,36], with some studies including athletes (n = 4, 17%) [24,[29][30][31] and school children (n = 2, 9%) [9,34]. One study used a sample of older women to examine physical activity and sedentary time [32].…”

“…In 21 studies (81%) participants wore a camera for between three to seven days; in one study, participants wore the cameras for seven weeks [37]. Studies classified as primary or secondary data analyses used wearable cameras to measure, observe or validate specific behaviours related to dietary intake (n = 10, 38%) [8][9][10][11]20,[22][23][24][25]39], physical activity (n =5, 17%) [26,27,30,32,40], a diverse range of daily activities such as travel, work and exposure to food marketing (n = 5, 17%) [12,[33][34][35]37], sedentary behaviour (n = 4, 15%) [13,29,31,36] and travel behaviours such as walking, cycling or motorised transport (n = 2, 8%) [14,28].…”