Applying Image-Based Food-Recognition Systems on Dietary Assessment: A Systematic Review

Abstract

Dietary assessment can be crucial for the overall well-being of humans and at least in some instances for the prevention and management of chronic, life-threatening diseases. Recall and manual record keeping methods for food intake monitoring are available, but often inaccurate when applied for a long period of time. On the other hand, automatic record keeping approaches that adopt mobile cameras and computer vision methods seem to simplify the process and can improve current human-centric diet monitoring methods. Here we present an extended critical literature overview of image-based food recognition systems (IBFRS) combining a camera of the user's mobile device with computer vision methods and publicly available food datasets (PAFD). In brief, such systems consist of several phases, such as the segmentation of the food items on the plate, the classification of the food items in a specific food category, and the estimation phase of volume, calories or nutrients of each food item. 159 studies were screened in this systematic review of IBFRS. A detailed overview of the methods adopted in each of the 78 included studies of this systematic review of IBFRS is provided along with their performance on PAFD. Studies that included IBFRS without presenting their performance in at least one of the abovementioned phases were excluded. Among the included studies, 45 (58%) studies adopted deep learning methods and especially Convolutional Neural Networks (CNNs) in at least one phase of the IBFRS with input PAFD. Among the implemented techniques, CNNs outperform all other approaches on the PAFD with a large volume of data, since the richness of these datasets provides adequate training resources for such algorithms. We also present evidence for the benefits of application of IBFRS in professional dietetic practice. Furthermore, challenges related to the IBFRS presented here are also thoroughly discussed along with future directions.

Use of digital food records in clinical settings - lessons in translation from a randomised controlled trial

“…Image or video food recognition systems, have various intelligence features to detect and analyse food items and portions without needing the participant to manually enter intake (36). Photo assisted apps have shown to provide good accuracy, positive diet changes and good acceptability (37). However, further research is needed to assess larger samples, validity, involved costs (38) and technical challenges (39) in complex clinical settings.…”

Digital Therapeutics: Emerging New Therapy for Nonalcoholic Fatty Liver Disease

“…To date, several diet quantification methods are widely used, such as weighed food records, hospital-ordered food delivery, the recall method, food frequency questionnaires, and vision-based measurement systems ( 74 – 77 ). Out of these methods, vision-based measurement systems show a promising future for study, as they need only a mobile camera to capture food images ( 78 ).…”

Wearables and smartphones for modifiable risk factors in metabolic health: a scoping review protocol

“…For instance, smartwatches can track physical activity and sleep patterns with minimal burden [15]. Further, mobile apps can detect nutrition via image-based food recognition [14], ecologic momentary assessment, or short surveys [16] in free-living conditions. These high-dimensional, longitudinal data can trigger personalized lifestyle interventions by deploying recommendations on nutrition, sleep, and physical activity [15,16].…”

“…For instance, smartwatches can track physical activity and sleep patterns with minimal burden [15]. Further, mobile apps can detect nutrition via image-based food recognition [14], ecologic momentary assessment, or short surveys [16] in free-living conditions.…”