Automatic food image recognition systems are alleviating the process of food-intake estimation and dietary assessment. However, due to the nature of food images, their recognition is a particularly challenging task, which is why traditional approaches in the field have achieved a low classification accuracy. Deep neural networks have outperformed such solutions, and we present a novel approach to the problem of food and drink image detection and recognition that uses a newly-defined deep convolutional neural network architecture, called NutriNet. This architecture was tuned on a recognition dataset containing 225,953 512 × 512 pixel images of 520 different food and drink items from a broad spectrum of food groups, on which we achieved a classification accuracy of 86.72%, along with an accuracy of 94.47% on a detection dataset containing 130,517 images. We also performed a real-world test on a dataset of self-acquired images, combined with images from Parkinson’s disease patients, all taken using a smartphone camera, achieving a top-five accuracy of 55%, which is an encouraging result for real-world images. Additionally, we tested NutriNet on the University of Milano-Bicocca 2016 (UNIMIB2016) food image dataset, on which we improved upon the provided baseline recognition result. An online training component was implemented to continually fine-tune the food and drink recognition model on new images. The model is being used in practice as part of a mobile app for the dietary assessment of Parkinson’s disease patients.

Background: The COVID-19 pandemic affects all aspects of human life including their food consumption. The changes in the food production and supply processes introduce changes to the global dietary patterns. Scope and Approach: To study the COVID-19 impact on food consumption process, we have analyzed two data sets that consist of food preparation recipes published before (69,444) and during the quarantine (10,009) period. Since working with large data sets is a time-consuming task, we have applied a recently proposed artificial intelligence approach called DietHub. The approach uses the recipe preparation description (i.e. text) and automatically provides a list of main ingredients annotated using the Hansard semantic tags. After extracting the semantic tags of the ingredients for every recipe, we have compared the food consumption patterns between the two data sets by comparing the relative frequency of the ingredients that compose the recipes. Key Findings and Conclusions: Using the AI methodology, the changes in the food consumption patterns before and during the COVID-19 pandemic are obvious. The highest positive difference in the food consumption can be found in foods such as “Pulses/ plants producing pulses”, “Pancake/Tortilla/Outcake”, and “Soup/pottage”, which increase by 300%, 280%, and 100%, respectively. Conversely, the largest decrease in consumption can be food for food such as “Order Perciformes (type of fish)”, “Corn/cereals/grain”, and “Wine-making”, with a reduction of 50%, 40%, and 30%, respectively. This kind of analysis is valuable in times of crisis and emergencies, which is a very good example of the scientific support that regulators require in order to take quick and appropriate response.

Evidence-based dietary information represented as unstructured text is a crucial information that needs to be accessed in order to help dietitians follow the new knowledge arrives daily with newly published scientific reports. Different named-entity recognition (NER) methods have been introduced previously to extract useful information from the biomedical literature. They are focused on, for example extracting gene mentions, proteins mentions, relationships between genes and proteins, chemical concepts and relationships between drugs and diseases. In this paper, we present a novel NER method, called drNER, for knowledge extraction of evidence-based dietary information. To the best of our knowledge this is the first attempt at extracting dietary concepts. DrNER is a rule-based NER that consists of two phases. The first one involves the detection and determination of the entities mention, and the second one involves the selection and extraction of the entities. We evaluate the method by using text corpora from heterogeneous sources, including text from several scientifically validated web sites and text from scientific publications. Evaluation of the method showed that drNER gives good results and can be used for knowledge extraction of evidence-based dietary recommendations.

No abstract

PD_Manager is a mobile health platform designed to cover most of the aspects regarding the management of Parkinson's disease (PD) in a holistic approach. Patients are unobtrusively monitored using commercial wrist and insole sensors paired with a smartphone, to automatically estimate the severity of most of the PD motor symptoms. Besides motor symptoms monitoring, the patient's mobile application also provides various non-motor self-evaluation tests for assessing cognition, mood and nutrition to motivate them in becoming more active in managing their disease. All data from the mobile application and the sensors is transferred to a cloud infrastructure to allow easy access for clinicians and further processing. Clinicians can access this information using a separate mobile application that is specifically designed for their respective needs to provide faster and more accurate assessment of PD symptoms that facilitate patient evaluation. Machine learning techniques are used to estimate symptoms and disease progression trends to further enhance the provided information. The platform is also complemented with a decision support system (DSS) that notifies clinicians for the detection of new symptoms or the worsening of existing ones. As patient's symptoms are progressing, the DSS can also provide specific suggestions regarding appropriate medication changes.

The European Food Safety Authority has developed a standardized food classification and description system called FoodEx2. It uses facets to describe food properties and aspects from various perspectives, making it easier to compare food consumption data from different sources and perform more detailed data analyses. However, both food composition data and food consumption data, which need to be linked, are lacking in FoodEx2 because the process of classification and description has to be manually performed—a process that is laborious and requires good knowledge of the system and also good knowledge of food (composition, processing, marketing, etc.). In this paper, we introduce a semi-automatic system for classifying and describing foods according to FoodEx2, which consists of three parts. The first involves a machine learning approach and classifies foods into four FoodEx2 categories, with two for single foods: raw (r) and derivatives (d), and two for composite foods: simple (s) and aggregated (c). The second uses a natural language processing approach and probability theory to describe foods. The third combines the result from the first and the second part by defining post-processing rules in order to improve the result for the classification part. We tested the system using a set of food items (from Slovenia) manually-coded according to FoodEx2. The new semi-automatic system obtained an accuracy of 89% for the classification part and 79% for the description part, or an overall result of 79% for the whole system.

The existence of annotated text corpora is essential for the development of public health services and tools based on natural language processing (NLP) and text mining. Recently organized biomedical NLP shared tasks have provided annotated corpora related to different biomedical entities such as genes, phenotypes, drugs, diseases and chemical entities. These are needed to develop named-entity recognition (NER) models that are used for extracting entities from text and finding their relations. However, to the best of our knowledge, there are limited annotated corpora that provide information about food entities despite food and dietary management being an essential public health issue. Hence, we developed a new annotated corpus of food entities, named FoodBase. It was constructed using recipes extracted from Allrecipes, which is currently the largest food-focused social network. The recipes were selected from five categories: ‘Appetizers and Snacks’, ‘Breakfast and Lunch’, ‘Dessert’, ‘Dinner’ and ‘Drinks’. Semantic tags used for annotating food entities were selected from the Hansard corpus. To extract and annotate food entities, we applied a rule-based food NER method called FoodIE. Since FoodIE provides a weakly annotated corpus, by manually evaluating the obtained results on 1000 recipes, we created a gold standard of FoodBase. It consists of 12 844 food entity annotations describing 2105 unique food entities. Additionally, we provided a weakly annotated corpus on an additional 21 790 recipes. It consists of 274 053 food entity annotations, 13 079 of which are unique. The FoodBase corpus is necessary for developing corpus-based NER models for food science, as a new benchmark dataset for machine learning tasks such as multi-class classification, multi-label classification and hierarchical multi-label classification. FoodBase can be used for detecting semantic differences/similarities between food concepts, and after all we believe that it will open a new path for learning food embedding space that can be used in predictive studies.

No abstract