Two-View 3D Reconstruction for Food Volume Estimation

Abstract

Abstract-The increasing prevalence of diet-related chronic diseases coupled with the ineffectiveness of traditional diet management methods have resulted in a need for novel tools to accurately and automatically assess meals. Recently, computer vision based systems that use meal images to assess their content have been proposed. Food portion estimation is the most difficult task for individuals assessing their meals and it is also the least studied area. The present paper proposes a three-stage system to calculate portion sizes using two images of a dish acquired by mobile devices. The first stage consists in understanding the configuration of the different views, after which a dense 3D model is built from the two images; finally, this 3D model serves to extract the volume of the different items. The system was extensively tested on 77 real dishes of known volume, and achieved an average error of less than 10% in 5.5 seconds per dish. The proposed pipeline is computationally tractable and requires no user input, making it a viable option for fully automated dietary assessment.

A Survey on Food Computing

“…To make the system fullautomatic, [Zhu et al 2010[Zhu et al , 2008 proposed a technology assisted dietary assessment system, where images obtained before and after foods are eaten, are used to estimate the type and amount of food consumed. Similar methods including sing-view reconstruction and multi-view reconstruction for food volume estimation [Almaghrabi et al 2012;Dehais et al 2017;Pouladzadeh et al 2014b;Shevchik et al 2013;Xu et al 2013] are proposed. Such methods generally need 3D reconstruction from food images.…”

Food Recommendation: Framework, Existing Solutions, and Challenges

“…Visual food analysis can obtain a high-level understanding of the type (e.g.,the food category and ingredients), the amount of food consumed by the user and even the calorie, and thus is very essential for food recommendation. This category can broadly be divided into different types, such as food category recognition [14], food ingredient recognition [7], cooking instruction recognition [49] and food quantity estimation [32], [41]. However, accurate visual food analysis is very challenging.…”

A multi-task learning approach for meal assessment

“…Puri et al [9] used a dense multi-view 3D reconstruction approach, which generated the 3D point cloud of the food based on a video sequence and plate-sized reference patterns. Recently, Dehais et al [10] proposed a two-view 3D reconstruction approach using a credit card sized reference card. The approach was extensively tested on real dishes of known volume, and achieved an average error of less than 10% in 5.5 seconds per dish.…”