A Novel Approach to Dining Bowl Reconstruction for Image-Based Food Volume Estimation

Abstract

Knowing the amounts of energy and nutrients in an individual’s diet is important for maintaining health and preventing chronic diseases. As electronic and AI technologies advance rapidly, dietary assessment can now be performed using food images obtained from a smartphone or a wearable device. One of the challenges in this approach is to computationally measure the volume of food in a bowl from an image. This problem has not been studied systematically despite the bowl being the most utilized food container in many parts of the world, especially in Asia and Africa. In this paper, we present a new method to measure the size and shape of a bowl by adhering a paper ruler centrally across the bottom and sides of the bowl and then taking an image. When observed from the image, the distortions in the width of the paper ruler and the spacings between ruler markers completely encode the size and shape of the bowl. A computational algorithm is developed to reconstruct the three-dimensional bowl interior using the observed distortions. Our experiments using nine bowls, colored liquids, and amorphous foods demonstrate high accuracy of our method for food volume estimation involving round bowls as containers. A total of 228 images of amorphous foods were also used in a comparative experiment between our algorithm and an independent human estimator. The results showed that our algorithm overperformed the human estimator who utilized different types of reference information and two estimation methods, including direct volume estimation and indirect estimation through the fullness of the bowl.

Food Classification and Meal Intake Amount Estimation through Deep Learning

“…The estimation methods based on the reference object have an inconvenience where the reference object should be captured with food. In order to overcome this inconvenience, a shape-known dish or bowl can be utilized as the reference object [15][16][17]. Jia et al [15] generate a 3D bowl model by measuring distances between line marks in the graduated tape attached at the bowl to estimate the meal intake amount.…”

“…In order to overcome this inconvenience, a shape-known dish or bowl can be utilized as the reference object [15][16][17]. Jia et al [15] generate a 3D bowl model by measuring distances between line marks in the graduated tape attached at the bowl to estimate the meal intake amount. Fang et al [16] and Yue et al [17] calculate the camera parameters such as a camera pose and a camera focus length through the shape-known dish.…”

“…Several attempts [6][7][8][9][10][11] have been made to estimate the food volumes using stereo vision images or an RGB-D image. Some studies [12][13][14][15][16][17] estimate the food volume based on a shape-known reference object in a single image. Nevertheless, these are unsuitable for practical applications so far because the meal intake amount estimation is possible under limited conditions.…”

Egocentric Image Captioning for Privacy-Preserved Passive Dietary Intake Monitoring

“…However, directly estimating food volume from an RGB image is difficult as food has various and irregular shapes, and food items/ingredients are often occluded. Researchers have tried to estimate food volume from an RGB image by first estimating the size of the food container [26], but this still is not the actual food volume. In this work, we annotate dietary intake images with food portion size measured with respect to its container, for example, half bowl, two bowls, and the bowl is empty.…”

Amount Estimation Method for Food Intake Based on Color and Depth Images through Deep Learning

“…However, food volume measurement based on color images is challenging work because color images lack spatial information for reconstructing 3D scenes. The following volume estimation studies for food have been conducted to overcome the limitations of volume measurement based on color image utilizing a reference object with a known size and a volume [14][15][16][17]; employing predefined 3D models for foods [18,19]; and acquiring 3D information through camera parameters [20][21][22]. However, the methods are hard to apply practically because they require prior knowledge about the size and shape of a reference object or a food container.…”