Automatic Estimation of Food Intake Amount Using Visual and Ultrasonic Signals

Lee, Ki-Seung

doi:10.3390/electronics10172153

Cited by 8 publications

(4 citation statements)

References 30 publications

(62 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the same purpose, a simple drink-and-eat classification model using CNN on raw accelerometer data has been developed [20]. Using the classification approach, food recognition on ultrasonic Doppler signal has been developed [21]. The proposed smartphone app was developed using CNN and 30 food categories, which are mainly single-dish.…”

Section: Related Workmentioning

confidence: 99%

Deep-Learning-Assisted Multi-Dish Food Recognition Application for Dietary Intake Reporting

et al. 2022

View full text Add to dashboard Cite

Artificial intelligence (AI) is among the major emerging research areas and industrial application fields. An important area of its application is in the preventive healthcare domain, in which appropriate dietary intake reporting is critical in assessing nutrient content. The traditional dietary assessment is cumbersome in terms of dish accuracy and time-consuming. The recent technology in computer vision with automatic recognition of dishes has the potential to support better dietary assessment. However, due to the wide variety of available foods, especially local dishes, improvements in food recognition are needed. In this research, we proposed an AI-based multiple-dish food recognition model using the EfficientDet deep learning (DL) model. The designed model was developed taking into consideration three types of meals, namely single-dish, mixed-dish, and multiple-dish, from local Taiwanese cuisine. The results demonstrate high mean average precision (mAP) = 0.92 considering 87 types of dishes. With high recognition performance, the proposed model has the potential for a promising solution to enhancing dish reporting. Our future work includes further improving the performance of the algorithms and integrating our system into a real-world mobile and cloud-computing-based system to enhance the accuracy of current dietary intake reporting tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep-Learning-Assisted Multi-Dish Food Recognition Application for Dietary Intake Reporting

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Since foods have their own unique shapes, textures, and colors, visual cues have been used to classify food types and estimate portion sizes [10][11][12][13][14][15][16][17][18][19][20][21]. From a classical vision-based pattern recognition perspective, automatic food classification is implemented through a series of processes: segmentation, feature selection, and classification of food images.…”

Section: Introductionmentioning

confidence: 99%

Multi-Spectral Food Classification and Caloric Estimation Using Predicted Images

Lee

2024

Foods

Self Cite

View full text Add to dashboard Cite

In nutrition science, methods that accomplish continuous recognition of ingested foods with minimal user intervention have great utility. Our recent study showed that using images taken at a variety of wavelengths, including ultraviolet (UV) and near-infrared (NIR) bands, improves the accuracy of food classification and caloric estimation. With this approach, however, analysis time increases as the number of wavelengths increases, and there are practical implementation issues associated with a large number of light sources. To alleviate these problems, we proposed a method that used only standard red-green-blue (RGB) images to achieve performance that approximates the use of multi-wavelength images. This method used RGB images to predict the images at each wavelength (including UV and NIR bands), instead of using the images actually acquired with a camera. Deep neural networks (DNN) were used to predict the images at each wavelength from the RGB images. To validate the effectiveness of the proposed method, feasibility tests were carried out on 101 foods. The experimental results showed maximum recognition rates of 99.45 and 98.24% using the actual and predicted images, respectively. Those rates were significantly higher than using only the RGB images, which returned a recognition rate of only 86.3%. For caloric estimation, the minimum values for mean absolute percentage error (MAPE) were 11.67 and 12.13 when using the actual and predicted images, respectively. These results confirmed that the use of RGB images alone achieves performance that is similar to multi-wavelength imaging techniques.

show abstract

“…Since types of food are easily distinguished according to their shape, texture, and color, visual cues have been used for the classification of food types and estimation of the food amount [10,[15][16][17][18][19][20][21][22][23]. In a vision-based approach, the classification of food types can be formulated as pattern recognition problems where segmentation, feature selection and classification are sequentially carried out for food images.…”

Section: Introductionmentioning

confidence: 99%

Multispectral Food Classification and Caloric Estimation Using Convolutional Neural Networks

Lee

2023

Foods

Self Cite

View full text Add to dashboard Cite

Continuous monitoring and recording of the type and caloric content of ingested foods with a minimum of user intervention is very useful in preventing metabolic diseases and obesity. In this paper, automatic recognition of food type and caloric content was achieved via the use of multi-spectral images. A method of fusing the RGB image and the images captured at ultra violet, visible, and near-infrared regions at center wavelengths of 385, 405, 430, 470, 490, 510, 560, 590, 625, 645, 660, 810, 850, 870, 890, 910, 950, 970, and 1020 nm was adopted to improve the accuracy. A convolutional neural network (CNN) was adopted to classify food items and estimate the caloric amounts. The CNN was trained using 10,909 images acquired from 101 types. The objective functions including classification accuracy and mean absolute percentage error (MAPE) were investigated according to wavelength numbers. The optimal combinations of wavelengths (including/excluding the RGB image) were determined by using a piecewise selection method. Validation tests were carried out on 3636 images of the food types that were used in training the CNN. As a result of the experiments, the accuracy of food classification was increased from 88.9 to 97.1% and MAPEs were decreased from 41.97 to 18.97 even when one kind of NIR image was added to the RGB image. The highest accuracy for food type classification was 99.81% when using 19 images and the lowest MAPE for caloric content was 10.56 when using 14 images. These results demonstrated that the use of the images captured at various wavelengths in the UV and NIR bands was very helpful for improving the accuracy of food classification and caloric estimation.

show abstract

Automatic Estimation of Food Intake Amount Using Visual and Ultrasonic Signals

Cited by 8 publications

References 30 publications

Deep-Learning-Assisted Multi-Dish Food Recognition Application for Dietary Intake Reporting

Deep-Learning-Assisted Multi-Dish Food Recognition Application for Dietary Intake Reporting

Multi-Spectral Food Classification and Caloric Estimation Using Predicted Images

Multispectral Food Classification and Caloric Estimation Using Convolutional Neural Networks

Contact Info

Product

Resources

About