Wide-Slice Residual Networks for Food Recognition

Abstract

Food diary applications represent a tantalizing market. Such applications, based on image food recognition, opened to new challenges for computer vision and pattern recognition algorithms. Recent works in the field are focusing either on hand-crafted representations or on learning these by exploiting deep neural networks. Despite the success of such a last family of works, these generally exploit off-the shelf deep architectures to classify food dishes. Thus, the architectures are not cast to the specific problem. We believe that better results can be obtained if the deep architecture is defined with respect to an analysis of the food composition. Following such an intuition, this work introduces a new deep scheme that is designed to handle the food structure. Specifically, inspired by the recent success of residual deep network, we exploit such a learning scheme and introduce a slice convolution block to capture the vertical food layers. Outputs of the deep residual blocks are combined with the sliced convolution to produce the classification score for specific food categories. To evaluate our proposed architecture we have conducted experimental results on three benchmark datasets. Results demonstrate that our solution shows better performance with respect to existing approaches (e.g., a top-1 accuracy of 90.27% on the Food-101 challenging dataset).

Application of Deep Learning in Food: A Review

“…There are various popular CNN architectures for image processing including AlexNet (Krizhevsky et al, 2012), a network using repetitive units called visual geometry group network (VGG) (Simonyan & Zisserman, 2014), GoogLeNet (Szegedy et al, 2015) that includes parallel data channels, and residual neural network (ResNet) (He, Zhang, Ren, & Sun, 2016) constructed by residual (Heravi, Aghdam, & Puig, 2018) 65.40 87.00 CNN-FOOD 70.41 / Multitask (H. Wu, Merler, Uceda-Sosa, & Smith, 2016) 72.11 / FoodNet (Pandey, Deepthi, Mandal, & Puhan, 2017) 72.12 91.61 DeepFood (Liu et al, 2016a) 77.40 93.70 Inception Module (C. 77.00 94.00 ResNet (Fu et al, 2017) 78.50 94.10 ResNet-50 (Ciocca, Napoletano, & Schettini, 2018) 82.54 95.79 Inception-v3+FPCNN (Zheng, Zou, & Wang, 2018) 87.96 / Inception V3 (Hassannejad et al, 2016) 88.28 96.88 wide-slice residual networks (WISeR) (Martinel, Foresti, & Micheloni, 2018) 90.27 98.71…”

“…DeepFood (Liu et al, 2016a) 54.70 81.50 Author defined (Heravi et al, 2018) 60.00 85.00 Inception Module (C. 63.60 87.00 DeepFoodCam (Kawano & Yanai, 2014) 63.77 85.82 CNN-FOOD 67.57 88.97 ResNet (Fu et al, 2017) 71.20 91.10 ResNet-50 71.70 91.33 Inception V3 (Hassannejad et al, 2016) 76.17 92.58 Inception-v3+FP-CNN 78.60 / WISeR (Martinel et al, 2018) 83.15 95.45 UECFood-100: Images of 100 kinds of Japanese food with at least 100 images per category…”

“…DeepFoodCam (Kawano & Yanai, 2014) 72.26 92.00 DeepFood (Liu et al, 2016a) 76.3 94.60 Inception Module (C. 76.30 94.60 CNN-FOOD 78.48 94.85 ResNet (Fu et al, 2017) 80.60 95.90 Inception V3 (Hassannejad et al, 2016) 81.45 97.27 MultiTaskCNN 82.12 97.29 Inception-v3+FP-CNN 86.51 / WISeR (Martinel et al, 2018) 89.58 99.23…”

“…Thus, they concluded that the dataset containing more features for food image could further promote the food recognition algorithms. Martinel et al (2018) were committed to improve the accuracy of food recognition from another point of view. They found that almost all published deep learning methods for food recognition just exploited off-the-shelf deep models or modified the ones without considering the specific characteristics in food images.…”

A Comparative Study of Indian Food Image Classification Using K-Nearest-Neighbour and Support-Vector-Machines

“…Hokuto Kagaya, Kiyoharu Aizawa and Makoto Ogawa in [16], proposed the tasks of food detection and recognition through parameter optimization and how to construct a dataset of the most frequent food items in a publicly available food-logging system, and used it to evaluate recognition performance. Niki Martinel, Gian Luca Foresti, and Christian Michelon in [17] this paper introduces a new deep scheme that is designed to handle the food structure and also explains about the recent success of residual deep network, introduce a slice convolution block to capture the vertical food layers. Outputs of the deep residual blocks are combined with the sliced convolution to produce the classification score for specific food categories.…”