A Retail Object Classification Method Using Multiple Cameras for Vision-Based Unmanned Kiosks

Abstract

Several unmanned retail stores have been introduced with the development of sensors, wireless communication, and computer vision technologies. A vision-based kiosk that is only equipped with a vision sensor has significant advantages such as compactness and low implementation cost. Using convolutional neural network (CNN)-based object detectors, the kiosk recognizes an object when a customer picks up a product. In retail object recognition, the key challenge is the limited number of detections and high interclass similarity. In this study, these challenges are addressed by utilizing the "view-specific" feature of an object; specifically, an object class is divided into multiple "view-based" subclasses, and the object detectors are trained using these data. Further, the "view-aware feature" is defined by aggregating subclass detection results from multiple cameras. A superclass classifier predicts a superclass by utilizing an informative subclass detection result that distinguishes the target object from other similar-looking objects. To verify the effectiveness of the proposed approach, a prototype of the vision-based unmanned kiosk system is implemented. Experimental results indicate that the proposed method outperforms the conventional method, even on a state-of-the-art detection network. The dataset used in this study has been subsequently provided in the IEEE DataPort for reproducibility.

Outlet Recognition Based on Deep Learning and Sensors for Large-Scale Cable Laying

“…In addition to being applied to augmented reality, sensors are combined with intelligent algorithms for reliable target recognition. Li et al [4] combined a hybrid tactile sensor with a robot and analyzed the tactile information feedback from the sensor through machine learning to achieve object recognition; Jeon et al [5] combined visual sensors with convolutional neural networks to achieve classification of retail products, solving the problem of similarity between classes; Chen et al [6] implemented heterogeneous fusion of LiDAR and visual sensors, combined with deep learning, to achieve obstacle detection and distance measurement in front of vehicles.…”

Use of Mobile Technologies to Streamline Pretriage Patient Flow in the Emergency Department: Observational Usability Study

“…For example, Sinha et al [ 11 ] demonstrated that the registration process can be easily completed by patients or their proxies using kiosks. However, unlike unmanned retail stores [ 12 ], there is no evidence that the use of kiosks can completely replace human-based triage (ie, triage nurses). Without proper control, full self-triage could lead to disparities.…”