Embedded Implementation of a Deep Learning Smile Detector

Abstract

In this paper we study the real time deployment of deep learning algorithms in low resource computational environments. As the use case, we compare the accuracy and speed of neural networks for smile detection using different neural network architectures and their system level implementation on NVidia Jetson embedded platform. We also propose an asynchronous multithreading scheme for parallelizing the pipeline. Within this framework, we experimentally compare thirteen widely used network topologies. The experiments show that low complexity architectures can achieve almost equal performance as larger ones, with a fraction of computation required.

On-Device Object Detection for More Efficient and Privacy-Compliant Visual Perception in Context-Aware Systems

“…[73,74,88,97,110] and smart cities [72,100,101,108], all of them, scenarios where constant and real-time object detection is necessary for enabling context-awareness on end devices. While further information on each of those domains will be incorporated into the discussion in successive paragraphs to draw a clearer picture, it should be noted first that additional application areas, albeit almost residually with only one or two related works identified, have emerged in the analysis: (i) robotics [81,94], a domain where vision represents one of the most important communication channels with the environment, and where object detection has traditionally shown to be critical for the perception, modeling, planning, and understanding of unknown terrains [94]; (ii) defense, where object detection constitutes a major factor for controlling UAVs [84] and detecting ships in radar images [86]; (iii) smart logistics, with two distinct but equally representative examples of the use of sensing technologies, one on embedded platforms (in situ detection and recognition of ships for more efficient port management) [83], and the second one on mobile devices (barcode detection) [99] and finally, (iv) human emotion recognition based on facial expression detection, as reported in [71]. Once the less representative options are covered, the remainder of this section will deal with the most common domains that have emerged in the analysis.…”

“…This transfer learning approach enables reusing already existing models [48,55,63,70,75,76,[78][79][80][81][82]87,91,100,103,107], previously trained on public standard object detection benchmarks, such as Pascal VOC [115] (used in [48]), and Microsoft COCO [126] (used in [63,75,78,80,91,107]). Still, as shown in Table 2, it is a common practice as well to exploit not the entire detection model but merely the backbone [49,50,59,68,71,95,106,109], being this a CNN embedded in the detection framework, responsible for extracting from some given input images the different feature maps subsequently exploited by the deeper layers of the detector for predicting the several classes and bounding boxes produced as output. In any case, either globally or just circumscribed to the backbone, weights are initialized with values taken directly from pre-trained models, and then, through a fine-tuning process, the detector is re-trained on an application-specific dataset in order to adjust it to the specific use case to be addressed.…”

“…The study revealed two different approaches devised in response to the noted datarelated shortcomings: the creation of ad hoc datasets for the problem addressed [50,64,68,[77][78][79][80]84,86,91,95,98,100,103,109], and data augmentation [48,50,52,53,57,60,61,64,70,71,73,74,76,81,84,85,87,97,98,100,106]. As shown in Table 3, both solutions have been widely adopted by the authors, a fact quite surprising considering the substantial effort typically required to address the various tasks involved in creating new datasets (image and video acquisition and homogenization, but fundamentally, the creation of annotations).…”

“…In this regard, in the AmI literature reviewed, it has been reported that a fair number of operations can be classified into two distinct groups: operations at the pixel level and geometric transformations. The pixel-level category involves operations that affect image pixel values, producing alterations in some properties, such as hue [50,53], saturation [50,53], contrast [76], brightness [76], exposure [50,53], color [60,61,64,76,85], even incorporating into images noise [76], or blurring [64,97], while geometric transformations includes practices, such as image scaling [48,71,76,85,100,106], translation [81], horizontal and vertical flipping [53,57,60,61,64,71,73,74,76,81,84,85,87,97,98,100], random cropping [50,52,53,60,61,73,…”

“…Table 4, in particular, the field tagged as "Test device", lists the several devices used in the AmI research analyzed for evaluation purposes. It is possible to classify those devices into three different groups: high-performance single-board computers [49,[52][53][54]56,57,65,66,70,71,[75][76][77]82,83,89,[91][92][93][94]103,105,106,108], low-power single-board computers [60,67,69,72,75,78,95,100,103], and mobile devices [63,68,73,74,80,87,99]. This categorization, however, does not contemplate papers where the hardware setup used for model deployment is not properly detailed or detailed at all [48,51,55,64,81,…”