Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps

Sehgal, Abhishek; Kehtarnavaz, Nasser

doi:10.3390/make1010027

Cited by 37 publications

(23 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As most of the smartphones are equipped with a GPU [ 37 ], Recurrent BIM-PoseNet can run in real-time on such devices, deeming it suitable for practical applications. However, there will be a lag in the camera pose estimation depending on the window length.…”

Section: Experiments and Resultsmentioning

confidence: 99%

A Recurrent Deep Network for Estimating the Pose of Real Indoor Images from Synthetic Image Sequences

Acharya

Roy

Khoshelham

et al. 2020

Sensors

View full text Add to dashboard Cite

Recently, deep convolutional neural networks (CNN) have become popular for indoor visual localisation, where the networks learn to regress the camera pose from images directly. However, these approaches perform a 3D image-based reconstruction of the indoor spaces beforehand to determine camera poses, which is a challenge for large indoor spaces. Synthetic images derived from 3D indoor models have been used to eliminate the requirement of 3D reconstruction. A limitation of the approach is the low accuracy that occurs as a result of estimating the pose of each image frame independently. In this article, a visual localisation approach is proposed that exploits the spatio-temporal information from synthetic image sequences to improve localisation accuracy. A deep Bayesian recurrent CNN is fine-tuned using synthetic image sequences obtained from a building information model (BIM) to regress the pose of real image sequences. The results of the experiments indicate that the proposed approach estimates a smoother trajectory with smaller inter-frame error as compared to existing methods. The achievable accuracy with the proposed approach is 1.6 m, which is an improvement of approximately thirty per cent compared to the existing approaches. A Keras implementation can be found in our Github repository.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

A Recurrent Deep Network for Estimating the Pose of Real Indoor Images from Synthetic Image Sequences

Acharya

Roy

Khoshelham

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…ML specific options are e.g. to optimize towards the target hardware [120] regarding CPU and GPU availability, to optimize towards the target operation system (demonstrated for Android and iOS by Sehgal and Kehtarnavaz [121]) or to optimize the ML workload for a specific platform [122]. Monitoring and maintenance (see section 3.6) have to be considered in the overall architecture.…”

Section: Deploymentmentioning

confidence: 99%

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Studer¹,

Bui²,

Drescher³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning is an established and frequently used technique in industry and academia but a standard process model to improve success and efficiency of machine learning applications is still missing. Project organizations and machine learning practitioners have a need for guidance throughout the life cycle of a machine learning application to meet business expectations. We therefore propose a process model for the development of machine learning applications, that covers six phases from defining the scope to maintaining the deployed machine learning application. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications, as the risk of model degradation in a changing environment is eminent. With each task of the process, we propose quality assurance methodology that is suitable to address challenges in machine learning development that we identify in form of risks. The methodology is drawn from practical experience and scientific literature and has proven to be general and stable. The process model expands on CRISP-DM, a data mining process model that enjoys strong industry support but lacks to address machine learning specific tasks. Our work proposes an industry and application neutral process model tailored for machine learning applications with focus on technical tasks for quality assurance.

show abstract

“…Researches show that the number of operations in a network model can effectively estimate inference time [ 5 ]. The number of FLOPs represents how computationally expensive a model is [ 50 ]. We customize the FLOPs approach suggested by Sehgal et al [ 50 ] to calculate the computational complexity of a neural network as defined in Eq.…”

Section: Proposed Approachesmentioning

confidence: 99%

“…This particular problem was selected because the solution could be applicable to larger objects (e.g., aircrafts, trucks, ships, buildings) and generalizable across other satellite imagery datasets. We have adapted a floating-point operation (FLOP) framework [ 50 ] to measure the model’s computational complexity (G-FLOPs) and establish its correlation with time-to-predict performance. Latency reduction in real-life practical experiments was tested by adapting two leading-edge computational architectures, modern GPU and TPU.…”

Section: Introductionmentioning

confidence: 99%

Deep learning-based object recognition in multispectral satellite imagery for real-time applications

Gudzius

Kurasova

Darulis

et al. 2021

Machine Vision and Applications

View full text Add to dashboard Cite

Satellite imagery is changing the way we understand and predict economic activity in the world. Advancements in satellite hardware and low-cost rocket launches have enabled near-real-time, high-resolution images covering the entire Earth. It is too labour-intensive, time-consuming and expensive for human annotators to analyse petabytes of satellite imagery manually. Current computer vision research exploring this problem still lack accuracy and prediction speed, both significantly important metrics for latency-sensitive automatized industrial applications. Here we address both of these challenges by proposing a set of improvements to the object recognition model design, training and complexity regularisation, applicable to a range of neural networks. Furthermore, we propose a fully convolutional neural network (FCN) architecture optimised for accurate and accelerated object recognition in multispectral satellite imagery. We show that our FCN exceeds human-level performance with state-of-the-art 97.67% accuracy over multiple sensors, it is able to generalize across dispersed scenery and outperforms other proposed methods to date. Its computationally light architecture delivers a fivefold improvement in training time and a rapid prediction, essential to real-time applications. To illustrate practical model effectiveness, we analyse it in algorithmic trading environment. Additionally, we publish a proprietary annotated satellite imagery dataset for further development in this research field. Our findings can be readily implemented for other real-time applications too.

show abstract

Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps

Cited by 37 publications

References 15 publications

A Recurrent Deep Network for Estimating the Pose of Real Indoor Images from Synthetic Image Sequences

A Recurrent Deep Network for Estimating the Pose of Real Indoor Images from Synthetic Image Sequences

Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

Deep learning-based object recognition in multispectral satellite imagery for real-time applications

Contact Info

Product

Resources

About