Energy characterization and optimization of image sensing toward continuous mobile vision

LiKamWa, Robert; Priyantha, Bodhi; Philipose, Matthai; Lin, Zhouchen; Bahl, Paramvir

doi:10.1145/2462456.2464448

Cited by 153 publications

(92 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In comparison, we are interested in a wider range of possibly spatially varying edits such as stylization and detail enhancement, and we specifically study their latency and energy consumption when computed in the cloud. LiKamWa et al [2013] describe an approach to reduce the energy consumption when taking a picture with a mobile device. Our work is complementary and seeks to reduce the energy consumption and latency when editing photographs.…”

Section: Related Workmentioning

confidence: 99%

Transform recipes for efficient cloud photo enhancement

Gharbi¹,

Shih²,

Chaurasia³

et al. 2015

ACM Trans. Graph.

View full text Add to dashboard Cite

Figure 1: Cloud computing is often thought as an ideal solution to enable complex algorithms on mobile devices; by exploiting remote resources, one expects to reduce running time and energy consumption. However, this ignores the cost of transferring data over the network, the overhead of which often negates the benefits of the cloud, especially for data-heavy image processing applications. We introduce a new image processing pipeline that reduces the amount of transmitted data. The core of our approach is the transform recipe, a representation of the image transformation applied to a photograph that is compact and can be accurately estimated from an aggressively compressed input photograph. These properties make cloud computing more efficient in situations where transferring standard JPEG-compressed images would be too costly in terms of energy and/or time. In the example above, where we used an aggressive setting, our approach reduces the amount of transmitted data up to 80× compared to standard JPEG compressed images. It produces an output visually similar to the ground-truth result computed directly from the original uncompressed photograph. Photograph from the MIT 5k dataset [2011]. AbstractCloud image processing is often proposed as a solution to the limited computing power and battery life of mobile devices: it allows complex algorithms to run on powerful servers with virtually unlimited energy supply. Unfortunately, this overlooks the time and energy cost of uploading the input and downloading the output images. When transfer overhead is accounted for, processing images on a remote server becomes less attractive and many applications do not benefit from cloud offloading. We aim to change this in the case of image enhancements that preserve the overall content of an image. Our key insight is that, in this case, the server can compute and transmit a description of the transformation from input to output, which we call a transform recipe. At equivalent quality, our recipes are much more compact than JPEG images: this reduces the client's download. Furthermore, recipes can be computed from highly compressed inputs which significantly reduces the data uploaded to the server. The client reconstructs a high-fidelity approximation of the output by applying the recipe to its local high-quality input. We demonstrate our results on 168 images and 10 image processing applications, showing that our recipes form a compact representation for a diverse set of image filters. With an equivalent transmission budget, they provide higher-quality results than JPEG-compressed input/output images, with a gain of the order of 10 dB in many cases. We demonstrate the utility of recipes on a mobile phone by profiling the energy consumption and latency for both local and cloud computation: a transform recipe-based pipeline runs 2-4× faster and uses 2-7× less energy than local or naive cloud computation.

show abstract

Section: Related Workmentioning

confidence: 99%

Transform recipes for efficient cloud photo enhancement

Gharbi¹,

Shih²,

Chaurasia³

et al. 2015

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…The study on energy consumption and efficiency of mobile systems are important because the limitation of power supply and processors performance. [28] studies the balancing of energy, latency and accuracy on smartphones and [29] presents analytical and empirical characterisations of imaging sensors in embedded cameras in order to optimise their energy consumption. Other examples of smartphone utilties include security and authentication [30], [31], visualisation [32] and localisation [33].…”

Section: Related Workmentioning

confidence: 99%

Face recognition on smartphones via optimised Sparse Representation Classification

Shen

Yang

et al. 2014

IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks

View full text Add to dashboard Cite

Face recognition is an element of many smartphone apps, e.g. face unlocking, people tagging and games. Sparse Representation Classification (SRC) is a state-of-the-art face recognition algorithm, which has been shown to outperform many classical face recognition algorithms in OpenCV. The success of SRC is due to its use of 1 optimisation, which makes SRC robust to noise and occlusions. Since 1 optimisation is computationally intensive, SRC uses random projection matrices to reduce the dimension of the 1 problem. However, random projection matrices do not give consistent classification accuracy. In this paper, we propose a method to optimise the projection matrix for 1-based classification 1 . Our evaluations, based on publicly available databases and real experiment, show that face recognition based on the optimised projection matrix can be 5-17% more accurate than its random counterpart and OpenCV algorithms. Furthermore, the optimised projection matrix does not have to be re-calculated even if new faces are added to the training set. We implement the SRC with optimised projection matrix on Android smartphones and find that the computation of residuals in SRC is a severe bottleneck, taking up 85-90% of the computation time. To address this problem, we propose a method to compute the residuals approximately, which is 50 times faster but without sacrificing recognition accuracy. Lastly, we demonstrate the feasibility of our new algorithm by the implementation and evaluation of a new face unlocking app and show its robustness to variation to poses, facial expressions, lighting changes and occlusions.

show abstract

“…Image matching operates in a pipeline. First, discriminative points are extracted for each image, called keypoints [19,27,20,18]. The keypoints in each image can be viewed as visual "words" describing the image [17].…”

Section: Vision-guided Navigationmentioning

confidence: 99%

“…Travi-Navi may reduce the image sampling rate to save energy and benefit from the energy efficient mobile vision techniques [18]. When magnetic field distortions exhibit sufficient discriminative patterns, Travi-Navi may disable WiFi scans and weight particles solely with magnetic measurements.…”

Section: Energy Consumptionmentioning

confidence: 99%

“…GigaSight [31] proposes a framework for crowdsourcing videos that enables deep content-based search. LiKamWa et al [18] analytically characterize energy consumption of image and video capture and propose quality-energy tradeoff strategies. Travi-Navi augments indoor navigation with mobile vision, and avoids unnecessary power consumption and computation leveraging motion hints inferred from IMU sensors.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Travi-Navi

Zheng

Shen

et al. 2014

Proceedings of the 20th Annual International Conference on Mobile Computing and Networking

View full text Add to dashboard Cite

We present Travi-Navi -a vision-guided navigation system that enables a self-motivated user to easily bootstrap and deploy indoor navigation services, without comprehensive indoor localization systems or even the availability of floor maps. Travi-Navi records high quality images during the course of a guider's walk on the navigation paths, collects a rich set of sensor readings, and packs them into a navigation trace. The followers track the navigation trace, get prompt visual instructions and image tips, and receive alerts when they deviate from the correct paths. TraviNavi also finds the most efficient shortcuts whenever possible. We encounter and solve several challenges, including robust tracking, shortcut identification, and high quality image capture while walking. We implement Travi-Navi and conduct extensive experiments. The evaluation results show that Travi-Navi can track and navigate users with timely instructions, typically within a 4-step offset, and detect deviation events within 9 steps.

show abstract

Energy characterization and optimization of image sensing toward continuous mobile vision

Cited by 153 publications

References 29 publications

Transform recipes for efficient cloud photo enhancement

Transform recipes for efficient cloud photo enhancement

Face recognition on smartphones via optimised Sparse Representation Classification

Travi-Navi

Contact Info

Product

Resources

About