No abstract

Continuous mobile vision is limited by the inability to efficiently capture image frames and process vision features. This is largely due to the energy burden of analog readout circuitry, data traffic, and intensive computation. To promote efficiency, we shift early vision processing into the analog domain. This results in RedEye, an analog convolutional image sensor that performs layers of a convolutional neural network in the analog domain before quantization. We design RedEye to mitigate analog design complexity, using a modular column-parallel design to promote physical design reuse and algorithmic cyclic reuse. RedEye uses programmable mechanisms to admit noise for tunable energy reduction. Compared to conventional systems, RedEye reports an 85% reduction in sensor energy, 73% reduction in cloudlet-based system energy, and a 45% reduction in computation-based system energy.

Current mobile imaging pipelines, provisioned for high quality photography, are ill-suited for wearable vision analytics, due to their high power consumption and privacy concerns, as exemplified by the slow adoption of wearables, such as Google Glass. Rather than constructing incremental improvements, we believe it is necessary to completely redesign a dedicated imaging pipeline for vision analytics.Toward this goal, we study a novel imaging pipeline, revolving around an in-imager analog vision processor that exports a low bandwidth irreversibly encoded signal, generating vision features before analog-to-digital conversion. To produce this signal at low power, we introduce energy-scaling mechanisms into the imager's analog frontend to produce the encoded signal. We use these mechanisms to generate a low-power signal that cannot be used to reconstruct the image, yet suffices as input for vision analytics. This imaging pipeline design will simultaneously achieve privacy and efficiency for continuous mobile vision tasks. The VisionTo date, imagers have pushed into ever higher resolutions and frame rates. Due to the high data rate and processing bandwidth, operating on the human-readable image incurs high energy in the sensor itself [6] and throughout the device as data is sent to memory and operated upon by the CPU. In total, this consumes a significant amount of energy. For example, simply capturing and reading camera data on a Google Glass consumes 2.5 watts, draining its 2100 mAh battery in less than one hour and raising the surface temperature by 28 degrees Celsius [7]. This energy use is much too high for always-on operation on a head-mounted or wrist-mounted wearable device.Furthermore, the public has mounted privacy concerns with wearable cameras, worried about activities being recorded without consent. Such privacy concerns can get worse under security leaks in operating systems, allowing hackers access to system resources, including image capture previews [1]. This raises a daunting barrier to the adoption of continuous mobile vision devices. This paper presents our motivation and early insights toward solving both the efficiency and privacy challenges by rethinking the imaging pipeline. In particular, we argue that certain image processing stages should happen before the analog-to-digital conversion is completed.By converting the image signal into a feature space, we can reduce the bandwidth of exported data and the footprint of the intermediate digital data on the system. Moreover, analog operations are significantly more efficient than their digital counterparts; summing analog current can be done over a single wire, while adding two 16-bit values requires hundreds of CMOS transistors in the digital domain. Keeping operations in the analog domain can also defray the invasive stigma of wearable vision by ensuring that the imaging pipeline does not expose private image data to the user, or even to the operating system.To produce such a signal, we propose an in-imager analog vision processor that co...