CAMEL: An Adaptive Camera With Embedded Machine Learning-Based Sensor Parameter Control

Mudassar, Burhan Ahmad; Saha, Priyabrata; Long, Yun; Amir, Mohammad Faisal; Gebhardt, Evan; Na, Taesik; Ko, Jong Hwan; Wolf, Marilyn; Mukhopadhyay, Saibal

doi:10.1109/jetcas.2019.2935207

Cited by 14 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saha et al propose a Reinforcement Learning (RL) based controller for multispectral fusion [19]. Mudassar et al propose spatio-temporal resolution control of individual pixels in a digital pixel sensor (DPS) using the output of object detection and action detection tasks [4]. Mukherjee et al propose a cross-layer control scheme for tuning the pixel depth of ROI and non-ROI pixels [20].…”

Section: Task-driven Feedbackmentioning

confidence: 99%

“…We characterize the uncertainty of a closed-loop imaging system with embedded taskdriven feedback [4]. The system consists of a processing-in-memory acceleration tier and a digital pixel sensor array (DPS) in a 3D-stacked topology as shown in Figure 6.…”

Section: Confidence-driven Feedback Control 41 System Architecturementioning

confidence: 99%

“…A logic layer at the bottom of the stack performs real-time task processing to realize the per-pixel control. The confidence-driven feedback control from [4] is used. We call this confidence-based as it solely relies on the confidence of the detections to determine whether they should be used for feedback.…”

Section: Confidence-driven Feedback Control 41 System Architecturementioning

confidence: 99%

“…A critical component of data acquisition in active sensors [1][2][3][4] is the controller that modules the output of the sensor. As Artificial Intelligence (AI) and Machine Learning (ML) components become ubiquitous, they are increasingly being used as part of the controller [2][3][4]. These design decisions have ramifications for the downstream systems that consume and act on this sensor data.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we pick a closed-loop imager with a task DNN embedded in the control loop [4] and study it from the lens of uncertainty estimation. We examine a number of tasks including object detection, object tracking, and action detection.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Task-Driven Feedback Imager with Uncertainty Driven Hybrid Control

Mudassar

Saha

Wolf

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Deep Neural Network (DNN) systems tend to produce overconfident or uncalibrated outputs. This poses problems for active sensor systems that have a DNN module as the main feedback controller. In this paper, we study a closed-loop feedback smart camera from the lens of uncertainty estimation. The uncertainty of the task output is used to characterize and facilitate the feedback operation. The DNN uncertainty in the feedback system is estimated and characterized using both sampling and non-sampling based methods. In addition, we propose a closed-loop control that incorporates uncertainty information when providing feedback. We show two modes of control, one that prioritizes false positives and one that prioritizes false negatives, and a hybrid approach combining the two. We apply the uncertainty-driven control to the tasks of object detection, object tracking, and action detection. The hybrid system improves object detection and tracking accuracy on the CAMEL dataset by 1.1% each respectively. For the action detection task, the hybrid approach improves accuracy by 1.4%.

show abstract

Section: Task-driven Feedbackmentioning

confidence: 99%

Section: Confidence-driven Feedback Control 41 System Architecturementioning

confidence: 99%

Section: Confidence-driven Feedback Control 41 System Architecturementioning

confidence: 99%