21.3 A 5.69mm<sup>2</sup> 0.98nJ/Pixel Image-Processing SoC with 24b High-Dynamic-Range and Multiple Sensor Format Support for Automotive Applications

Chen, Ju; Liu, Tsu-Ming; Chang, Yung-Chang; Wang, Chih–Ming; Tsai, Chang-Hung; Chen, Ying-Jui; Wu, Tong; Lin, Hue-Min; Chou, Han-Liang; Chen, Abrams; Wang, Andy-HB; Gu, Wc; Hsieh, Wayne; Chang, J. R.; Liao, Shou-Chun; Ho, CT; Chu, Larry K.; Wei, Sokonisa; Ch, Wang; Jou, Kevin

doi:10.1109/isscc19947.2020.9063063

Cited by 2 publications

(1 citation statement)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This number is expected to decrease significantly given the large community working on CNN optimizations across the software-algorithm-hardware stack [22]. Depending on their specs, image sensors consume about 0.2-2 nJ/pixel [12], while high-end ISP macros consume 1-2 nJ/pixel [14] in the latest CMOS technology. While skipping ISP stages can reduce the latter significantly, we show that the benefits of a log-gradient approach is not merely limited to ISP energy savings but extends to potential improvements in robustness and CNN compressibility.…”

Section: Introductionmentioning

confidence: 99%

Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images

Lu¹,

Murmann²

2022

Preprint

View full text Add to dashboard Cite

This paper studies the merits of applying log-gradient input images to convolutional neural networks (CNNs) for tinyML computer vision (CV). We show that log gradients enable: (i) aggressive 1.5bit quantization of first-layer inputs, (ii) potential CNN resource reductions, and (iii) inherent robustness to illumination changes (1.7% accuracy loss across 2 −5 • • • 2 3 brightness variation vs. up to 10% for JPEG). We establish these results using the PASCAL RAW image data set and through a combination of experiments using neural architecture search and a fixed three-layer network. The latter reveal that training on log-gradient images leads to higher filter similarity, making the CNN more prunable. The combined benefits of aggressive first-layer quantization, CNN resource reductions, and operation without tight exposure control and image signal processing (ISP) are helpful for pushing tinyML CV toward its ultimate efficiency limits.

show abstract

Section: Introductionmentioning

confidence: 99%

Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images

Lu¹,

Murmann²

2022

Preprint

View full text Add to dashboard Cite

show abstract

Enhancing the Energy Efficiency and Robustness of tinyML Computer Vision Using Coarsely-quantized Log-gradient Input Images

Murmann

2024

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

This paper studies the merits of applying log-gradient input images to convolutional neural networks (CNNs) for tinyML computer vision (CV). We show that log gradients enable: (i) aggressive 1-bit quantization of first-layer inputs, (ii) potential CNN resource reductions, (iii) inherent insensitivity to illumination changes (1.7% accuracy loss across 2 − 5 ⋅⋅⋅2 3 brightness variation vs. up to 10% for JPEG), and (iv) robustness to adversarial attacks (>10% higher accuracy than JPEG-trained models). We establish these results using the PASCAL RAW image data set and through a combination of experiments using quantization threshold search, neural architecture search, and a fixed three-layer network. The latter reveal that training on log-gradient images leads to higher filter similarity, making the CNN more prunable. The combined benefits of aggressive first-layer quantization, CNN resource reductions, and operation without tight exposure control and image signal processing (ISP) are helpful for pushing tinyML CV toward its ultimate efficiency limits.

show abstract

21.3 A 5.69mm² 0.98nJ/Pixel Image-Processing SoC with 24b High-Dynamic-Range and Multiple Sensor Format Support for Automotive Applications

Cited by 2 publications

References 3 publications

Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images

Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images

Enhancing the Energy Efficiency and Robustness of tinyML Computer Vision Using Coarsely-quantized Log-gradient Input Images

Contact Info

Product

Resources

About

21.3 A 5.69mm2 0.98nJ/Pixel Image-Processing SoC with 24b High-Dynamic-Range and Multiple Sensor Format Support for Automotive Applications

Cited by 2 publications

References 3 publications

Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images

Improving the Energy Efficiency and Robustness of tinyML Computer Vision using Log-Gradient Input Images

Enhancing the Energy Efficiency and Robustness of tinyML Computer Vision Using Coarsely-quantized Log-gradient Input Images

Contact Info

Product

Resources

About

21.3 A 5.69mm² 0.98nJ/Pixel Image-Processing SoC with 24b High-Dynamic-Range and Multiple Sensor Format Support for Automotive Applications