Efficient Neural Vision Systems Based on Convolutional Image Acquisition

Pad, Pedram; Narduzzi, Simon; Kundig, Clement; Türetken, Engin; Bigdeli, Siavash Arjomand; Dunbar, L. A.

doi:10.1109/cvpr42600.2020.01230

Cited by 17 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Experiment with an extensive optical convolution that uses logarithmic activation as an image sensor before feeding the image into a perceptron network with 1600 input vectors, three hidden layers with a total of 256 neurons, each with a rectified linear unit (ReLu) activation function, and one fully connected layer with linear activation for output units performed by [10]. From the experiments, the test results on the EMNIST Letters dataset obtained an accuracy of 93.65%.…”

Section: A Related Workmentioning

confidence: 99%

Handwritten Character Recognition using Deep Learning Algorithm with Machine Learning Classifier

Liman,

Josef,

Kusuma

2024

JOIV : Int. J. Inform. Visualization

View full text Add to dashboard Cite

Handwritten character recognition is a problem that has been worked on for many mainstream languages. Handwritten letter recognition has been proven to achieve promising results. Several studies using deep learning models have been conducted to achieve better accuracies. In this paper, the authors conducted two experiments on the EMNIST Letters dataset: Wavemix-Lite and CoAtNet. The Wavemix-Lite model uses Two-Dimensional Discrete Wavelet Transform Level 1 to reduce the parameters and speed up the runtime. The CoAtNet is a combined model of CNN and Visual Transformer where the image is broken down into fixed-size patches. The feature extraction part of the model is used to embed the input image into a feature vector. From those two models, the authors hooked the value of the features of the Global Average Pool layer using EMNIST Letters data. The features hooked from the training results of the two models, such as SVM, Random Forest, and XGBoost models, were used to train the machine learning classifier. The experiments conducted by the authors show that the best machine-learning model is the Random Forest, with 96.03% accuracy using the Wavemix-Lite model and 97.90% accuracy using the CoAtNet model. These results showcased the benefit of using a machine learning model for classifying image features that are extracted using a deep learning model.

show abstract

Section: A Related Workmentioning

confidence: 99%

Handwritten Character Recognition using Deep Learning Algorithm with Machine Learning Classifier

Liman,

Josef,

Kusuma

2024

JOIV : Int. J. Inform. Visualization

View full text Add to dashboard Cite

show abstract

“…Another study implements the first CNN layer in the optical domain using a controllable (grayscale) mask [131]. All filters (output channels) are displayed in the same plane, allowing the image sensor behind to capture all convolution results in parallel, forwarding them to the last layers implemented in the digital domain.…”

Section: Early Data Reductionmentioning

confidence: 99%

A Construction Kit for Efficient Low Power Neural Network Accelerator Designs

Jokic¹,

Azarkhish²,

Bonetti³

et al. 2022

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Implementing embedded neural network processing at the edge requires efficient hardware acceleration that combines high computational throughput with low power consumption. Driven by the rapid evolution of network architectures and their algorithmic features, accelerator designs are constantly being adapted to support the improved functionalities. Hardware designers can refer to a myriad of accelerator implementations in the literature to evaluate and compare hardware design choices. However, the sheer number of publications and their diverse optimization directions hinder an effective assessment. Existing surveys provide an overview of these works but are often limited to system-level and benchmark-specific performance metrics, making it difficult to quantitatively compare the individual effects of each utilized optimization technique. This complicates the evaluation of optimizations for new accelerator designs, slowing-down the research progress. In contrast to previous surveys, this work provides a quantitative overview of neural network accelerator optimization approaches that have been used in recent works and reports their individual effects on edge processing performance. The list of optimizations and their quantitative effects are presented as a construction kit, allowing to assess the design choices for each building block individually. Reported optimizations range from up to 10’000x memory savings to 33x energy reductions, providing chip designers an overview of design choices for implementing efficient low power neural network accelerators.

show abstract

“…These studies were all done using the phase-varying optical masks which are usable for monochromatic and coherent light. Pad et al [17] furthered this work by benefiting from an amplitude varying mask to implement a large optical convolution kernel which is able to work with polychromatic and incoherent light available in the natural scenes. They showed two orders of magnitude reduction in the computational cost while keeping the state-of-the-art accuracy.…”

Section: Hybrid Network and Optical Convolutionmentioning

confidence: 99%

“…Thus, the best way to preserve privacy in such tasks is to remove the sensitive data in the optical domain before reaching the image sensor. In doing this, we benefit from a relatively simple optical front-end proposed in [17] which performs the convolution of the scene with a pretrained amplitude varying mask in the optical domain. Figure 1 shows a schematic view of the privacy-preserving approach and Figure 2 shows its realization.…”

Section: Training An Optical Kernel For Preserving Privacymentioning

confidence: 99%

Privacy-Preserving Image Acquisition Using Trainable Optical Kernel

Sepehri,

Pad,

Frossard

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Preserving privacy is a growing concern in our society where sensors and in particular cameras are ubiquitous. In this work, for the first time, we propose a trainable image acquisition method that removes the sensitive identity revealing information in the optical domain before it reaches the image sensor. The method benefits from a trainable optical convolution kernel which transmits the desired information while filters out the sensitive information. As the sensitive information is suppressed before it reaches the image sensor, it does not enter the digital domain therefore is unretrievable by any sort of privacy attack. This is in contrast with the current digital privacy-preserving methods that are all vulnerable to direct access attack. Also, in contrast with the previous optical privacy-preserving methods that cannot be trained, our method is data-driven and optimized for the specific application at hand. Moreover, there is no additional computation, memory, or power burden on the acquisition system since this processing happens passively in the optical domain and can even be used together and on top of the fully digital privacy-preserving systems. The proposed approach is generic and adaptable to different digital neural networks, and desired and sensitive content pairs. We demonstrate our new method for several scenarios such as smile detection as the desired attribute while the gender is filtered out as the sensitive content. We trained the optical kernel in conjunction with two adversarial neural networks where the analysis network tries to detect the desired attribute and the adversarial network tries to detect the sensitive content. We show that this method is able to reduce 65.1% of sensitive content when it is selected to be the gender and as the kernel is optimized, it only causes 7.3% reduction of the desired information content. Moreover, we reconstruct the original faces using the deep image reconstruction method that confirms the ineffectiveness of reconstruction attacks to obtain the sensitive content.

show abstract

Efficient Neural Vision Systems Based on Convolutional Image Acquisition

Cited by 17 publications

References 27 publications

Handwritten Character Recognition using Deep Learning Algorithm with Machine Learning Classifier

Handwritten Character Recognition using Deep Learning Algorithm with Machine Learning Classifier

A Construction Kit for Efficient Low Power Neural Network Accelerator Designs

Privacy-Preserving Image Acquisition Using Trainable Optical Kernel

Contact Info

Product

Resources

About