Exploitation of image statistics with sparse coding in the case of stereo vision

Ecke, Gerrit A.; Papp, Harald M.; Mallot, Hanspeter A.

doi:10.1016/j.neunet.2020.12.016

Cited by 9 publications

(4 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, the goodness-of-fit parameter ( R 2 ) provides an intuitive measure of how Gabor-like a given RF is. The fitting was carried out using an adapted version of the code available at https://uk.mathworks.com/matlabcentral/fileexchange/60700-fit2dgabor-data-options ( Ecke et al, 2021 ).…”

Section: Methodsmentioning

confidence: 99%

Sub-Optimality of the Early Visual System Explained Through Biologically Plausible Plasticity

2021

View full text Add to dashboard Cite

The early visual cortex is the site of crucial pre-processing for more complex, biologically relevant computations that drive perception and, ultimately, behaviour. This pre-processing is often studied under the assumption that neural populations are optimised for the most efficient (in terms of energy, information, spikes, etc.) representation of natural statistics. Normative models such as Independent Component Analysis (ICA) and Sparse Coding (SC) consider the phenomenon as a generative, minimisation problem which they assume the early cortical populations have evolved to solve. However, measurements in monkey and cat suggest that receptive fields (RFs) in the primary visual cortex are often noisy, blobby, and symmetrical, making them sub-optimal for operations such as edge-detection. We propose that this suboptimality occurs because the RFs do not emerge through a global minimisation of generative error, but through locally operating biological mechanisms such as spike-timing dependent plasticity (STDP). Using a network endowed with an abstract, rank-based STDP rule, we show that the shape and orientation tuning of the converged units are remarkably close to single-cell measurements in the macaque primary visual cortex. We quantify this similarity using physiological parameters (frequency-normalised spread vectors), information theoretic measures [Kullback–Leibler (KL) divergence and Gini index], as well as simulations of a typical electrophysiology experiment designed to estimate orientation tuning curves. Taken together, our results suggest that compared to purely generative schemes, process-based biophysical models may offer a better description of the suboptimality observed in the early visual cortex.

show abstract

Section: Methodsmentioning

confidence: 99%

Sub-Optimality of the Early Visual System Explained Through Biologically Plausible Plasticity

2021

View full text Add to dashboard Cite

show abstract

“…This feature helps decrease the power consumption in the perception and processing components, especially for a stationary scene. Thus, this spike representation encoding technique could provide efficient scene signals in an energy-efficient way, which is more suitable for the processing framework of retinal prostheses (Ecke, Papp and Mallot (2021)).…”

Section: Spike Representation Encoding Techniquementioning

confidence: 99%

SpikeSEE: An Energy-Efficient Dynamic Scenes Processing Framework for Retinal Prostheses

Fang¹,

Zou²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

Intelligent and low-power retinal prostheses are highly demanded in this era, where wearable and implantable devices are used for numerous healthcare applications. In this paper, we propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model to achieve intelligent processing and extreme low-power computation for retinal prostheses. The spike representation encoding technique could interpret dynamic scenes with sparse spike trains, decreasing the data volume. The SRNN model, inspired by the human retina's special structure and spike processing method, is adopted to predict the response of ganglion cells to dynamic scenes. Experimental results show that the Pearson correlation coefficient of the proposed SRNN model achieves 0.93, which outperforms the state-of-the-art processing framework for retinal prostheses. Thanks to the spike representation and SRNN processing, the model can extract visual features in a multiplication-free fashion. The framework achieves 12 times power reduction compared with the convolutional recurrent neural network (CRNN) processing-based framework. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption, which alleviates the precision and power issues of retinal prostheses and provides a potential solution for wearable or implantable prostheses.

show abstract

“…Since 2010, gesture recognition and 3D visual recognition based on optical principles and visual recognition algorithms have been applied to the game console market. The humanmachine interface for gesture-based control previously used in controllers has gradually been replaced with freehand technology in a new generation of devices, such as those based on timeof-flight, (17) stereo vision, (18) and structured light. (19) In 2015, Intel Corporation commercialized the RealSense module and introduced it to notebook computers.…”

Section: Operation Through Gesturesmentioning

confidence: 99%

Development of Smart Home Gesture-based Control System

Wang¹,

Lai²,

Jhuang³

et al. 2021

Sensors and Materials

View full text Add to dashboard Cite

In recent years, smart speakers have been widely used to control related smart home appliances. For people with speech disabilities, smart speakers cannot be used to perform related voice control tasks. To help people with speech difficulties and effectively control related smart facilities, we use the Internet of Things to develop a smart home gesture-based control system. The prototype of the system is established by means of sensors and a control panel system. When a sensor detects a control gesture command, it can send related control signals to a controlled smart device. The controlled devices used in this study include smart home appliances such as doors and windows, air conditioners, curtains, dehumidifiers, and air purifiers. The system functions include the opening/closing of doors, windows, and curtains; the activation/ deactivation of heating and cooling systems, dehumidifiers, and air purifiers; temperature and humidity adjustment; and fan operation. With the development of this system, it is possible for people with language disabilities and elderly people to control the operation of various smart devices with gestures. It can also solve the inconvenience of needing to look for remote controls for various facilities. Moreover, it can save precious time and provide a safe, convenient, and smart home living environment for people with speech disabilities.

show abstract

Exploitation of image statistics with sparse coding in the case of stereo vision

Cited by 9 publications

References 76 publications

Sub-Optimality of the Early Visual System Explained Through Biologically Plausible Plasticity

Sub-Optimality of the Early Visual System Explained Through Biologically Plausible Plasticity

SpikeSEE: An Energy-Efficient Dynamic Scenes Processing Framework for Retinal Prostheses

Development of Smart Home Gesture-based Control System

Contact Info

Product

Resources

About