Locally-Connected, Irregular Deep Neural Networks for Biomimetic Active Vision in a Simulated Human

Nakada, Masaki; Chen, Honglin; Lakshmipathy, Arjun; Terzopoulos, Demetri

doi:10.1109/icpr48806.2021.9412771

Cited by 3 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because visual aberrations play significant role in vision and affect each individual, incorporating them into the simulation pipeline is necessary to achieve the desired results. Therefore, Nakada et al [104,105] utilized a ray-tracing approach for the optical simulation of vision. With the advancements and availability of a wide range of vision simulation approaches, we believe that this area could greatly benefit from future experiments with other types of algorithms.…”

Section: Training Virtual Humansmentioning

confidence: 99%

Rendering algorithms for aberrated human vision simulation

Csoba

Kunkli

2023

Vis. Comput. Ind. Biomed. Art

View full text Add to dashboard Cite

Vision-simulated imagery―the process of generating images that mimic the human visual system―is a valuable tool with a wide spectrum of possible applications, including visual acuity measurements, personalized planning of corrective lenses and surgeries, vision-correcting displays, vision-related hardware development, and extended reality discomfort reduction. A critical property of human vision is that it is imperfect because of the highly influential wavefront aberrations that vary from person to person. This study provides an overview of the existing computational image generation techniques that properly simulate human vision in the presence of wavefront aberrations. These algorithms typically apply ray tracing with a detailed description of the simulated eye or utilize the point-spread function of the eye to perform convolution on the input image. Based on the description of the vision simulation techniques, several of their characteristic features have been evaluated and some potential application areas and research directions have been outlined.

show abstract

Section: Training Virtual Humansmentioning

confidence: 99%

Rendering algorithms for aberrated human vision simulation

Csoba

Kunkli

2023

Vis. Comput. Ind. Biomed. Art

View full text Add to dashboard Cite

show abstract

Biomimetic Oculomotor Control with Spiking Neural Networks

Saquib

Terzopoulos

2022

Advances in Visual Computing

View full text Add to dashboard Cite

Biomimetic oculomotor control with spiking neural networks

Saquib,

Terzopoulos

2023

Machine Vision and Applications

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are comprised of artificial neurons that, like their biological counterparts, communicate via electrical spikes. SNNs have been hailed as the next wave of deep learning as they promise low latency and low-power consumption when run on neuromorphic hardware. Current deep neural network models for computer vision often require power-hungry GPUs to train and run, making them great candidates to replace with SNNs. We develop and train a biomimetic, SNN-driven, neuromuscular oculomotor controller for a realistic biomechanical model of the human eye. Inspired by the ON and OFF bipolar cells of the retina, we use event-based data flow in the SNN to direct the necessary extraocular muscle-driven eye movements. We train our SNN models from scratch, using modified deep learning techniques. Classification tasks are straightforward to implement with SNNs and have received the most research attention, but visual tracking is a regression task. We use surrogate gradients and introduce a linear layer to convert membrane voltages from the final spiking layer into the desired outputs. Our SNN foveation network enhances the biomimetic properties of the virtual eye model and enables it to perform reliable visual tracking. Overall, with event-based data processed by an SNN, our oculomotor controller successfully tracks a visual target while activating 87.3% fewer neurons than a conventional neural network.

show abstract

Locally-Connected, Irregular Deep Neural Networks for Biomimetic Active Vision in a Simulated Human

Cited by 3 publications

References 14 publications

Rendering algorithms for aberrated human vision simulation

Rendering algorithms for aberrated human vision simulation

Biomimetic Oculomotor Control with Spiking Neural Networks

Biomimetic oculomotor control with spiking neural networks

Contact Info

Product

Resources

About