A biologically inspired controller for fast eye movements

Lesmana, Martin; Pai, Dinesh K.

doi:10.1109/icra.2011.5980432

Cited by 5 publications

(2 citation statements)

References 34 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In robotics, Lesmana and Pai [2011] and Lesmana et al [2014] implemented a robotic oculomotor pulse-step controller that learns an internal model of the eye plant from measurements and produces realistic eye movements. Other robotic eye models and controllers are cited in their papers.…”

Section: Related Workmentioning

confidence: 99%

Biomimetic eye modeling & deep neuromuscular oculomotor control

et al. 2019

View full text Add to dashboard Cite

We present a novel, biomimetic model of the eye for realistic virtual human animation. We also introduce a deep learning approach to oculomotor control that is compatible with our biomechanical eye model. Our eye model consists of the following functional components: (i) submodels of the 6 extraocular muscles that actuate realistic eye movements, (ii) an iris submodel, actuated by pupillary muscles, that accommodates to incoming light intensity, (iii) a corneal submodel and a deformable, ciliary-muscle-actuated lens submodel, which refract incoming light rays for focal accommodation, and (iv) a retina with a multitude of photoreceptors arranged in a biomimetic, foveated distribution. The light intensity captured by the photoreceptors is computed using ray tracing from the photoreceptor positions through the finite aperture pupil into the 3D virtual environment, and the visual information from the retina is output via an optic nerve vector. Our oculomotor control system includes a foveation controller implemented as a locally-connected, irregular Deep Neural Network (DNN), or "LiNet", that conforms to the nonuniform retinal photoreceptor distribution, and a neuromuscular motor controller implemented as a fully-connected DNN, plus auxiliary Shallow Neural Networks (SNNs) that control the accommodation of the pupil and lens. The DNNs are trained offline through deep learning from data synthesized by the eye model itself. Once trained, the oculomotor control system operates robustly and efficiently online. It innervates the intraocular muscles to perform illumination and focal accommodation and the extraocular muscles to produce natural eye movements in order to foveate and pursue moving visual targets. We additionally demonstrate the operation of our eye model (binocularly) within our recently introduced sensorimotor control framework involving an anatomically-accurate biomechanical human musculoskeletal model.

show abstract

Section: Related Workmentioning

confidence: 99%

Biomimetic eye modeling & deep neuromuscular oculomotor control

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Various camera positioning devices inspired by the human ocular system have been designed. Several were designed to be driven by traditional servo motors with rigid links (Lesmana and Pai, 2011;Song and Zhang, 2012;Lee et al, 2013). These mechanisms successfully generated a fast motion that is comparable with saccade but have little in common with biological muscle actuation.…”

Section: Introductionmentioning

confidence: 99%

Dynamics-based motion de-blurring for a PZT-driven, compliant camera orientation mechanism

Kim

Ueda

2015

The International Journal of Robotics Research

View full text Add to dashboard Cite

This paper proposes a method for removing motion blur from images captured by a fast-moving robot eye. Existing image techniques focused on recovering blurry images due to camera shake with long exposure time. In addition, previous studies relied solely on properties of the images or used external sensors to estimate a blur kernel, or point spread function (PSF). This paper focuses on estimating a latent image from the blur images taken by the robotic camera orientation system. A PZT-driven, compliant camera orientation system was employed to demonstrate the effectiveness of this approach. Discrete switching commands were given to the robotic system to create a rapid point-to-point motion while suppressing the vibration with a faster response. The blurry images were obtained when the robotic system created a rapid point-topoint motion, like human saccadic motion. This paper proposes a method for estimating the PSF in knowledge of system dynamics and input commands, resulting in a faster estimation. The proposed method was investigated under various motion conditions using the single-degree-of-freedom camera orientation system to verify the effectiveness and was compared with other approaches quantitatively and qualitatively. The experiment results show that overall the performance metric of the proposed method was 27.77% better than conventional methods. The computation time of the proposed method was 50 times faster than that of conventional methods.

show abstract