Active gaze stabilization

Lesmana, Martin; Landgren, Axel; Forssén, Per-Erik; Pai, Dinesh K.

doi:10.1145/2683483.2683565

Cited by 2 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In robotics, Lesmana and Pai [2011] and Lesmana et al [2014] implemented a robotic oculomotor pulse-step controller that learns an internal model of the eye plant from measurements and produces realistic eye movements. Other robotic eye models and controllers are cited in their papers.…”

Section: Related Workmentioning

confidence: 99%

Biomimetic eye modeling & deep neuromuscular oculomotor control

et al. 2019

View full text Add to dashboard Cite

We present a novel, biomimetic model of the eye for realistic virtual human animation. We also introduce a deep learning approach to oculomotor control that is compatible with our biomechanical eye model. Our eye model consists of the following functional components: (i) submodels of the 6 extraocular muscles that actuate realistic eye movements, (ii) an iris submodel, actuated by pupillary muscles, that accommodates to incoming light intensity, (iii) a corneal submodel and a deformable, ciliary-muscle-actuated lens submodel, which refract incoming light rays for focal accommodation, and (iv) a retina with a multitude of photoreceptors arranged in a biomimetic, foveated distribution. The light intensity captured by the photoreceptors is computed using ray tracing from the photoreceptor positions through the finite aperture pupil into the 3D virtual environment, and the visual information from the retina is output via an optic nerve vector. Our oculomotor control system includes a foveation controller implemented as a locally-connected, irregular Deep Neural Network (DNN), or "LiNet", that conforms to the nonuniform retinal photoreceptor distribution, and a neuromuscular motor controller implemented as a fully-connected DNN, plus auxiliary Shallow Neural Networks (SNNs) that control the accommodation of the pupil and lens. The DNNs are trained offline through deep learning from data synthesized by the eye model itself. Once trained, the oculomotor control system operates robustly and efficiently online. It innervates the intraocular muscles to perform illumination and focal accommodation and the extraocular muscles to produce natural eye movements in order to foveate and pursue moving visual targets. We additionally demonstrate the operation of our eye model (binocularly) within our recently introduced sensorimotor control framework involving an anatomically-accurate biomechanical human musculoskeletal model.

show abstract

Section: Related Workmentioning

confidence: 99%

Biomimetic eye modeling & deep neuromuscular oculomotor control

et al. 2019

View full text Add to dashboard Cite

show abstract

Deep learning of biomimetic sensorimotor control for biomechanical human animation

et al. 2018

View full text Add to dashboard Cite

We introduce a biomimetic framework for human sensorimotor control, which features a biomechanically simulated human musculoskeletal model actuated by numerous muscles, with eyes whose retinas have nonuniformly distributed photoreceptors. The virtual human's sensorimotor control system comprises 20 trained deep neural networks (DNNs), half constituting the neuromuscular motor subsystem, while the other half compose the visual sensory subsystem. Directly from the photoreceptor responses, 2 vision DNNs drive eye and head movements, while 8 vision DNNs extract visual information required to direct arm and leg actions. Ten DNNs achieve neuromuscular control---2 DNNs control the 216 neck muscles that actuate the cervicocephalic musculoskeletal complex to produce natural head movements, and 2 DNNs control each limb; i.e., the 29 muscles of each arm and 39 muscles of each leg. By synthesizing its own training data, our virtual human automatically learns efficient, online, active visuomotor control of its eyes, head, and limbs in order to perform nontrivial tasks involving the foveation and visual pursuit of target objects coupled with visually-guided limb-reaching actions to intercept the moving targets, as well as to carry out drawing and writing tasks.

show abstract

Active gaze stabilization

Cited by 2 publications

References 23 publications

Biomimetic eye modeling & deep neuromuscular oculomotor control

Biomimetic eye modeling & deep neuromuscular oculomotor control

Deep learning of biomimetic sensorimotor control for biomechanical human animation

Contact Info

Product

Resources

About