Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses

Lozano, Antonio; Suárez, Juan Sebastián; Soto-Sánchez, Cristina; Garrigós, Javier; Martínez-Álvarez, J. Javier; Vicente, José Manuel Ferrández; Fernández, Eduardo

doi:10.1142/s0129065720500458

Cited by 41 publications

(21 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The visual encoder system has been described elsewhere (12,13,(55)(56)(57)(58)(59). In brief it consisted of a video camera attached to an eyeglass frame for image acquisition using head scanning, and custom hardware/software which performs a real-time analysis of the light patterns that are received by the light sensors in the camera and a multichannel spatio-temporal filtering of the visual information to extract and enhance the most relevant features of the scene.…”

Section: Bio-inspired Retinal Encodermentioning

confidence: 99%

Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex

Fernández¹,

Alfaro²,

Soto-Sánchez³

et al. 2021

Journal of Clinical Investigation

Self Cite

111

133

View full text Add to dashboard Cite

BACKGROUND.A long-held dream of scientists is to transfer information directly to the visual cortex of blind individuals, thereby restoring a rudimentary form of sight. However, no clinically available cortical visual prosthesis yet exists. METHODS.We implanted an intracortical microelectrode array consisting of 96 electrodes in the visual cortex of a 57-year-old person with complete blindness for a sixmonth period. We measured thresholds and the characteristics of the visual percepts elicited by intracortical microstimulation. RESULTS.Implantation and subsequent explantation of intracortical microelectrodes were carried out without complications. The mean stimulation threshold for single electrodes was 66.8  36.5 A. We consistently obtained high-quality recordings from visually deprived neurons and the stimulation parameters remained stable over time. Simultaneous stimulation via multiple electrodes were associated with a significant reduction in thresholds (p<0.001, ANOVA test) and evoked discriminable phosphene percepts, allowing the blind participant to identify some letters and recognize object boundaries. CONCLUSIONS.Our results demonstrate the safety and efficacy of chronic intracortical microstimulation via a large number of electrodes in human visual cortex, showing its high potential for restoring functional vision in the blind. TRIAL REGISTRATION. ClinicalTrials.gov identifier NCT02983370.

show abstract

Section: Bio-inspired Retinal Encodermentioning

confidence: 99%

Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex

Fernández¹,

Alfaro²,

Soto-Sánchez³

et al. 2021

Journal of Clinical Investigation

Self Cite

111

133

View full text Add to dashboard Cite

show abstract

“…Recently, machine-learning algorithms, in particular deep neural networks (Díaz-Vico et al, 2020;Lara-Benıteza et al, 2020;Lozano et al, 2020;Yang et al, 2019), have been used for damage detection of structures (Ni et al, 2020;Wu et al, 2019). Abdeljaber et al (2018)…”

Section: Vibration-based Structural Health Monitoringmentioning

confidence: 99%

Integrating structural control, health monitoring, and energy harvesting for smart cities

2021

View full text Add to dashboard Cite

Cities that are adopting innovative and technology-driven solutions to improve the city's efficiency are considered smart cities. With the increased attention on smart cities with self-driving vehicles, drones, and robots, designing smart infrastructure is only a natural extension. Smart infrastructures aim to self-diagnose, self-power, selfadapt, and self-heal during normal and extreme operating conditions. Structural vibration control (SVC) and structural health monitoring (SHM) technologies, in particular, are expected to play pivotal roles in the development of modern smart and resilient structures. SVC methodologies intend to provide supplemental damping and reduce the structural dynamic responses during normal and extreme events. SHM methodologies offer valuable information about the structure's condition that is useful for maintenance purposes and rapid damage detection in post-hazard events. The collapse of the 12-story Champlain Towers South, a beachfront condominium in the Miami suburb of Surfside, Florida, could have been known in advance with an embedded SHM technology. More recently, the integrated structural control and health monitoring (ISCHM) systems have shown promise in the development of smart cities of the future. The integrated architecture incorporates the control and health monitoring components as complementary technologies and simultaneously takes advantage of both technologies. This article provides a state-of-the-art review of ISCHM ideas and systems. It presents recent significant developments in structural control, SHM, and energy harvesting that are paving the way towards the advent of integrated ISCHM systems, including damage-tolerant control systems. This article also identifies future promising research areas for designing the next generation of autonomous ISCHM systems for smart cities.

show abstract

“…The translation of complex visual input into a phosphene percept (which by definition is limited) requires an efficient reduction of information and selection of the mere essential visual features for a given task. This can be achieved with the use of traditional computer vision approaches, such as edge detection ( Boyle, Maeder, & Boles, 2001 ; Dowling, Maeder, & Boles, 2004 ; Guo, Yang, & Gao, 2018 ), but deep neural network models have also gained increasing interest of prosthetic engineers (e.g., Sanchez-Garcia, Martinez-Cantin, & Guerrero, 2020 ; Han et al, 2021 ; Bollen et al, 2019 ; Bollen, van Wezel, van Gerven, & Güçlütürk, 2019 ; De Ruyter Van Steveninck, Güçlü, van Wezel, & Van Gerven, 2020 ; Lozano et al, 2020 ; Lozano et al, 2018 ). Various image processing approaches have been proposed for mobility in particular ( Barnes et al, 2011 ; Dagnelie et al, 2007 ; Dowling, Boles, & Maeder, 2006 ; Dowling, Maeder, & Boles, 2004 ; Feng & McCarthy, 2013 ; McCarthy et al, 2015 ; McCarthy, Feng, & Barnes, 2013 ; Parikh, Itti, Humayun, & Weiland, 2013 ; Srivastava, Troyk, & Dagnelie, 2009 ; van Rheede, Kennard, & Hicks, 2010 ; Vergnieux, Mace, & Jouffrais, 2014 ; Vergnieux, Macé, & Jouffrais, 2017 ; Zapf, Boon, Lovell, & Suaning, 2016 ).…”

Section: Introductionmentioning

confidence: 99%

Real-world indoor mobility with simulated prosthetic vision: The benefits and feasibility of contour-based scene simplification at different phosphene resolutions

et al. 2022

View full text Add to dashboard Cite

Neuroprosthetic implants are a promising technology for restoring some form of vision in people with visual impairments via electrical neurostimulation in the visual pathway. Although an artificially generated prosthetic percept is relatively limited compared with normal vision, it may provide some elementary perception of the surroundings, re-enabling daily living functionality. For mobility in particular, various studies have investigated the benefits of visual neuroprosthetics in a simulated prosthetic vision paradigm with varying outcomes. The previous literature suggests that scene simplification via image processing, and particularly contour extraction, may potentially improve the mobility performance in a virtual environment. In the current simulation study with sighted participants, we explore both the theoretically attainable benefits of strict scene simplification in an indoor environment by controlling the environmental complexity, as well as the practically achieved improvement with a deep learning-based surface boundary detection implementation compared with traditional edge detection. A simulated electrode resolution of 26 × 26 was found to provide sufficient information for mobility in a simple environment. Our results suggest that, for a lower number of implanted electrodes, the removal of background textures and within-surface gradients may be beneficial in theory. However, the deep learning-based implementation for surface boundary detection did not improve mobility performance in the current study. Furthermore, our findings indicate that, for a greater number of electrodes, the removal of within-surface gradients and background textures may deteriorate, rather than improve, mobility. Therefore, finding a balanced amount of scene simplification requires a careful tradeoff between informativity and interpretability that may depend on the number of implanted electrodes.

show abstract

Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses

Cited by 41 publications

References 58 publications

Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex

Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex

Integrating structural control, health monitoring, and energy harvesting for smart cities

Real-world indoor mobility with simulated prosthetic vision: The benefits and feasibility of contour-based scene simplification at different phosphene resolutions

Contact Info

Product

Resources

About