Tactile sensory coding and learning with bio-inspired optoelectronic spiking afferent nerves

Abstract

The integration and cooperation of mechanoreceptors, neurons and synapses in somatosensory systems enable humans to efficiently sense and process tactile information. Inspired by biological somatosensory systems, we report an optoelectronic spiking afferent nerve with neural coding, perceptual learning and memorizing capabilities to mimic tactile sensing and processing. Our system senses pressure by MXene-based sensors, converts pressure information to light pulses by coupling light-emitting diodes to analog-to-digital circuits, then integrates light pulses using a synaptic photomemristor. With neural coding, our spiking nerve is capable of not only detecting simultaneous pressure inputs, but also recognizing Morse code, braille, and object movement. Furthermore, with dimensionality-reduced feature extraction and learning, our system can recognize and memorize handwritten alphabets and words, providing a promising approach towards e-skin, neurorobotics and human-machine interaction technologies.

Artificially Intelligent Tactile Ferroelectric Skin

“…Thus, it is essential to develop and design an integrated sensory neural network system that can employ a nonvolatile synaptic array that is connected to tactile sensors. Considering this, a variety of integrated forms that employ synaptic devices, such as memristors [12][13][14] and field-effect transistor memories, [15][16][17] combined with independent tactile sensors have been suggested and demonstrated.…”

Neuromorphic Engineering for Hardware Computational Acceleration and Biomimetic Perception Motion Integration

“…In recent years, various emerging material engineering‐based memristive neuromorphic systems that integrate mechanical stress sensing, signal conversion, transmission, and processing modules for tactile perception have been extensively studied, including but not limited to inorganic, [ 195 ] organic, [ 196 ] self‐energizing, [ 44 ] and stretchable rubber‐like materials. [ 40 ] In particular, a proof‐of‐concept tactile sensing platform with mechanical sensing, neuromorphic coding, learning, and memory capabilities through optical communication was reported, [ 193 ] which integrates MXene pressure receptor, stress–light conversion module, and oxide photoelectric memristor to convert mechanical information into optical signals for transmission and processing, as shown in Figure 18d. It is noteworthy that optical communication enables multichannel signal input, which more realistically simulates multiple synaptic connections and AP integration between axons and dendrites.…”

Defect Engineering in Ambipolar Layered Materials for Mode‐Regulable Nociceptor

“…Because of the complexity of biological system, most reported works on artificial nociceptors using memristors or transistors focused on their inherent zero adaptation characteristics, while little consideration was dedicated to exploiting the extra adaptive mode functions. [54][55][56] To improve the adaptivity of the nociceptor based on the most mechnoreceptors prototype to handle a diverse range of real-world risks, we attempted to integrate the adaptation/no adaptation characteristics into one artificial machine using our PdSe 2 ambipolar transistor with photo-induced plasticity. In detail, the signal detected from external environments by the nociceptor is represented by the input laser stimulus; the extracted PSC weight of the PdSe 2 channel is regarded as its action potential in the sensory system.…”