Multimode‐fused sensing in the somatosensory system helps people obtain comprehensive object properties and make accurate judgments. However, building such multisensory systems with conventional metal–oxide–semiconductor technology presents serious device integration and circuit complexity challenges. Here, a multimode‐fused spiking neuron (MFSN) with a compact structure to achieve human‐like multisensory perception is reported. The MFSN heterogeneously integrates a pressure sensor to process pressure and a NbOx‐based memristor to sense temperature. Using this MFSN, multisensory analog information can be fused into one spike train, showing excellent data compression and conversion capabilities. Moreover, both pressure and temperature information are distinguished from fused spikes by decoupling the output frequencies and amplitudes, supporting multimodal tactile perception. Then, a 3 × 3 MFSN array is fabricated, and the fused frequency patterns are fed into a spiking neural network for enhanced tactile pattern recognition. Finally, a larger MFSN array is simulated for classifying objects with different shapes, temperatures, and weights, validating the feasibility of the MFSNs for practical applications. The proof‐of‐concept MFSNs enable the building of multimodal sensory systems and contribute to the development of highly intelligent robotics.
Neuromorphic computing powered by spiking neural networks (SNN) provides a powerful and efficient information processing paradigm. To harvest the advantage of SNNs, compact and low‐power synapses that can reliably practice local learning rules are required, posing significant challenges to the conventional silicon‐based platform in terms of area‐ and energy‐efficiency, as well as computing throughput. Here, electrolyte‐gated transistors (EGTs) paired with transistors are employed to implement power‐efficient neuromorphic computing systems. The one‐transistor‐one‐EGT (1T1E) synapse not only alleviates the self‐discharging of EGT but also provides a flexible and efficient way to practice the important spike‐timing‐dependent plasticity learning rule. Based on that, an SNN with a temporal coding scheme is implemented for associative memory that can learn and recover images of handwritten digits with high robustness. Thanks to the temporal coding scheme and low operation current of EGTs, the energy‐efficiency of 1T1E‐based SNN is ≈30× lower than that of the prevalent rate coding scheme, and the peak performance is estimated to be 2 pJ/SOP (picojoule per synaptic operation) at the training phase and 80 TOPs−1 W−1 (tera operations per second per watt) at inference phase, respectively. These results pave the way for power‐efficient neuromorphic computing systems with wide applications for edge computing.
Conventional machine vision systems suffer from great data latency and energy consumption in cognitive tasks due to the separated vision sensors, memory units, and processors. In‐sensor computing based on optoelectronic synapses allows efficient computation by directly sensing and processing optical signals. Herein, an optoelectronic synapse based on Au/ZnO:N/IGZO/TiN structure is proposed. It shows uniform optical SET and electrical RESET behaviors, with various light‐tunable plasticity. Furthermore, a 4‐bit reservoir is experimentally implemented on the device, which is ideal to construct in‐sensor reservoir computing (RC) system. By converting spatiotemporal optical signals to higher dimensional feature space, in‐sensor RC has a great advantage in processing sequential visual information. Simulation results demonstrate that the in‐sensor RC system based on the proposed synapse achieves a considerable recognition accuracy (90.45%) for the MNIST dataset with very limited 36‐30‐10 perceptron network, and a 97.14% accuracy for human action classification from sequential vision data based on the Weizmann dataset. This work proves the low training cost and great efficiency for processing spatiotemporal and sequential optical signals, which may pave a new way for future machine vision applications.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.