Neural coding, including encoding and decoding, 1 is one of the key problems in neuroscience for understanding 2 how the brain uses neural signals to relate sensory perception 3 and motor behaviors with neural systems. However, most of 4 the existed studies only aim at dealing with the analogy signal 5 of neural systems, while lacking a unique feature of biological 6 neurons, termed spike, which is the fundamental information 7 unit for neural computation as well as a building block for 8 brain-machine interface. Aiming at these limitations, we pro-9 pose a transcoding framework to encode multi-modal sensory 10 information into neural spikes, then reconstruct stimuli from 11 spikes. Sensory information can be compressed into 10% in 12 terms of neural spikes, yet re-extract 100% of information 13 by reconstruction. Our framework can not only feasibly and 14 accurately reconstruct dynamical visual and auditory scenes, 15 but also rebuild the stimulus patterns from functional magnetic 16 resonance imaging brain activities. Importantly, it has a superb 17 ability of noise-immunity for various types of artificial noises 18 and background signals. The proposed framework provides 19 efficient ways to perform multimodal feature representation and 20 reconstruction in a high-throughput fashion, with potential usage
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.