A Novel Approach of Dynamic Vision Reconstruction from fMRI Profiles Using Siamese Conditional Generative Adversarial Network

Abstract

This paper aims to improve the quality of reconstructed visual stimuli and reduce the computational complexity of the visual stimuli reconstruction processes in the form of functional Magnetic Resonance Imaging (fMRI) profiles. The preceding work envisions the non-cognitive contents of brain activity vain to integrate visual data of diverse hierarchical levels. Existing approaches such as Deep Canonically Correlated Auto Encoder detect the significant challenges of reconstructing visual stimuli from brain activity: fMRI noise, large dimensionality of a limited number of data instances, and complex structure of visual stimuli. In this activity, we will also analyze the scope for utilizing the spatiotemporal data to resolve the neural correlates of visual stimulus representations and reconstruct the resembling visual stimuli. The purpose of this work is to manipulate those suffering from developmental disabilities. A novel Siamese conditional Generative Adversarial Network (ScGAN) approach is proposed to resolve these significant issues. The key features of ScGAN are as follows: 1. Siamese Neural Network (SNN) is a dimensionality reduction approach that takes as visual stimulus information alloy component and its goals to discover each of them effectively. It shows the critical component of visual stimuli. 2. In a conditional Generative Adversarial Network, the labels portrayan expansion to a latent variable to better generate and discriminate visual stimuli. Experiments on four fMRI datasets prove that our technique can reconstruct visual stimuli precisely. The performance metrics are evaluated by Mean Squared Error (MSE), Accuracy, Pearson Correlation Coefficient (PCC), Losses, HIGHLIGHTS• We combined Siamese Neural Network with CGAN to create a SCGAN framework.• Visual perception reconstructed by Conditional Generative Adversarial Network (CGAN).• Reducing image loss, the linear version learns to be expecting the latent area out of Blood Oxygen Level Dependent.• Reconstruction on video fMRI dataset was objectively identifiable.