This article, proposes a reconfigurable quantum photonic convolutional layer (QPCL) based on the reconfigurable photonic gates. The QPCL is used in the classical photonic CNN, where, an array of reconfigurable photonic gates (RPG) are arranged in a systematic way. The designed reconfigurable photonic gate serves as a unit cell for quantum photonic operations such as beam splitting, rotation, displacement, squeezing, and cubic- phase shifting. The designed RPG provides the features namely broadband operation, low insertion loss and compact layout. The entangled states are created based on the normalized pixel value of the input image. The configuration of reconfigurable photonic gate is accomplished using electro-optic P-i-N carrier injection mechanism. As compared to Mach-Zehnder interferometer (MZI) based realization, the proposed silicon reconfigurable photonic gate provides scalable operation and compact footprint. The reconfigurable photonic gate is modeled using 2D finite element beam propagation method (FEBPM). Finally, a compact numerical model is developed which performs Gaussian based continuous-variable (CV) quantum photonic operations and are verified with Xanadu’s strawberryfields quantum photonic simulator and PennyLane deep learning framework. The optimized accuracy (loss) is obtained with the utilization of QPCL layer and the values are 0.7627 (0.9595), this optimum result is obtained using a single QPCL layer with an epoch number of 30. Finally, a comparative analysis is made between quantum CNN and classical photonic CNN, where the quantum CNN resulted in 6.553% high accuracy and 6.988% low loss compared to the classical photonic CNN.