Additive manufacturing (AM) is a type of advanced manufacturing process that enables fast prototyping to realize personalized products in complex shapes. However, quality defects existed in AM products can directly lead to significant failures in practice. Thus, various inspection techniques have been investigated to evaluate the quality of AM products, where X-ray computed tomography serves as one of the most accurate techniques to detect defects. Taking a selective laser melting process (SLM) as an example, voids can be detected by investigating CT images after the fabrication of products. However, limited by the sensor size and scanning speed issue, CT is difficult to be used for online (i.e., layer-wise) voids detection, monitoring, and process control to mitigate the defects. As an alternative, optical cameras can provide layer-wise images to support online voids detection. The intricate texture of the layer-wise image restricts the accuracy of void detection in AM products. Therefore, we propose a new method called pyramid ensemble convolutional neural network to efficiently detect voids and predict the texture of CT images by using layer-wise optical images. The proposed PECNN can efficiently extract informative features based on the ensemble of the multiscale feature-maps from optical images. Unlike deterministic ensemble strategies, this ensemble strategy is optimized by training a neural network in a data-driven manner to learn the fine-grained information from the extracted feature-maps. The merits of the proposed method are illustrated by both simulations and a real case study in SLM.