Efficient Pix2Vox++ for 3D Cardiac Reconstruction from 2D Echo Views

Abstract

Accurate geometric quantification of the human heart is a key step in the diagnosis of numerous cardiac diseases, and in the management of cardiac patients. Ultrasound imaging is the primary modality for cardiac imaging, however acquisition requires high operator skill, and its interpretation and analysis is difficult due to artifacts. Reconstructing cardiac anatomy in 3D can enable discovery of new biomarkers and make imaging less dependent on operator expertise, however most ultrasound systems only have 2D imaging capabilities. We propose both a simple alteration to the Pix2Vox++ networks for a sizeable reduction in memory usage and computational complexity, and a pipeline to perform reconstruction of 3D anatomy from 2D standard cardiac views, effectively enabling 3D anatomical reconstruction from limited 2D data. We evaluate our pipeline using synthetically generated data achieving accurate 3D whole-heart reconstructions (peak intersection over union score $$> 0.88$$ > 0.88 ) from just two standard anatomical 2D views of the heart. We also show preliminary results using real echo images.

Synthetic Echocardiograms Generation Using Diffusion Models

“…We first compare our method with the Echo from Noise method proposed by Stojanovski et al [18]. This method is limited to the generation of one 2D echocardiogram image, not a sequence, whereas we generate echocardiogram sequences with cardiac motion [18]. We only use the FID score for comparison here because Echo from Noise only creates a single frame.…”

“…We only use the FID score for comparison here because Echo from Noise only creates a single frame. We used the real and synthetic images provided by [18] to estimate the FID score of the Echo from Noise method. The FID score of echocardiogram images generated using Echo from Noise is 75.36 while FID score of images generated using our method is 17.65.…”

“…Diffusion models have been used to generate images [18], [24], [25], and speech signals [26]. Recently, several methods have been proposed which explore the generation of video sequences [27]–[29].…”

“…Stojanovski et al . [18] proposed “Echo from Noise” to produce one annotated image of an echocardiogram. This method does not produce echocardiogram sequences.…”

“…One way to address the problems above is to use synthetic data for echocardiogram analysis. Generative machine learning methods, such as GANs [16] or diffusion models [17] have been used to generate synthetic data that can be used for training [18] 22]. In this paper, we propose a video diffusion method to generate echocardiograms.…”