Cardiovascular diseases are the leading cause of mortality worldwide. Cardiac imaging is critical for precise characterization of cardiac structure and function, and is key in diagnosis, therapeutic management, and prognosis. In this work, we propose a novel representation of spatio-temporal cardiac data as a multiplex graph and develop a multi-level message passing neural network to classify clinical groups corresponding to different cardiovascular diseases. Using open data from the Automated Cardiac Diagnosis Challenge (N=150), our results show that the multiplex representation extracts discriminative features from the data (up to 94% accuracy compared to up to 80% for well-tuned baselines such as XGboost and MLP). Ablation studies show the bias induced by the graph's spatio-temporal structure improves generalization.