As the healthcare industry increasingly adopts Electronic Health Records (EHRs), there is growing interest in leveraging machine learning (ML) algorithms for proactive risk assessments and effective interventions. However, the true potential of ML and artificial intelligence (AI) in healthcare remains largely untapped, lacking a systematic exploration.In this research, our focus is on using effective baseline models in tackling class imbalance in machine learning. To achieve this, we investigate the effectiveness of tuning probability thresholds and applying ensemble methods as a means to improve the F1 score when dealing with complex ML problems.We aim to look at how baseline models perform when designed as ensemble methods with proper tuning probability thresholds using two evaluation tasks: (a) predicting cardiac arrhythmia and (b) hospital readmissions in ICU patients. Leveraging the publicly available MIMIC-III database, we implemented three baseline ML models: Logistic Regression, Extreme Gradient Boosting (XGBoost), and Neural Network (ANN). The main findings from this study demonstrate the effectiveness of ensemble learning methods by combining predictions tailored to specific patient cohorts. Additionally, this study underscores the significance of tuning probability thresholds to enhance F1 scores, especially in handling imbalanced healthcare data. Notably, in both evaluation tasks, XGBoost outperforms ANN models, consolidating it a promising baseline for intricate deep learning applications.