Breast cancer poses a significant threat to women’s health, emphasizing the crucial role of timely detection. Traditional pathology reports, though widely used, face challenges prompting the development of automated Deep Learning (DL) tools. DL models, gaining traction in radiology, offer precise diagnoses; however, issues with generalization on varying dataset sizes persist. This paper introduces a computationally efficient DL framework, addressing dataset imbalance through a hybrid model design, ensuring both accuracy and speed in breast cancer image classification. Proposed model novel design excels in accuracy and generalization across medical imaging datasets, providing a robust tool for precise diagnostics. The proposed model integrates features from two classifiers, Inception ResNet V2 and Vision Transformers (ViT), to enhance the classification of breast cancer. This synergistic blend enhances adaptability, ensuring consistent performance across diverse dataset scales. A key contribution is the introduction of an Efficient Attention Mechanism within one of the classifiers, optimizing focus on critical features for improved accuracy and computational efficiency. Further, a Resource-Efficient Optimization model through feature selection is proposed, streamlining computational usage without compromising accuracy. Addressing the inherent heterogeneity within classifiers, our framework integrates high dimensional features comprehensively, leading to more accurate tumor class predictions. This consideration of heterogeneity marks a significant leap forward in precision for breast cancer diagnosis. An extensive analysis on datasets, BreakHis and BACH, that are imbalanced in nature is conducted by evaluating complexity, performance, and resource usage. Comprehensive evaluation using the datasets and standard performance metrics accuracy, precision, Recall, F1-score, MCC reveals the model’s high efficacy, achieving a testing accuracy of 0.9936 and 0.994, with precision, recall, F1-score and MCC scores of 0.9919, 0.987, 0.9898, 0.9852 and 0.989, 1.0, 0.993, 0.988 on the BreakHis and BACH datasets, respectively. Our proposed model outperforms state-of-the-art techniques, demonstrating superior accuracy across different datasets, with improvements ranging from 0.25% to 15% on the BACH dataset and from 0.36% to 15.02% on the BreakHis dataset. Our results position the framework as a promising solution for advancing breast cancer prediction in both clinical and research applications. The collective contributions, from framework and hybrid model design to feature selection and classifier heterogeneity consideration, establish a holistic and state-of-the-art approach, significantly improving accuracy and establishing optimization in breast cancer classification from MRI images. Future research for the DL framework in breast cancer image classification includes enhancing interpretability, integrating multi-modal data, and developing personalized treatments.
