Breast cancer is among the most common cancers women got, which can be effectively cured providing that it is diagnosed at the early stages. In the current study, we attempted to classify breast cancer into two groups of malignant and benign by proposing a new ensemble learning method using Multi-Verse Optimizer (MVO) and Gradient Boosting Decision Tree (GBDT). Moreover, the prediction rate of GBDT has been shown to be desirable, its efficiency and classification accuracy are significantly dependent on feature selection and parameter setting. Based on the MVO, we attempted to propose an efficient approach to optimize feature selection and GBDT's parameters at the same time. In other words, the MVO algorithm is able to play the role of a tuner to set the GBDT's main parameters and optimize feature selection results. To implement and test the proposed approach, standard criteria (i.e. accuracy, sensitivity, specificity, etc.) was used for performance evaluation. Also, the datasets of Wisconsin Diagnostic Breast Cancer and Wisconsin Breast Cancer were considered for this purpose. Comparing the results of GBDT-MVO model with other proposed models demonstrated that this model is more precise and has considerably lower variance in the case of a breast cancer diagnosis.