In this paper, a methodology that combines a classification algorithm with a combinatorial optimization problem is proposed. Using the proposed algorithm, classification problems of medical data such as breast cancer data can be performed more accurately. Using the conventional method, the breast cancer data is classified into malignant growth and benign tumor. However, the conventional classification seems insufficient, because there are erroneous cases sometimes, such as the data, having classified into benign, would be found out as malignant, later. To conquer the defect, the proposed method classify the breast cancer data into two groups; (1) the data which definitely belongs to malignant growth area, and (2) the data which has the possibility to belong to either malignant growth area or benign tumor area. Since the libraries of two value classification learning method, used in the conventional method, are highly evaluated in their performance and their easiness of use, the proposed method uses two value classification learning method as well, to achieve the goal. This problem is formulated as combinatorial optimization problem. In this formulation, the malignant growth area is maximized, because of the data which has the possibility to belong to either malignant or benign. The classification algorithm based on learning data selection is proposed, and the effectiveness of the proposed algorithm is discussed through the numerical experiments.