The most common type of cancer that affects women worldwide is breast cancer. After lung, it is the second most cause of the greatest number of cancer deaths among women. A positive research outlook is essential for classifying breast cancer to increase the rate of early diagnosis and prolong the lives of sufferers. For this purpose, the different types of medical image processing mechanisms are developed in the existing works. Still, it is facing some significant problems regarding overfitting, high segmentation error, increased false predictions, and deployment complexity. Therefore, this research intends to develop an innovative and automated breast cancer diagnosis framework using a Piece Variation based Semantic Segmentation (PVSS) - Continuous Gate Recurrent Neural Network (CGRNN) classification mechanisms. Here, the PVSS mechanism is implemented to accurately segment the normalized breast image for improving the classifier's training and testing operations. Then, an Energy based Textural Histogram (ETH) feature extraction algorithm is used to obtain the relevant features for increasing the accuracy of detection. Finally, the CGRNN model is utilized to accurately categorize the healthy and abnormal breast images based on the optimized parameters. To fine tune the parameters of the classifier, an advanced Red Fox Optimization (RFO) algorithm is utilized that provides the best solution to select the parameters. During the evaluation, the performance and results of the proposed PVSS-CGRNN mechanism is validated and compared by using various metrics.