Brain tumor is caused by the growth of abnormal cells, which forms a mass and affects the brain functions. The existing methods did not provide sufficient accuracy with high computational complexity. Therefore, in this manuscript, a Dropout AlexNet-Extreme Learning optimized with Fast Gradient Descent optimization algorithm is proposed for detecting and classifying images of brain tumor. Here, the input magnetic resonance imaging images are taken from three datasets: BRATS dataset, ISLES dataset, and RemBRANDT Dataset. Then the imageries are preprocessed to remove the noise as well as improve the superiority of the images. The image features are extracted using the Gray-Level Co-Occurrence Matrix methods. The extracted features are given to the Dropout AlexNet-XtremeLearning Machine architecture for classification. Finally, the DrpXLM classifier classifying the brain images as benign, malignant, and normal.The simulation is implemented in MATLAB. For BRATS dataset, the proposed strategy achieves34.64%, 45.36%, and 33.32% higher accuracy for benign, 37.85%, 28.94%, and 56.74% higher accuracy for malignant and 46.76%, 38.96%, and 44.86% better accuracy for normal compared with the existing methods, like Gaussian filters and long short-term memory based brain tumor detection (GF-LSTM-BTD), Shannon's-Entropy and Social-Group-Optimization based brain tumor detection (SE-SGO-BTD), Alex and Google networks with softmax layer based brain tumor detection