Spectrum sensing describes, whether the spectrum is occupied or empty. Main objective of cognitive radio network (CRN) is to increase probability of detection (Pd) and reduce probability of error (Pe) for energy consumption. To reduce energy consumption, probability of detection should be increased. In cooperative spectrum sensing (CSS), all secondary users (SU) transmit their data to fusion center (FC) for final measurement according to the status of primary user (PU). Cluster should be used to overcome this problem and improve performance. In the clustering technique, all SUs are grouped into clusters on the basis of their similarity. In cluster technique, SU transfers their data to cluster head (CH) and CH transfers their combined data to FC. This paper proposes the detection performance optimization of CRN with a machine learning-based metaheuristic algorithm using clustering CSS technique. This article presents a hybrid support vector machine (SVM) and Red Deer Algorithm (RDA) algorithm named Hybrid SVM-RDA to identify spectrum gaps. Algorithm proposed in this work outperforms the computational complexity, an issue reported with various conventional cluster techniques. The proposed algorithm increases the probability of detection (up to 99%) and decreases the probability of error (up to 1%) at different parameters.