The prediction of surface finish during machining is critical for determining tool change conditions. The present work methodology identifies different classes of surface finish during machining i.e., good, satisfactory, or poor. To identify the surface quality, the time-domain, frequency-domain, and fast fourier transform (FFT) image features of vibration data during machining were used. These features have been fed to the Bayesian optimized Support Vector Machine (SVM) model and compared. The comparison criteria considered are confusion matrix, Receiver Operator Characteristic (ROC) curve and accuracy. The model with FFT image features and cutting parameters as input provides 84.84 % accurate classification. However, 91.90 % accuracy has been observed using the model with frequency-domain features included with cutting parameters. The variation of cutting parameters concerning the response variable has been verified using Taguchi analysis and found satisfactory. The prediction of different classes of surface roughness based on vibration data will help in the automation of quality systems to accept or reject the product.