To address the issue of low precision in classifying color differences of yarn‐dyed fabrics and high cost of manual detection, a color difference classification method relying on an improved seagull optimization algorithm(SOA)optimized regularized random vector function link(RRVFL)model is proposed for dyed fabrics. Firstly, to address the issue of slow convergence speed of seagull optimization algorithm (SOA), this paper proposes to optimize the initial group of SOA by Marine predators algorithm (MPA). So it can effectively improve the convergence ability and global optimization ability of SOA algorithm. Subsequently, the enhanced SOA algorithm is applied to fine‐tune the parameters of regularized random vector functional link (RRVFL). Compared with the methods that only optimize weights and bias, the MSOA‐RRVFL model in this paper also increases the optimization of the count of nodes in the hidden layer and regularization parameters, which also effectively avoids the issue of low classification accuracy of the RRVFL model due to random related parameters. Finally, by comparing the RRVFL model with other optimization algorithms, and the experimental outcomes demonstrate that the convergence ability of the improved SOA algorithm has been improved, and the average accuracy of color difference classification of the MSOA‐RRVFL model is as high as 99.79%, and the classification error fluctuation can be stabilized below 0.2%. In general, the MSOA‐RRVFL model has excellent performance in terms of stability and significance.This article is protected by copyright. All rights reserved.