SummaryWith the rapid growth of malicious codes, personal privacy, and Internet security are seriously threatened. Existing transfer learning‐based malicious code detection improves detection accuracy by transferring pre‐trained neural networks. However, it cannot efficiently tune the structure and parameters of the neural networks. Here, we first propose a novel many‐objective transfer model. It mainly focuses on the detection accuracy and the total number of parameters of the neural network model. The optimal structure and parameters are captured from the pre‐trained neural network by many‐objective optimization algorithm. Second, the partitioned crossover‐mutation vector angle‐based evolutionary algorithm for unconstrained many‐objective optimization is proposed to solve the model. The algorithm performs crossover mutation operations in different ways on different regions of the candidate solution to improve population diversity. The simulation results show that the model can reduce the pre‐trained neural network structure by 49% while maintaining the accuracy in malicious code detection.

No abstract