The rapid and accurate identification of spider sex is the first step in spider image recognition. The traditional artificial method used to identify the sex of mature spiders is mainly based on their genital structures (male palps or female epigynum) and highly dependent on the professional background of the identifiers. This article uses computer-based deep learning and transfer learning to identify the sex of spider, explores the design and application of convolutional neural networks in deep learning for spider sex recognition from images, and establishes a neural network model that displays excellent performance in experiments. In addition to optimizing the network model, we select appropriate hyperparameters to improve the accuracy of recognition and reduce the influence of human factors in the identification process. Through a comparison of multiple sets of experiments based on existing sample data collected in the laboratory, we find that the transfer learning method based on Xception can obtain better prediction accuracy than ResNet-152. After data augmentation, the optimization of a combined activation function and the fine-tuning of frozen layers, the prediction accuracy reaches 98.02%, and for an actual measurement of independent samples, the recognition accuracy reaches 92.38%. Therefore, the proposed method can basically replace manual identification and provide a reference for the artificial intelligence-based identification of spider species. Additionally, the model results indicate that male and female dimorphism may exist beyond the non-genital characteristics of spiders. INDEX TERMS deep learning; transfer learning; convolutional neural network; spider sex identification