As an indispensable part of transmission lines, metal fittings play an important role in the safe and stable operation of power system. Automatic detection of metal fittings in the aerial images of power inspection is an urgent problem to be solved. In this study, a deep learning method is proposed to perform metal fittings detection in UAV-based images. Firstly, metal fittings data set used for targets detection is created, including Damper, XC Clamp, and NZ Clamp. Secondly, a new deep learning model is proposed for metal fittings detection. To further improve the accuracy of metal fittings identification in aerial images, ResNet50, attention mechanism, BiFPN and Focal loss are adopted to improve RetinaNet model. Finally, the proposed model and comparative models are trained and tested on metal fittings data set. Experimental results show that the proposed model has a good effect on metal fittings detection in aerial images from power inspection. More importantly, the mAP of the proposed model can reach 93.22%, which is 7%, 9.5%, 16%, and 0.4% higher than that of original RetinaNet, Faster R-CNN, SSD and YOLOv7, respectively. And the AP values of Damper, XC Clamp and NZ Clamp detection can reach 99%, 96%, and 85% by the proposed model. In addition, compared with the mainstream target detection models, the proposed model can obtain better performance in the detection of metal fittings with complex backgrounds and small targets.