Since real experimental shearography images are usually few, the application of deep learning for defect detection in digital shearography is limited. A simulation dataset preparation method of shearography images is proposed in this paper. Firstly, deformation distributions are estimated by finite element analysis (FEA); secondly, phase maps are calculated according to the optical shearography system; finally, simulated shearography images are obtained after 2π modulus and gray transform. Various settings in the parameters of object, defect, load and shearing in those three steps could prepare a diverse simulation dataset for deep learning. Together with the real experimental images taken from a shearography setup, hybrid trainings of deep learning for defect detection are performed and discussed. The results show that a simulation dataset, generated without any real defective specimen, shearography system or manual experiment, can greatly improve the generalization of a deep learning network when the number of experimental training images is small.