In order to improve the accuracy of convolutional neural networks (CNN) in imbalanced dataset classification, a novel hierarchical CNN-LSVM is proposed. Considering the imbalance in the number and spatial distribution of wheat leaf disease images, the improved local support vector machine (LSVM) replaces Softmax as the classifier of the model, and meanwhile a cost sensitive matrix is designed to assign the value for penalty factors in the optimized objective function of LSVM. It effectively improves the sensitivity of misclassification caused by imbalanced data. To verify the validity and practicability of CNN-LSVM, 6028 wheat leaf disease images containing 8 species are collected from planting bases in Shandong Agricultural University. Then the imbalanced and balanced standard image sets are generated by data augmentation and Borderline-Synthetic Minority Oversampling (Borderline-SMOTE). They have 36168 and 46176 images, respectively. The experimental results show that the average identification accuracies of the CNN-LSVM obtained on imbalanced and balanced standard datasets are 90.32 % and 93.68 %, respectively. And it starts to converge when the iteration times are close to 13000. CNN-LSVM has higher classification accuracy and lower iteration times, compared with CNN-Softmax, CNN-SVM, LSVM and support vector machine (SVM).