Improving cervical cancer classification with imbalanced datasets combining taming transformers with T2T-ViT

Abstract

Cervical cell classification has important clinical significance in cervical cancer screening at early stages. However, there are fewer public cervical cancer smear cell datasets, the weights of each classes’ samples are unbalanced, the image quality is uneven, and the classification research results based on CNN tend to overfit. To solve the above problems, we propose a cervical cell image generation model based on taming transformers (CCG-taming transformers) to provide high-quality cervical cancer datasets with sufficient samples and balanced weights, we improve the encoder structure by introducing SE-block and MultiRes-block to improve the ability to extract information from cervical cancer cells images; we introduce Layer Normlization to standardize the data, which is convenient for the subsequent non-linear processing of the data by the ReLU activation function in feed forward; we also introduce SMOTE-Tomek Links to balance the source data set and the number of samples and weights of the images we use Tokens-to-Token Vision Transformers (T2T-ViT) combing transfer learning to classify the cervical cancer smear cell image dataset to improve the classification performance. Classification experiments using the model proposed in this paper are performed on three public cervical cancer datasets, the classification accuracy in the liquid-based cytology Pap smear dataset (4-class), SIPAKMeD (5-class), and Herlev (7-class) are 98.79%, 99.58%, and 99.88%, respectively. The quality of the images we generated on these three data sets is very close to the source data set, the final averaged inception score (IS), Fréchet inception distance (FID), Recall and Precision are 3.75, 0.71, 0.32 and 0.65 respectively. Our method improves the accuracy of cervical cancer smear cell classification, provides more cervical cell sample images for cervical cancer-related research, and assists gynecologists to judge and diagnose different types of cervical cancer cells and analyze cervical cancer cells at different stages, which are difficult to distinguish. This paper applies the transformer to the generation and recognition of cervical cancer cell images for the first time. Supplementary Information The online version contains supplementary material available at 10.1007/s11042-022-12670-0.

Recent advances of Transformers in medical image analysis: A comprehensive review

“…There exist only a few public cervical cancer datasets, the quality of which were also unsatisfactory in image quality and sample distribution. Thus, Zhao et al 82…”

“…There exist only a few public cervical cancer datasets, the quality of which were also unsatisfactory in image quality and sample distribution. Thus, Zhao et al 82 introduced the taming Transformer design to launch a novel cervical cell image generation model: T2T‐ViT to improve the classification results of cervical cancer. This Tokens‐to‐Token Vision Transformers (T2T‐ViT) model can provide balanced and sufficient cervical cancer datasets with high quality.…”

“…This Tokens‐to‐Token Vision Transformers (T2T‐ViT) model can provide balanced and sufficient cervical cancer datasets with high quality. With an encoder‐to‐decoder structure, T2T‐ViT introduced SE‐block and MultiRes‐block in the encoder and SMOTE‐Tomek Links 82 to adjust the sample numbers and image weights of the data set.…”

Virtual perturbations to assess explainability of deep-learning based cell fate predictors

“…As a result, this field has seen a recent explosion of studies applying deep learning [5]. The capability of DNNs to extract patterns from complex data has led to their application by cell biologists in tasks as varied as feature extraction [6,7,8,9], morphology-based classification [10,11,12,13,14,15,16], image segmentation [17,18,19,20,21,22,23,24] , synthetic data generation [25,26] and more.…”

Pap Smear Images Classification Using Machine Learning: A Literature Matrix

“…Cervical cytopathology image classification is an important method for diagnosing cervical cancer [ 18 ]. Moreover, previous studies have mentioned that cervical cell classification has important clinical consequences in cervical cancer screening at an early stage [ 19 , 20 ]. The effective classification of Pap smear cell images may be used to create automated and precise cervical cancer classification systems for early diagnosis [ 9 ].…”