Deep learning algorithm for visual quality assessment of the spirograms

Abstract

The quality of spirometry manoeuvres is crucial for correctly interpreting the values of spirometry parameters. A fundamental guideline for proper quality assessment is the ATS/ERS Standards for spirometry, updated in 2019, which describe several start-of-test and end-of-test criteria which can be assessed automatically. However, the spirometry standards also require a visual evaluation of the spirometry curve to determine the spirograms’ acceptability or usability. In this study, we present an automatic algorithm based on a convolutional neural network (CNN) for quality assessment of the spirometry curves as an alternative to manual verification performed by specialists. The algorithm for automatic assessment of spirometry measurements was created using a set of randomly selected 1998 spirograms which met all quantitative criteria defined by ATS/ERS Standards. Each spirogram was annotated as 'confirm' (remaining acceptable or usable status) or 'reject' (change the status to unacceptable) by four pulmonologists, separately for FEV1 and FVC parameters. The database was split into a training (80%) and test set (20%) for developing the CNN classification algorithm. The algorithm was optimised using a cross-validation method. The accuracy, sensitivity and specificity obtained for the algorithm were 92.3%, 92.8% and 90.0% for FEV1 and 92.6%, 93.4% and 89.6% for FVC, respectively. The algorithm provides an opportunity to significantly improve the quality of spirometry tests, especially during unsupervised spirometry. It can also serve as an additional tool in clinical trials to quickly assess the quality of a large group of tests.