Towards implementation of AI in New Zealand national screening program: Cloud-based, Robust, and Bespoke

Abstract

25Convolutional Neural Networks (CNN)s have become a prominent method of AI 26 implementation in medical classification tasks. Grading Diabetic Retinopathy (DR) has been 27 at the forefront of the development of AI for ophthalmology. However, major obstacles remain 28 in the generalization of these CNN's onto real-world DR screening programs. We believe these 29 difficulties are due to use of 1) small training datasets (<5,000 images), 2) private and 'curated' 30 repositories, 3) offline CNN implementation methods, while 4) relying on accuracy measured 31 as area under the curve (AUC) as the sole measure of CNN performance. 32 To address these issues, the public EyePACS Kaggle Diabetic Retinopathy dataset was 33 uploaded onto Microsoft Azure™ cloud platform. Two CNNs were trained as a "Quality 34 Assurance", and a "Classifier". The "Classifier" CNN performance was then tested both on 35 'un-curated' as well as the 'curated' test set created by the "Quality Assessment" CNN. Finally, 36 the sensitivity of the "Classifier" CNNs was boosted post-training using two post-training 37 techniques. 38Our "Classifier" CNN proved to be robust, as its performance was similar on 'curated' and 'un-39 curated' sets. The implementation of 'cascading thresholds' and 'max margin' techniques led 40 to significant improvements in the "Classifier" CNN's sensitivity, while also enhancing the 41 specificity of other grades. 42 43 4 44 45It is estimated that by 2040, nearly 600 million people will have diabetes worldwide(1). 46Diabetic retinopathy (DR) is a common diabetes-related microvascular complication, and is 47 the leading cause of preventable blindness in people of working age worldwide(2, 3). It has 48 been estimated that the overall prevalence of non-vision-threatening DR, vision-threatening 49 DR and the blinding diabetic eye disease were 34·6%, 10·2%, and 6·8% respectively (3-6). 50Clinical trials have shown that the risk of DR progression can be significantly reduced by 51 controlling major risk factors such as hyperglycaemia and hypertension (7-9). It is further 52 estimated that screening, appropriate referral and treatment can reduce the vision loss from DR 53 by 50% (10-12). However, DR screening programs are expensive to set up and administrate. It 54 is estimated that even in developed countries, these programs do not reach up to 30% of the 55 diabetic population (13, 14). 56 Artificial intelligence (AI) and its subcategory of deep learning have gained popularity in 57 medical screening programs, including DR screening. In deep learning, a convolutional neural 58 network (CNN) is designed and trained based on large datasets of ground truth data and labels. 59The CNN algorithm adjusts its weights and discovers which features to extract from medical 60 data (e.g. fundus photos) to achieve the best classification accuracy, when compared to human 61 performance (15-20). CNNs use layers with convolutions, which are defined as mathematical 62 functions that use filters to extract features from an image (21-23). The out...

THEIA™ development, and testing of artificial intelligence‐based primary triage of diabetic retinopathy screening images in New Zealand

“…To address any class imbalance issue of our dataset, the weighted loss function strategy was adopted 12,13 . An initial 5000 images, 500 images from each of the five DR severity categories (according to NZ grading standards 14 ) and 500 images from each of the five maculopathy severity categories, were randomly selected for AI training.…”