Comparison of an Automated Plate Assessment System (APAS Independence) and Artificial Intelligence (AI) to Manual Plate Reading of Methicillin-Resistant and Methicillin-Susceptible Staphylococcus aureus CHROMagar Surveillance Cultures

Abstract

Background: The Automated Plate Assessment System (APAS Independence) [Clever Culture System, Bäch, Switzerland] is an automated imaging station linked with interpretive software that detects target colonies of interest on chromogenic media and sorts samples as negative or presumptive positive. We evaluated the accuracy of the APAS to triage methicillin-resistant Staphylococcus aureus (MRSA) and S. aureus ) cultures using chromogenic media compared to human interpretation. Methods: Patient samples collected from the nares on Eswabs were plated to BD BBL™ CHROMagar™ MRSA II and BD BBL CHROMagar Staph aureus and allowed to incubate for 20-24 h at 37°C in non-CO2. Mauve colonies are suggestive of S. aureus and were confirmed with latex agglutination. Following incubation, samples were first interrogated by APAS before being read by a trained technologist blinded to the APAS interpretation. The triaging by both APAS and the technologists were compared for accuracy. Any discordant results required further analysis by a third reader. Results: Over a five-month period, 5,913 CHROMagar MRSA cultures were evaluated. Of those, 236 were read as concordantly positive, 5,525 were read as concordantly negative, and 152 required discordant analysis. Positive and negative percent agreements (PPA, NPA) were 100% and 97.3%, respectively. The APAS detected 5 positive cultures that were missed by manual reading, and determined to be true positives. In a separate analysis, 744 CHROMagar Staph aureus plates were read in parallel. Of these, 133 were concordantly positive, 585 were concordantly negative, and 26 required discordant analysis. PPA and NPA were 95.7% and 96.7%, respectively. Conclusion: This study confirmed the high sensitivity of digital image analysis by the APAS Independence such that negative cultures can be reliably reported without technologist intervention (NPV 100% for CHROMagar MRSA and 99.0% for CHROMagar Staph aureus). Triaging using the APAS Independence may provide great efficiencies in a laboratory with high throughput of MRSA and S. aureus surveillance cultures.

Comparison of the APAS Independence Automated Plate Reader System with the Manual Standard of Care for Processing Urine Culture Specimens

“…Not requiring manual verification of the results is advantageous because it cuts down the time needed for patients to receive a final report. This key difference has resulted in the classification of the APAS Independence as a class II medical device in the United States, and it is the only such classified device ( 4 ). While our laboratory urine culture volumes only resulted in a savings of approximately 1 h of hands-on time, greater urine culture volumes likely would have resulted in more substantial time and labor savings.…”

“…UTIs can be caused by a myriad of bacterial pathogens, and there is some subjectivity in the interpretation of urine culture results. Consequently, diagnoses can be subject to variation among the technicians analyzing culture results ( 4 ). The gold standard for processing a urine test involves planting a urine specimen onto culture medium and enumerating the potential pathogens to determine whether there is sufficient growth to be reported.…”

“…The APAS Independence system, created by Clever Culture Systems, is an artificial intelligence-driven automated culture plate-reading system that can feed into existing laboratory infrastructure and provide a balance between manual and automated workflows. With the APAS system, laboratories can expedite the overall diagnostic process by reducing plate sorting times (processing up to 200 plates per hour) and allowing technicians to prioritize the analysis of positive cultures rather than evaluating the plates for all the negative cultures ( 4 ). The system functions by relying upon artificial intelligence to interpret growth patterns on urine plates and determine whether that growth may be clinically significant ( 12 – 14 ).…”

Microbiology 2.0–A “behind the scenes” consideration for artificial intelligence applications for interpretive culture plate reading in routine diagnostic laboratories

“…Despite the years of training and professional desire to be 100% accurate 100% of the time, this is unfortunately not realistic. There is a fundamental risk with any manual interpretation of a culture plate that the interpretation is incorrect, especially when looking for single colonies amongst commensal flora ( Gammel et al, 2021 ). Non-consensus results between microbiologists have also been reported by Glasson et al (2016) who found that agreement between microbiologists for colony morphology is 97.5% when reading MacConkey agar but only 87.5% when reading blood agar.…”

“…Currently, the PhenoMatrix™ (Copan Italia, Brescia, Italy) application provides an assessment of an agar plate in conjunction with expert rules where the user must accept the decision from the algorithm to proceed with reporting or further work, with published studies covering group B Streptococcus and Streptococcus pyogenes ( Van et al, 2019 ; Baker et al, 2020 ). The APAS ® Independence (Clever Culture Systems) is currently the only system with FDA clearance as a class II medical device which is diagnostic for a negative no growth result from urine samples ( Glasson et al, 2017 ), and for no significant growth when used in MRSA detection ( Gammel et al, 2021 ). In these cases, human intervention is only required for significant growths.…”