Performance Comparison of CGM Systems

Abstract

Currently three manufacturers from the United States have needle-type systems for real-time continuous glucose monitoring (CGM) on the market. Each of these manufacturers has introduced new generations of its respective CGM system to the market over the past decade; each new generation has shown substantial improvements in analytical performance, size, handling, and so on (Dexcom G4® Platinum, Abbott FreeStyle® Navigator II [currently not available in the US market], Medtronic Enlite®).1 As well, one manufacturer from the European Union (EU) has announced a new product. 2While the rapid development and subsequent improvement are quite positive for clinical usage, they hamper the evaluation of the safety and efficacy of such systems in the classic evaluation setting, that is, the performance of clinical head-tohead studies, which typically require years for completion.One parameter often used to characterize the analytical performance of CGM systems is MARD, the mean (sometimes also the median value is used) absolute relative difference between the CGM readings and the values measured at the The ongoing progress of continuous glucose monitoring (CGM) systems results in an increasing interest in comparing their performance, in particular in terms of accuracy, that is, matching CGM readings with reference values measured at the same time. Most often accuracy is evaluated by the mean absolute relative difference (MARD). It is frequently overseen that MARD does not only reflect accuracy, but also the study protocol and evaluation procedure, making a crossstudy comparison problematic.

Limits to the Evaluation of the Accuracy of Continuous Glucose Monitoring Systems by Clinical Trials

“…As already stated, MARD is a stochastic quantity. With an increasing number of paired points, the stochastic portion of MARD tends towards 0, as already shown phenomenologically in [ 27 ].…”

“…The most important factors influencing the MARD value—besides the measurement accuracy of the glucose sensor itself—are either of physiological nature or related to the study protocol (see also [ 27 , 28 ]).…”

“…This metric has many advantages, first among which is the simplicity, but the MARD can be misleading for several reasons (e.g., [ 27 , 28 ]), mainly related to the limits of the clinical study protocols—some of which, such as the small number of patients, are inherent. Indeed, an assessment based on clinical tests will reflect not only the “true” accuracy, but also several other factors independent of it and strongly related to the design of the clinical study.…”

Monitoring of Pediatric Type 1 Diabetes

“…In 1999, the first commercial CGM system had a MARD of 26% (48), whereas the MARD of currently available sensors ranges from 9.0% without any calibration for the Dexcom G6 and 9.4% for the Freestyle Libre, to 9.6% for the Medtronic Guardian with three to four daily calibrations (30,(49)(50)(51). It should be noted, however, that the MARD value is impacted by many factors beyond the accuracy of the sensor, including: the glucose concentration, the absolute number of data points, the rate of glucose change, intensive exercise, and missing data points (52)(53)(54). Therefore, different studies have resulted in different MARD values for the same device (55).…”

Self-measurement of Blood Glucose and Continuous Glucose Monitoring – Is There Only One Future?

“…Additionally, there is a high degree of variability in MARD values for different CGM systems and even with the same CGM system across different studies, with different study designs, conditions and reference methods. 29 Therefore, usage of the MARD alone is insufficient to characterise the analytical performance of a given CGM system.…”