Evaluator Training and Reliability for SMA Global Nusinersen Trials1

Abstract

Background:Training methodology was established to optimize reliability of outcome measures in the nusinersen clinical trials. The Children’s Hospital of Philadelphia Infant Test of Neuromuscular Disorders (CHOP INTEND), Hammersmith Functional Motor Scale Expanded (HFMSE), and Revised Upper Limb (RULM) were primary or secondary outcomes.Methods:Video review, quarterly conference calls, and item scoring checks supported evaluator competence. Baseline and screening along with video review established intra and inter-rater reliability.Results:Inter and intra-rater reliability were both excellent. Intraclass correlation coefficients (ICC) ranged between 0.906–0.994 across initial training meetings and 0.824–0.996 across annual retraining meetings. This was similar for CHOP INTEND (ICC = 0.824–0.951), HFMSE (ICC = 0.981–0.996), and RULM (ICC = 0.966–0.990). Intra-rater reliability for the CHOP INTEND, HFMSE, and RULM were ICC = 0.895 (95% CI: 0.852–0.926; n = 116), ICC = 0.959 (95% CI: 0.942–0.971; n = 125), and ICC = 0.948 (95% CI: 0.927–0.963; n = 126) respectively.Conclusions:Rigorous evaluator training ensures reliability of assessment of subjects with spinal muscular atrophy (SMA) in multicenter international trials.

Longitudinal natural history of type I spinal muscular atrophy: a critical review

“…This finding is apparently in contrast with previous observations in other diseases suggesting that placebo groups may show a different disease progression compared to NH both for a possible "placebo effect" and for a likely difference in standard of care when entering a clinical trial. The lack of obvious placebo effect is probably partly justified by the fact that in all three NH studies the patients were followed in tertiary care centers with full adherence to the updated standards of care recommendations and data were collected by trained observers who followed a trial-like protocol including reliability training sessions [14]. These findings strengthen what has been recently discussed in international forums -including scientists, advocacy groups and regulators -that in a severely progressive disease such as type I SMA, there is no need for placebo arms in future clinical trials [15].…”

ACTIVE (Ability Captured Through Interactive Video Evaluation) workspace volume video game to quantify meaningful change in spinal muscular atrophy

“…First, an evaluator‐rated ordinal scale may be less responsive to change as large improvements in function may be required to improve the total score and have potential floor and ceiling effects in various SMA cohorts . Additionally, these outcomes require significant training and reliability to ensure consistent administration and scoring across sites . An outcome utilizing a continuous scale, requiring minimal training, and capturing wide‐ranging function has the potential to improve the efficiency of evaluation across sites.…”

“…8,9 Additionally, these outcomes require significant training and reliability to ensure consistent administration and scoring across sites. 10 An outcome utilizing a continuous scale, requiring minimal training, and capturing wide-ranging function has the potential to improve the efficiency of evaluation across sites.…”

SMA: Des échelles d’évaluation motrice pour le public francophone

“…[9]. Leurs propriétés métrologiques (validité, fiabilité) ont été évaluées pour la plupart de ces outils et rapportées comme satisfaisantes dans les populations étudiées [5,[10][11][12][13]. En France, dans les différentes consultations de maladies neuromusculaires (MNM), la MFM est l'outil de mesure de la fonction motrice le plus utilisé car plus de 1 000 professionnels ont été formés depuis 2005 et plus de 50 centres français participent à l'implémentation d'une base de données nationale (Base de données MFM).…”

“…Plus le score est élevé, plus les capacités fonctionnelles sont importantes. La fiabilité de cette échelle a été évaluée[5,12]. Cette échelle a également été utilisée dans des articles récents[1,15,17].Les autres tests et échelles présentés ici sont destinés à mesurer les limitations d'activité au sens de la Classification Internationale du Fonctionnement (CIF).…”