Improving the Efficiency of Stroke Trials

Abstract

C linical trials in acute stroke are large and expensive, even with recent innovations in trial design.1 The modified Rankin scale (mRS) is the most commonly used outcome measure in stroke research. 2 The mRS is an ordinal scale of 7 categories describing full recovery, by increasing degrees of disability, to death.3 Typically, mRS assessment is based on a clinician's rating of a patient interview, and interobserver variability is common. 4 Meta-analysis suggests an overall reliability of κ=0.62 (κ w =0.9), 4 but this may be less (κ=0.25) in multicenter studies. 5 Mandatory training in mRS assessment is used in most trials to mitigate this, 6 but the problem persists. The end point misclassification inherent in this interobserver variability may affect trial power 7 and treatment effect size. 8 Central adjudication of trial end points is routinely used in a variety of settings but has been rarely used in stroke. Group adjudication of mRS has been based on review of written summaries 9 or telephone interview. 10,11 Advantages to a remote adjudication approach include the following: expert opinion and experience; quality control of mRS interviews and avoiding potential bias from use of mRS where local observers are not or cannot be blinded to treatment allocation. To date, remote functional assessment has been limited as a result of the difficulty in capturing a high-fidelity recording, suitable for off-line review. Furthermore, most trials are international, and culturally sensitive translation may also prove problematic. Our pilot data suggest that a video-based mRS assessment is a valid and reliable solution. 12Background and Purpose-Use of the modified Rankin scale (mRS) in multicenter trials may be limited by interobserver variability. We assessed the effect of this on trial power and developed a novel group adjudication approach. Methods-We generated power and sample size estimates from simulated trials modeled with varying mRS reliability.We conducted a virtual acute stroke trial across 14 UK sites to develop a group adjudication approach. Traditional mRS interviews, performed at local sites, were digitally recorded and scored by adjudication committee. We assessed the effect of translation by comparing scores in translated mRS interviews, originally conducted in English and Mandarin. Agreement was measured using κ and weighted κ (κ w ) statistics and intraclass correlation coefficient. Results-Statistical simulations suggest that improving mRS reliability from κ=0.25 to κ=0.5 or 0.7 may allow reductions in sample size of n=386 or 490 in a typical n=2000 study. Our virtual acute stroke trial included 370 participants and 563 mRS video assessments. We adjudicated mRS in 538 of 563 (96%) study visits. Methods Sample Size SimulationsWe performed simulations to demonstrate the effect of increasing mRS reliability and using multiple observers to assign mRS scores. We generated power estimates from simulated mRS studies under various combinations of sample size (N), effect size (δ), reliability (unweighted κ...

Validation of a Structured Interview for Telephone Assessment of the Modified Rankin Scale in Brazilian Stroke Patients

“…This parallels some prior reports [15,16]. We believe that the discrimination between absolute independence and minor dependence in the first months after the stroke might be clouded by possible over-reactive concerns from caregivers as well as by the incipient adaptation process.…”

“…In our sample, there was a moderate weighted agreement among caregivers and a good weighted agreement among the patients. Although these results are similar to previous literature [15], they may not be representative due to the small number of subjects, since the kappa statistics is severely penalized by reductions in sample size.…”

“…The information of mRS obtained by the caregiver is suggested to be less favorable than the information obtained by the patients themselves [15]. In our sample, there was a moderate weighted agreement among caregivers and a good weighted agreement among the patients.…”

Phone and Video-Based Modalities of Central Blinded Adjudication of Modified Rankin Scores in an Endovascular Stroke Trial

“…Agreement was good for both modalities, but it was higher with video-based adjudications than phone-based adjudications. Agreement between local and central video assessor was higher compared with CARS study, 7 where they reported a k w of 0.80 (0.75-0.84) for adjudicated panel score versus local score at 90 days. In CARS study, there was initial disagreement from ≥1 of the 4 panel members in around 50% of evaluations, reaching unanimous agreement after discussion in majority of cases.…”

“…Correlation between mRS obtained with each modality and 24-hour follow-up infarct volumes was studied. 7 Agreement between the centrally adjudicated and local evaluations was good. They also demonstrated the feasibility of using translated interviews simulating an international multicenter study.…”