MMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets

Abstract

In recent years, the use of the last observation carried forward (LOCF) approach in imputing missing data in clinical trials has been greatly criticized, and several likelihood-based modeling approaches are proposed to analyze such incomplete data. One of the proposed likelihood-based methods is the Mixed-Effect Model Repeated Measure (MMRM) model. To compare the performance of LOCF and MMRM approaches in analyzing incomplete data, two extensive simulation studies are conducted, and the empirical bias and Type I error rates associated with estimators and tests of treatment effects under three missing data paradigms are evaluated. The simulation studies demonstrate that LOCF analysis can lead to substantial biases in estimators of treatment effects and can greatly inflate Type I error rates of the statistical tests, whereas MMRM analysis on the available data leads to estimators with comparatively small bias, and controls Type I error rates at a nominal level in the presence of missing completely at random (MCAR) or missing at random (MAR) and some possibility of missing not at random (MNAR) data. In a sensitivity analysis of 48 clinical trial datasets obtained from 25 New Drug Applications (NDA) submissions of neurological and psychiatric drug products, MMRM analysis appears to be a superior approach in controlling Type I error rates and minimizing biases, as compared to LOCF ANCOVA analysis. In the exploratory analyses of the datasets, no clear evidence of the presence of MNAR missingness is found.

Transcatheter Interatrial Shunt Device for the Treatment of Heart Failure With Preserved Ejection Fraction (REDUCE LAP-HF I [Reduce Elevated Left Atrial Pressure in Patients With Heart Failure])

“…17 We assumed a mean change in exercise PCWP of -6.0 mm Hg in the treatment group and 0.0 mm Hg in the control group at each of 20 W, 40 W, 60 W, and 80 W stages, and we assumed a standard deviation in PCWP change of 7.2 mm Hg in each treatment group at each of the exercise stages. Based on these assumptions, a sample size of 20 evaluable participants per treatment arm yielded 82% power at a 2-sided 0.05 level of significance to detect a significant beneficial effect of IASD System II over control when comparing treatment means using a mixedeffects model repeated measures 21 analysis of covariance that included data from all available stages of exercise, assuming the compound symmetry correlation structure where the pairwise correlations among 20 W, 40 W, 60 W, and 80 W stages of exercise are ≤0.8.…”

Treatment of Recent-Onset Type 1 Diabetic Patients With DiaPep277: Results of a Double-Blind, Placebo-Controlled, Randomized Phase 3 Trial

“…Every participant for whom there existed at least one postbaseline measurement was included in the MMRM analysis, with the model imputing the missing data points (33,34). The MMRM was used to analyze the change from baseline in the primary and secondary end points using SAS PROC MIXED.…”

Selecting the best unbalanced repeated measures model

“…Second, with small samples the standard selection criteria may be highly inefficient, particularly if some data are missing and/or the form of the matrix plays an important role in the estimation. In these cases, it may be more appropriate to select the best mean model assuming the unstructured pattern to model the within-subject errors in the analysis (Siddiqui, Hung, & O'Neill, 2009). Third, it should be noted that information criteria do not automatically select the best model from all possible candidate models.…”