This study examined the psychometric properties and feasibility of the Illness Management and Recovery (IMR) Fidelity scale. Despite widespread use of the scale, the psychometric properties have received limited attention. Trained fidelity assessors conducted assessments four times over 18 months at 11 sites implementing IMR. The IMR Fidelity scale showed excellent interrater reliability (.99), interrater item agreement (94%), internal consistency (.91-.95 at three time points), and sensitivity to change. Frequency distributions generally showed that item ratings included the entire range. The IMR Fidelity scale has excellent psychometric properties and should be used to evaluate and guide the implementation of IMR.Trial registration: ClinicalTrials.gov Identifier: NCT03271242.
The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as “endpoints.” A “digital endpoint” is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient’s free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone’s microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer’s disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug’s effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient’s experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.
To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77-.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices. Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242
This study examined psychometric properties and feasibility of the Family Psychoeducation (FPE) Fidelity Scale. Fidelity assessors conducted reviews using the FPE fidelity scale four times over 18 months at five sites in Norway. After completing fidelity reviews, assessors rated feasibility of the fidelity review process. The FPE fidelity scale showed excellent interrater reliability (.99), interrater item agreement (88%), and internal consistency (mean = .84 across four time points). By the 18-month follow-up, all five sites increased fidelity and three reached adequate fidelity. Fidelity assessors rated feasibility as excellent. The FPE fidelity scale has good psychometric properties and is feasible for evaluating the implementation of FPE programs. Trial registration ClinicalTrials.gov Identifier: NCT03271242.
The paper describes the Antipsychotic Medication Management Fidelity Scale and its psychometric properties, including interrater reliability, frequency distribution, sensitivity to change and feasibility. Fidelity assessors conducted fidelity reviews four times over 18 months at eight sites receiving implementation support for evidence-based antipsychotic medication management. Data analyses shows good to fair interrater reliability, adequate sensitivity to change over time and good feasibility. At 18 months, item ratings varied from poor to full fidelity on most items. Use of the scale can assess fidelity to evidence-based guidelines for antipsychotic medication management and guide efforts to improve practice. Further research should improve and better calibrate some items, and improve the procedures for access to information. Trial registration: ClinicalTrials.gov Identifier: NCT03271242.
Mental health programs need an instrument to monitor adherence to evidence-based physical health care for people with serious mental illness. The paper describes the Physical Health Care Fidelity Scale and study interrater reliability, frequency distribution, sensitivity to change and feasibility. Four fidelity assessments were conducted over 18 months at 13 sites randomized to implementation support for evidence-based physical health care. We found good to excellent interrater reliability, adequate sensitivity for change, good feasibility and wide variability in fidelity across sites after 18 months of implementation. Programs were more successful in establishing Policies stating physical health care standards than in implementing these Policies. The Physical Health Care Fidelity Scale measures and guides implementation of evidence-based physical health care reliably.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.