Reliability and Feasibility of the First-Episode Psychosis Services Fidelity Scale–Revised for Remote Assessment

Addington, Donald; Noël, Valerie; Landers, Matthew; Bond, Gary R.

doi:10.1176/appi.ps.202000072

Cited by 19 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 24 , 69 However, remote or self-reported fidelity assessments may mitigate the burden of onsite fidelity audits. 23 , 25 For jurisdictions currently developing EIS on a large scale, embedding quality assurance monitoring within the global implementation scheme should be considered. 20 , 70 Other innovative methods to evaluate program operations and improve adherence to essential components such as rapid learning health systems are currently being studied.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The Impact of Policy Changes, Dedicated Funding and Implementation Support on Early Intervention Programs for Psychosis

2022

View full text Add to dashboard Cite

Introduction Early intervention services for psychosis (EIS) are associated with improved clinical and economic outcomes. In Quebec, clinicians led the development of EIS from the late 1980s until 2017 when the provincial government announced EIS-specific funding, implementation support and provincial standards. This provides an interesting context to understand the impacts of policy commitments on EIS. Our primary objective was to describe the implementation of EIS three years after this increased political involvement. Methods This cross-sectional descriptive study was conducted in 2020 through a 161-question online survey, modeled after our team's earlier surveys, on the following themes: program characteristics, accessibility, program operations, clinical services, training/supervision, and quality assurance. Descriptive statistics were performed. When relevant, we compared data on programs founded before and after 2017. Results Twenty-eight of 33 existing EIS completed the survey. Between 2016 and 2020, the proportion of Quebec's population having access to EIS rose from 46% to 88%; >1,300 yearly admissions were reported by surveyed EIS, surpassing governments’ epidemiological estimates. Most programs set accessibility targets; adopted inclusive intake criteria and an open referral policy; engaged in education of referral sources. A wide range of biopsychosocial interventions and assertive outreach were offered by interdisciplinary teams. Administrative/organisational components were less widely implemented, such as clinical/administrative data collection, respecting recommended patient-to-case manager ratios and quality assurance. Conclusion Increased governmental implementation support including dedicated funding led to widespread implementation of good-quality, accessible EIS. Though some differences were found between programs founded before and after 2017, there was no overall discernible impact of year of implementation. Persisting challenges to collecting data may impede monitoring, data-informed decision-making, and quality improvement. Maintaining fidelity and meeting provincial standards may prove challenging as programs mature and adapt to their catchment area's specificities and as caseloads increase. Governmental incidence estimates may need recalculation considering recent epidemiological data.

show abstract

Section: Discussionmentioning

confidence: 99%

“… 22 The FEPS-FS, which operationalised essential components of the model on a five-point scale, is among the most used fidelity tools and was recently revised to enable remote assessments. 23 It has been used and adapted in North America 20 , 24 and Italy. 25 When relevant, data on programs founded before and after 2017 were compared.…”

Section: Methodsmentioning

confidence: 99%

The Impact of Policy Changes, Dedicated Funding and Implementation Support on Early Intervention Programs for Psychosis

2022

View full text Add to dashboard Cite

show abstract

“…Trained assessors will review data abstracted from health records and conduct interviews with staff to complete the FEPS-FS. The fidelity assessment can be performed remotely with excellent reliability [ 48 , 49 ]. The fidelity assessment of virtual NAVIGATE will be compared to previous fidelity assessments of traditional, in-person, NAVIGATE at CAMH and at other sites across Ontario [ 36 ].…”

Section: Methodsmentioning

confidence: 99%

Adapting Evidence-Based Early Psychosis Intervention Services for Virtual Delivery: Protocol for a Pragmatic Mixed Methods Implementation and Evaluation Study

Tempelaar¹,

Barwick²,

Crawford³

et al. 2021

JMIR Res Protoc

Self Cite

View full text Add to dashboard Cite

Background Timely and comprehensive treatment in the form of early psychosis intervention (EPI) has become the standard of care for youth with psychosis. While EPI services were designed to be delivered in person, the COVID-19 pandemic required many EPI programs to rapidly transition to virtual delivery, with little evidence to guide intervention adaptations or to support the effectiveness and satisfaction with virtual EPI services. Objective This study aims to explore the adaptations required to deliver NAVIGATE, a model of coordinated specialty care used in EPI, in a virtual format. This study will evaluate implementation of the NAVIGATE model delivered virtually by describing the nature of the adaptations to the intervention, assessing fidelity to the EPI model and the satisfaction of clients, family members, and care providers. We will investigate barriers and facilitators to virtual NAVIGATE implementation, service engagement, and health equity impacts of this work. Methods The Centre for Addiction and Mental Health (Toronto, Ontario, Canada) transitioned to delivering NAVIGATE virtually early in the COVID-19 pandemic. The Framework for Reporting Adaptations and Modifications for Evidence-Based Interventions will be used to describe the adaptations required to deliver NAVIGATE virtually. Fidelity to the EPI model will be measured using the First Episode Psychosis Services Fidelity Scale and fidelity to NAVIGATE will be assessed by investigating adherence to its core components. Implementation facilitators and barriers will be explored using semistructured interviews with providers informed by the Consolidated Framework for Implementation Research. Satisfaction with virtually delivered NAVIGATE will be assessed with virtual client and provider experience surveys and qualitative interviews with clients, family members, and providers. Service engagement data will be collected through review of medical records, and potential impacts of virtually delivered NAVIGATE on different population groups will be assessed with the Health Equity Impact Assessment. Results Virtual clinical delivery of NAVIGATE started in March 2020 with additional adaptations and data collection is ongoing. Data will be analyzed using descriptive statistics and survival analysis for quantitative data. Qualitative data will be analyzed using thematic content analysis. Integration of qualitative and quantitative data will occur at the data collection, interpretation, and reporting levels following a convergent design. Conclusions This study will provide information regarding the type of intervention adaptations required for virtual delivery of NAVIGATE for youth with early psychosis, ensuring access to high-quality care for this population during the pandemic and beyond by guiding future implementation in similar contexts. International Registered Report Identifier (IRRID) DERR1-10.2196/34591

show abstract

“…This first version of the fidelity scale was tested on programs in the United States and Canada (Addington et al, 2016). The scale was then modified in response to feedback from two multisite studies, one in Canada and one in the United States, the former based on a representative sample of 9 sites, the latter on remote assessment of 36 programs from 32 States (Addington et al, 2020b; Durbin et al, 2019). The scale was applied with a self‐report method to assess the Italian Early Psychosis Services (Addington et al, 2020).…”

Section: The First Episode Psychosis Services Fidelity Scalementioning

confidence: 99%

“…The authors compare the NCAP, used in the England and Wales National Health Services and from 2020, the Republic of Ireland to the First Episode Psychosis Services Fidelity Scale (FEPS‐FS) which has been used in Canada, the United States of America and Italy (Addington, 2021a; Addington et al, 2020; Addington et al, 2020a; Durbin et al, 2019). In this article, we refer to the 2020 versions of the measures.…”

Section: Introductionmentioning

confidence: 99%

A comparison of the National Clinical Audit of Psychosis 2019/2020 standards and First Episode Psychosis Services Fidelity Scale 1.0

Addington

French

Smith

2021

Early Intervention Psych

Self Cite

View full text Add to dashboard Cite

Aim The authors compare two approaches to assessment of the quality of early psychosis intervention services, the National Clinical Audit of Psychosis used in the United Kingdom and the First Episode Psychosis Services Fidelity Scale used in North America and Europe. Methods We compare the two approaches on the source of standards, measurement type, data collection, time requirements, scoring and reliability. Finally, we review their strengths and limitations. Results Both measures are based on standards derived from the same research evidence base. Both methods rely on data from health records and administrative data. The audit is supplemented with user survey data, the fidelity scale with clinician interviews. The audit requires more time. The audit is based on quality indicators rated as present or absent which yields a statistical benchmark. The Fidelity Scale is based on quality indicators that are rated on a five‐point scale yielding a standards‐based measure. The two methods cover similar service components, but the FEPS‐FS has a broader coverage of team functioning. The National audit also collects data on the user experience directly from patients. The fidelity scale has achieved good to excellent inter‐rater reliability, the reliability of the audit has not been tested. Conclusions Both methods have face validity and provide reliable and useful measures of quality of care. The NCAP works in the context of a single provider health system, the FEPS‐FS works in a more variable health system. Comparing the two systems in the field would support international comparison of standards of care.

show abstract

Reliability and Feasibility of the First-Episode Psychosis Services Fidelity Scale–Revised for Remote Assessment

Cited by 19 publications

References 17 publications

The Impact of Policy Changes, Dedicated Funding and Implementation Support on Early Intervention Programs for Psychosis

The Impact of Policy Changes, Dedicated Funding and Implementation Support on Early Intervention Programs for Psychosis

Adapting Evidence-Based Early Psychosis Intervention Services for Virtual Delivery: Protocol for a Pragmatic Mixed Methods Implementation and Evaluation Study

A comparison of the National Clinical Audit of Psychosis 2019/2020 standards and First Episode Psychosis Services Fidelity Scale 1.0

Contact Info

Product

Resources

About