Performance indicators for public mental healthcare: a systematic international inventory

Abstract

BackgroundThe development and use of performance indicators (PI) in the field of public mental health care (PMHC) has increased rapidly in the last decade. To gain insight in the current state of PI for PMHC in nations and regions around the world, we conducted a structured review of publications in scientific peer-reviewed journals supplemented by a systematic inventory of PI published in policy documents by (non-) governmental organizations.MethodsPublications on PI for PMHC were identified through database- and internet searches. Final selection was based on review of the full content of the publications. Publications were ordered by nation or region and chronologically. Individual PI were classified by development method, assessment level, care domain, performance dimension, diagnostic focus, and data source. Finally, the evidence on feasibility, data reliability, and content-, criterion-, and construct validity of the PI was evaluated.ResultsA total of 106 publications were included in the sample. The majority of the publications (n = 65) were peer-reviewed journal articles and 66 publications specifically dealt with performance of PMHC in the United States. The objectives of performance measurement vary widely from internal quality improvement to increasing transparency and accountability. The characteristics of 1480 unique PI were assessed. The majority of PI is based on stakeholder opinion, assesses care processes, is not specific to any diagnostic group, and utilizes administrative data sources. The targeted quality dimensions varied widely across and within nations depending on local professional or political definitions and interests. For all PI some evidence for the content validity and feasibility has been established. Data reliability, criterion- and construct validity have rarely been assessed. Only 18 publications on criterion validity were included. These show significant associations in the expected direction on the majority of PI, but mixed results on a noteworthy number of others.ConclusionsPI have been developed for a broad range of care levels, domains, and quality dimensions of PMHC. To ensure their usefulness for the measurement of PMHC performance and advancement of transparency, accountability and quality improvement in PMHC, future research should focus on assessment of the psychometric properties of PI.

Key performance indicators for mental health and substance use disorders: a literature review and discussion paper

“…Canada, NHS, HSE, VHA) <insert Table 1 here> <insert Table 2 here> Individual performance measures from the four representative health systems were classified by the method outlined by Lauriks et al (Lauriks, Buster, de Wit, Arah, & Klazinga, 2012). A total of six government publications outlined unique KPIs from -8 -the systems (Table 2).…”

“…While developing KPIs based on expert opinion alone is suboptimal compared to other development methods due to lack of evidence/justification and bias (Lauriks et al, 2012), this approach requires the least resources and may be helpful as an initial starting point in developing a set of KPIs. Alternatively, a system may choose to adapt KPIs utilized in another system to their own needs; however, this can be fraught with obstacles such as lack of sufficient fiscal, organisational, infrastructural and personnel resources to appropriately collect KPIs and ensure dissemination of findings (Garnick, Lee, Horgan, Acevedo, & Washington Circle Public, 2009).…”

“…Establishing a collaborative process is integral in the development of KPIs as MESUD academics, clinicians, consumers and decisionmakers will differ in their opinion of what KPIs are of most importance (Waraich et -11 -al., 2010). The structured consensus is ultimately the most rigorous development method for KPIs; however, this, in itself, requires sufficient resources and infrastructure to execute, such that it is the least used development method for MESUD KPIs (Lauriks et al, 2012;Waraich et al, 2010;Watkins & Pincus, 2011).…”

“…Alternatively, a system may choose to adapt KPIs utilized in another system to their own needs; however, this can be fraught with obstacles such as lack of sufficient fiscal, organisational, infrastructural and personnel resources to appropriately collect KPIs and ensure dissemination of findings (Garnick, Lee, Horgan, Acevedo, & Washington Circle Public, 2009). As the System progresses, there are ample literature resources providing structure and guidance for using more robust methods, which enhance KPI development through collaboration, applying pre-existing instruments and consulting with key stakeholders (Addington, Kyle, Desai, & Wang, 2010;Baars, Evers, Arntz, & van Merode, 2010;Lauriks et al, 2012;Lin & Durbin, 2008;Rush, Martin, Corea, & Rotondi, 2012).…”

Call for information, call for quality in mental health care

“…In a recent review, Lauriks et al (2012) analyzed more than 300 indicators produced by more than 50 stakeholders. This tumultuous scenario indicates a growing interest in quality assessment, but at the same time reflects the marked differences between mental health systems and stakeholder expectations.…”