Participatory Systems Modelling for Youth Mental Health: An Evaluation Study Applying a Comprehensive Multi-Scale Framework

Abstract

The youth mental health sector is persistently challenged by issues such as service fragmentation and inefficient resource allocation. Systems modelling and simulation, particularly utilizing participatory approaches, is offering promise in supporting evidence-informed decision making with limited resources by testing alternative strategies in safe virtual environments before implementing them in the real world. However, improved evaluation efforts are needed to understand the critical elements involved in and to improve methods for implementing participatory modelling for youth mental health system and service delivery. An evaluation protocol is described to evaluate the feasibility, value, impact, and sustainability of participatory systems modelling in delivering advanced decision support capabilities for youth mental health. This study applies a comprehensive multi-scale evaluation framework, drawing on participatory action research principles as well as formative, summative, process, and outcome evaluation techniques. Novel data collection procedures are presented, including online surveys that incorporate gamification to enable social network analysis and patient journey mapping. The evaluation approach also explores the experiences of diverse stakeholders, including young people with lived (or living) experience of mental illness. Social and technical opportunities will be uncovered, as well as challenges implementing these interdisciplinary methods in complex settings to improve youth mental health policy, planning, and outcomes. This study protocol can also be adapted for broader international applications, disciplines, and contexts.

Applying systems approaches to stakeholder and community engagement and knowledge mobilisation in youth mental health system modelling

“…As detailed in Program’s evaluation protocol [ 5 ], and to ensure a transfer of learnings and research quality assurance process, a PAR approach will be embedded to draw insights from the evaluation process through reflective research team discussions at key time points in the project e.g., following each site visit and participatory workshop. This process will seek to highlight the importance of having a disciplined but adaptive approach to site and stakeholder engagement when conducting research with health services.…”

“…The participatory process and methodology applied will enable partnering with local communities to collectively improve the trajectory for young people. Principles of a genuine shared decision-making process will be applied to the systems modelling [ 1 ], participatory modelling process [ 2 ], evaluation process [ 5 ] and the economic analysis. Such applications can be regionally implemented and provide sustainable health system decision support tools, digital infrastructure and service innovation.…”

“…This Program will facilitate regions to effectively respond to the challenges ahead and establish a coordinated service ecosystem that supports young people to access the right level of care, delivered early (first time) in the course of illness, and for a sufficiently long-period, to ensure they thrive. A comprehensive evaluation of the: (i) Feasibility; (ii) Value; (iii) Change & Action (Impact), and; (iv) Sustainability of the Program will be conducted alongside the Program implementation and also incorporates a PAR approach [ 5 ].…”

Unleashing the Potential of Systems Modeling and Simulation in Supporting Policy-Making and Resource Allocation for Suicide Prevention

“…Stakeholder insight into contextual barriers and facilitators for the translation of interventions into practice can guide the development of locally relevant implementation strategies (Zimmerman et al, 2016). A participatory modeling approach allows for transparency in how models are conceptualized, developed, and evaluated, thus increasing model credibility and confidence in the findings (Lee et al, 2022). Novel frameworks guiding the documentation of stakeholder engagement and evaluation of participatory modeling methods for SDM in youth mental health and suicide prevention are now available (Lee et al, 2022).…”

“…A participatory modeling approach allows for transparency in how models are conceptualized, developed, and evaluated, thus increasing model credibility and confidence in the findings (Lee et al, 2022). Novel frameworks guiding the documentation of stakeholder engagement and evaluation of participatory modeling methods for SDM in youth mental health and suicide prevention are now available (Lee et al, 2022). Such frameworks are highly transferable and allow for the systematic assessment of the feasibility, value, impact, and sustainability of participatory modeling methods for developing system dynamics models (Lee et al, 2022).…”