Assessing and Strengthening Evidence-Based Program Registries’ Usefulness for Social Service Program Replication and Adaptation

Abstract

Evidence-based program registries provide insufficient information to guide context-sensitive decision making about program replication and adaptation. Registries should supplement their evidence base with nonexperimental evaluations and revise their methodological screens and synthesis-writing protocols to prioritize reporting-by both evaluators and the registries themselves-of context-specific implementation factors that affect program outcomes.

Systematic mapping of checklists for assessing transferability

“…The descriptions of the included checklists suggested that five checklists were primarily intended to be used by practitioners [19, 42, 46, 48, 49], seven by decision makers (this could include practitioners, as well as program managers, policy makers, politicians, etc.) [8, 34, 36, 39, 44, 45, 47], four by primary researchers (to presumably improve conduct and/or reporting) [18, 24, 35, 41], and five of the checklists were aimed at assisting both decision makers and researchers in making assessments [9, 17, 37, 38, 40]. Four of the identified checklists were intended for systematic review authors [11, 20, 43, 50] (see more details regarding these checklists in Table 3).…”

“…The majority of the 25 checklists were developed for use in health research ( N = 23). However, two checklists were developed for assessing research on interventions within the social sciences [44, 50]. All of the included checklists were aimed at assessing effectiveness research (quantitative data).…”

“…Characteristics of the population in this case is conceptualized quite broadly and includes not only the demographic characteristics of the population, but also characteristics of the participants condition/illness, the acceptability of the intervention to the participants and/or their preferences, from where or how the participants were referred to the intervention, participants’ need for/access to information about the intervention, availability of personal support for participants, participants’ exposure to other interventions or previous exposure to current intervention, participants’ compliance and/or satisfaction with the intervention. The majority of the included checklists included one or more checklist criterion that were coded into one or more of the categories under the theme Population ( N = 20; [8, 9, 11, 17, 19, 20, 24, 34–40, 42, 44, 46–49]). These criteria were described using different terms.…”

“…Some categories describe criteria that ask users of a checklist to consider whether there are standard procedures for the intervention in a real life setting, who pays for the intervention and whether any other general details related to intervention delivery could influence transferability. Twelve checklists included criteria that were coded into one or more categories under this subtheme [8, 9, 11, 17, 18, 35, 39, 40, 44, 46, 48, 49].…”

“…Other categories in this subtheme are related to practices around monitoring and supervision of the service providers, their compliance, and factors that influence the motivation of service providers. Sixteen checklists included criteria that were coded into the categories of criteria under this subtheme [8, 9, 17–20, 37–42, 44, 48–50].…”