Methods used to address fidelity of receipt in health intervention research: a citation analysis and systematic review

Abstract

BackgroundThe American Behaviour Change Consortium (BCC) framework acknowledges patients as active participants and supports the need to investigate the fidelity with which they receive interventions, i.e. receipt. According to this framework, addressing receipt consists in using strategies to assess or enhance participants’ understanding and/or performance of intervention skills. This systematic review aims to establish the frequency with which receipt is addressed as defined in the BCC framework in health research, and to describe the methods used in papers informed by the BCC framework and in the wider literature.MethodsA forward citation search on papers presenting the BCC framework was performed to determine the frequency with which receipt as defined in this framework was addressed. A second electronic database search, including search terms pertaining to fidelity, receipt, health and process evaluations was performed to identify papers reporting on receipt in the wider literature and irrespective of the framework used. These results were combined with forward citation search results to review methods to assess receipt. Eligibility criteria and data extraction forms were developed and applied to papers. Results are described in a narrative synthesis.Results19.6% of 33 studies identified from the forward citation search to report on fidelity were found to address receipt. In 60.6% of these, receipt was assessed in relation to understanding and in 42.4% in relation to performance of skill. Strategies to enhance these were present in 12.1% and 21.1% of studies, respectively. Fifty-five studies were included in the review of the wider literature. Several frameworks and operationalisations of receipt were reported, but the latter were not always consistent with the guiding framework. Receipt was most frequently operationalised in relation to intervention content (16.4%), satisfaction (14.5%), engagement (14.5%), and attendance (14.5%). The majority of studies (90.0%) included subjective assessments of receipt. These relied on quantitative (76.0%) rather than qualitative (42.0%) methods and studies collected data on intervention recipients (50.0%), intervention deliverers (28.0%), or both (22.0%). Few studies (26.0%) reported on the reliability or validity of methods used.ConclusionsReceipt is infrequently addressed in health research and improvements to methods of assessment and reporting are required.Electronic supplementary materialThe online version of this article (doi:10.1186/s12913-016-1904-6) contains supplementary material, which is available to authorized users.

Measures of fidelity of delivery of, and engagement with, complex, face‐to‐face health behaviour change interventions: A systematic review of measure quality

“…Intervention records (e.g., attendance or homework) were also used to measure engagement. Intervention records can be considered an objective measure of receipt (Gearing et al ., 2011; Rixon et al ., 2016) and participation (Saunders, Evans, & Joshi, 2005). However, these measures are limited by their inability to monitor how much participants understand and use the intervention.…”

“…This finding demonstrates that the reporting of psychometric qualities in the complex, face‐to‐face health behaviour change interventions included in this review, may not be as infrequent as previously suggested in different populations (Baer et al ., 2007; Breitenstein et al ., 2010; Maynard et al ., 2013; Rixon et al ., 2016). However, not all studies reported psychometric qualities, and fewer reported implementation qualities, despite the importance of psychometric and implementation qualities (Gearing et al ., 2011; Glasgow et al ., 2005; Holmbeck & Devine, 2009; Lohr, 2002; Stufflebeam, 2000).…”

“…Methods used to assess engagement include self‐report measures (Bellg et al ., 2004; Burgio et al ., 2001; Carroll et al ., 2007), observation of skills (Burgio et al ., 2001), and homework reviews (Bellg et al ., 2004). Systematic reviews of measures used to monitor fidelity of delivery demonstrate that these measures have consistently been used in intervention research, in both educational (Maynard, Peters, Vaughn, & Sarteschi, 2013) and health settings (Rixon et al ., 2016). For example, a review of 55 studies found that intervention receipt was mostly measured by assessing understanding and performance of skills (Rixon et al ., 2016).…”

Developing quality fidelity and engagement measures for complex health interventions

“…Audio-recording all sessions and using multiple researchers to reliably rate a percentage for fidelity is the current gold standard (Lorencatto et al, 2013). To measure engagement, self-report, attendance records, and multiple measures have been used (Gearing et al, 2011;Hankonen et al, 2015;Rixon et al, 2016;Walton et al, 2017). There is currently no consensus regarding the gold standard method to measure engagement in face-to-face interventions (Walton et al, 2017).…”

“…There is currently no consensus regarding the gold standard method to measure engagement in face-to-face interventions (Walton et al, 2017). For example, receipt has been measured most commonly using quantitative measures (Rixon et al, 2016). For example, receipt has been measured most commonly using quantitative measures (Rixon et al, 2016).…”