Can an electronic monitoring system capture implementation of health promotion programs? A focussed ethnographic exploration of the story behind program monitoring data

Abstract

Background: There is a pressing need for policy makers to demonstrate progress made on investments in prevention, but few examples of monitoring systems capable of tracking population-level prevention policies and programs and their implementation. In New South Wales, Australia, the scale up of childhood obesity prevention programs to over 6000 childcare centres and primary schools is monitored via an electronic monitoring system, "PHIMS". Methods: Via a focussed ethnography with all 14 health promotion implementation teams in the state, we set out to explore what aspects of program implementation are captured via PHIMS, what aspects are not, and the implications for future IT implementation monitoring systems as a result. Results: Practitioners perform a range of activities in the context of delivering obesity prevention programs, but only specific activities are captured via PHIMS. PHIMS thereby defines and standardises certain activities, while noncaptured activities can be considered as "extra" work by practitioners. The achievement of implementation targets is influenced by multi-level contextual factors, with only some of the factors accounted for in PHIMS. This evidences incongruencies between work done, recorded and, therefore, recognised.

Scale-up of prevention programmes: sustained state-wide use of programme delivery software is explained by normalised self-organised adoption and non-adoption

“…This was problematic because PHIMS data could give a false impression of success as a site could appear to be going really well (according to the practice changes recorded) but actually be completely disengaged from the program. We describe this in a separate paper [ 22 ].…”

“…The study we report on here is a part of a larger investigation of this question [ 18 ] within a university-policy-maker-practitioner partnership centre [ 19 ]. Earlier papers report on the data recording methods that have grown up alongside PHIMS [ 20 ]; the style of practice we observed [ 21 ]; the breadth and intensity of work to achieve a change in nutrition or physical activity policy/practice in a site [ 22 ]; the role of programme materials [ 23 ]; the co-production methods within our team [ 24 , 25 ]; and reactions from PHIMS designers and administrators to a preliminary “rich picture” of how PHIMS might work as a system [ 26 ].…”

The Weight of Words: Co-Analysis of Thick Ethnographic Description and “Friction” as Methodological Strategies in a Health Policy Research Partnership

“…We undertook an ethnographic study of PHIMS use among health promotion practitioners working in local health districts. The commitment was to co-designed research with policymakers and collaborative write-up of the findings (Conte et al, 2019, 2020; Grøn et al, 2020; Hawe et al, 2020) in the form of published papers, internal reports, conference presentations, and participant feedback. The partnership also provided the basis to steward literature synthesis and experiments with graphic forms of data presentation (Conte & Davidson, 2020; Conte & Hawe, 2018).…”

“…The interactive presentation showed how many quotes were coded under each theme and subtheme from the data set. In a number of the co-authored publications, the ethnographic fieldnotes were coded and analyzed through the extraction of patterns and generalizations (Conte et al, 2019, 2020; Grøn et al, 2020). However, these approaches to ethnographic analysis represented only one way of understanding ethnographic fieldnotes.…”

“…The research event we present draws on a tape-recorded meeting that was organized to actively engage all members of the health policy research partnership in developing one of the manuscripts for publication in a social science journal. In contrast to other co-authored publications developed by the team (Conte et al, 2019, 2020; Grøn et al, 2020), the ethnographic material presented in this manuscript was underpinned by a case-based, situational approach (Greenhalgh et al, 2011; Pedersen, 2017). That is, instead of attending mostly to facilitators, barriers, and the diversity of implementation adaptations of PHIMS across the data set, the goal was to develop more fine-grained understandings of the meanings and logics underpinning the use of PHIMS as social practice (Greenhalgh et al, 2011; Tsoukas, 2017).…”