Evaluating Oregon's occupational public health surveillance system based on the CDC updated guidelines

Abstract

Background The Oregon Occupational Public Health Program (OOPHP) monitors occupational health indicators (OHIs) to inform occupational safety and health (OSH) surveillance. In 2018, OOPHP evaluated the performance of the OSH surveillance system and identified areas for future improvement. Methods Following the Centers for Disease Control and Prevention (CDC) updated guidelines for evaluating public health surveillance systems, the OOPHP evaluation team engaged internal and external stakeholders using a mixed‐methods approach. Operational measures for ten surveillance attributes were developed. Multiple data collection methods resulted in credible evidence for evaluation conclusions. Analyses included summary statistics and qualitative analysis of interviews, a focus group, and online surveys. Results Twenty stakeholders took part in this evaluation, with an average participation rate of 55%. Results showed the Oregon OSH surveillance system was simple, flexible, and highly accepted by its stakeholders. Funding security presents challenges for stability. A lack of timeliness of OHIs, low relevance of OHIs to local OSH issues, and the system's ineffectual data dissemination all limit the usefulness of the OSH surveillance system. A review of key data sources for the system showed good data quality and predictive value positive, but relatively poor sensitivity and representativeness. Conclusions The evaluation team successfully adapted attributes and examples in the CDC guidelines to this Oregon OSH surveillance evaluation. The evaluation findings have informed the development of recommendations for improvements to OOPHP's OSH surveillance. Future research is needed to develop guidance specific to OSH surveillance evaluation.

Enhancing global health security in Thailand: Strengths and challenges of initiating a One Health approach to avian influenza surveillance

“…CDC Evaluation Guidelines”) framework [ 38 ]. The U.S. CDC Evaluation Guidelines has been used and validated in studies to evaluate surveillance programs across the world, including Brazil [ 39 ], Ghana [ 40 ], Jordan [ 41 ], Yemen [ 42 ], and others [ 43 ]. The U.S. CDC Evaluation Guidelines considers nine metrics: i) simplicity, ii) flexibility, iii) data quality, iv) acceptability, v) sensitivity, vi) predictive value positivity, vii) representativeness, viii) timeliness, and ix) stability.…”

Understanding occupational safety and health surveillance: expert consensus on components, attributes and example measures for an evaluation framework

“…The two guidelines are probably by far the most well-known and the de facto authoritative guidelines in public health surveillance evaluation. The evaluation method established in the two guidelines, i.e., the use of attributes, such as system simplicity, flexibility, acceptability, data sensitivity and quality, to describe and assess characteristics and performance of a surveillance system, have been widely adopted in existing surveillance evaluations [ 27 – 36 ]. It also establishes a framework for many other surveillance evaluation guidelines and frameworks [ 37 – 43 ].…”

“…It also establishes a framework for many other surveillance evaluation guidelines and frameworks [ 37 – 43 ]. However, the CDC guidelines are limited in that they provide only generic recommendations and are insufficient in guiding different types of surveillance systems [ 36 , 44 – 46 ].…”

“…Further, existing guidelines and frameworks are insufficient in terms of providing a comprehensive list of attributes and evaluation questions, as well as practical guidance on selecting, defining and assessing these attributes [ 46 ]. We have reported elsewhere the results and challenges of an OSH surveillance evaluation in Oregon [ 36 ]. We found that it was difficult to find consistent definitions of attributes and evaluation questions in existing references that were appropriate for our system under evaluation.…”

Evaluation of a technical advisory board for an occupational injury surveillance research project: A qualitative study

“…Many private entities, nonprofit organizations, and academic institutions operate advisory boards but little research has been done on the impact of advisory boards and how to evaluate them. A recent effort to improve the evaluation of OSH surveillance systems has been undertaken, 5 but did not specifically address advisory board evaluations. Some literature was available describing an evaluation of community health‐based advisory boards and private entity advisory boards 6,7 .…”