BackgroundOver the past three decades, the capacity to develop and implement injury surveillance systems (ISS) has grown worldwide and is reflected by the diversity of data gathering environments in which ISS operate. The capacity to evaluate ISS, however, is less advanced and existing evaluation guidelines are ambiguous. Furthermore, the applied relevance of these guidelines to evaluate ISS operating in various settings is unclear. The aim of this paper was to examine how the World Health Organization (WHO) injury surveillance guidelines have been applied to evaluate systems operating in three different contexts.MethodsThe attributes of a good surveillance system as well as instructions for conducting evaluations, outlined in the WHO injury surveillance guidelines, were used to develop an analytical framework. Using this framework, a comparative analysis of the application of the guidelines was conducted using; the Aboriginal Community-Centered Injury Surveillance System (ACCISS) from Canada, the Shantou-Emergency Department Injury Surveillance Project (S-EDISP) from China, and the Yorkhill-Canadian Hospitals Injury Reporting and Prevention Program (Y-CHIRPP) imported from Canada and implemented in Scotland.ResultsThe WHO guidelines provide only a basic platform for evaluation. The guidelines over emphasize epidemiologic attributes and methods and under emphasize public health and injury prevention perspectives requiring adaptation for context-based relevance. Evaluation elements related to the dissemination and use of knowledge, acceptability, and the sustainability of ISS are notably inadequate. From a public health perspective, alternative reference points are required for re-conceptualizing evaluation paradigms. This paper offers an ISS evaluation template that considers how the WHO guidelines could be adapted and applied.ConclusionsFindings suggest that attributes of a good surveillance system, when used as evaluation metrics, cannot be weighted equally across ISS. In addition, the attribute of acceptability likely holds more relevance than previously recognized and should be viewed as a critical underpinning attribute of ISS. Context-oriented evaluations sensitive to distinct operational environments are more likely to address knowledge gaps related to; understanding links between the production of injury data and its use, and the effectiveness, impact, and sustainability of ISS. Current frameworks are predisposed to disassociating epidemiologic approaches from subjective factors and social processes.