Use of systematic literature reviews in Canadian government departments: Where do we need to go?

Abstract

The article reports on ongoing reflections on how to improve the structures and processes by which relevant research findings produced outside Canadian government departments (for example, in universities, think tanks or other research institutions) can be more effectively found, assessed for potential biases, synthesized and disseminated to provide support to government analysts, advisers and decision makers. The focus is on how to structure and routinize research use by government analysts and advisers within a large Canadian department that has a strategic research directorate and many program divisions. Our starting point is the current situation where literature reviews that are produced and used by government analysts and advisers do not correspond to the systematic review standards. We discuss four alternative models. In two models, the whole production process would be controlled from within the department. In the other two models, external actors such as university‐based teams or independent non‐governmental evidence centres would carry the leadership in producing those reviews.

The Systematic Literature Review: Advantages and Applications in Nonprofit Scholarship

“…So, SLRs also force authors to confront their own biases about ''what matters'' and ''what counts''. For these reasons, biased approaches to literature reviews have already been noted as a problem in governmental knowledge synthesis, a field closely allied to voluntary sector studies (Ouimet et al, 2017).…”

“We don’t want to hear about discrimination”: Evidence slippage and evidence invention in the politics of foreign credential recognition in Canada

“…In sum, there is stark disagreement in the literature on the extent to which genuine evidence‐informed policymaking occurs. Nevertheless, most researchers acknowledge that such a practice is more likely under some conditions than under others (Ouimet et al ; Sabatier ; Bedard and Ouimet ; Sá and Hamlin ). These conditions are usually divided into two categories.…”

Lessons from the evidence on evidence‐based policy

“…The context of problems and of policy‐making is critical to the use or otherwise of organized knowledge as evidence; evidence for policy‐making does not have the universal applicability assumed in the scientific ideal (Bocking : 17; Bogenschneider and Corbett : 12, 41‐4; Cartwright and Hardie ; Davies and Nutley , 87; Mitchell : 105; Mulgan : 224; Munro , 60‐4; Ouimet et al : 401‐3; Williams and Glasby ; Wynne : 62‐73); those proponents of EBP who promote randomized controlled trials (RCTs) as the “gold standard” for “what works” in policies and programs create a “hierarchy of evidence” which both oversimplifies the task of providing policy‐relevant evidence and fails to account for the complexities of different policy contexts, such that the external validity of many RCT’s is mistakenly assumed.…”

“…In an important survey of research use, Sandra Nutley, Isabel Walter and Huw Davies note that:

As anyone working in the field of research use knows, a central irony is the only limited extent to which evidence advocates can themselves draw on a robust evidence base to support their convictions that greater evidence use will ultimately be beneficial to public services. Our conclusions…are that we are unlikely any time soon to see such comprehensive evidence neatly linking research, research use, and research impacts, and that we should instead be more modest about what we can attain through studies that look for these (Nutley et al : 271; see also Albæk : 82; Ouimet et al : 401).

…”