Defining and Assessing the Value of Canonical Mixed Methods Research Designs in Public Policy and Public Administration

Abstract

Mixed methods research (MMR) designs are well suited for answering policy‐relevant questions, yet they remain underutilized in public policy and public administration scholarship. To provide a deeper understanding of the effective use of such designs, this article examines the prevalence of MMR in public policy and public administration journals, drawing a key distinction between “canonical” and “non‐canonical” MMR. Canonical mixed methods studies are characterized by (1) an explicit rationale for using mixed methods (i.e., a clear connection between methodological decisions and research questions), (2) effective integration of qualitative and quantitative strands, and (3) design transparency. We demonstrate the value of a canonical approach in public policy and public administration research by highlighting differences in quality between canonical and non‐canonical mixed methods studies. Our findings indicate that a canonical approach to mixed methods research makes positive contributions to methodological quality and knowledge development.

How Mixed-Methods Research Can Improve the Policy Relevance of Impact Evaluations

“…In the foregoing, this paper has sought to show how mixed methods can be used to improve impact evaluations, and in this section, we explore the extent to which mixed methods have been used in impact evaluations and what can be learned from such efforts. To identify examples of articles that use mixed methods in impact evaluations, we build on the approach used in Richwine et al (2022) to identify and review research in recent years (2000–2022) in leading public administration, public policy, economics, and evaluation journals by conducting a search using Web of Science. We first present data on the prevalence of mixed methods in impact evaluations for selected journals in the fields of economics, public administration, and evaluation.…”

“…This paper expands on Hendren et al (2018) and Richwine et al (2022) by broadening the search beyond the public administration and public policy journals to include economics and program evaluation journals where impact evaluation studies are likely to be published. To identify articles for our review, in March 2023, we conducted a literature search using Web of Science for 29 public administration and public policy journals from Richwine et al (2022), 6 purposively selected program evaluation journals, and the top 71 economics journal based on the Social Science Citation Index. The search terms we used include mixed methods terms used in prior studies (Hendren et al, 2018; Richwine et al, 2022) and two additional terms to focus on impact evaluations.…”

“…To identify articles for our review, in March 2023, we conducted a literature search using Web of Science for 29 public administration and public policy journals from Richwine et al (2022), 6 purposively selected program evaluation journals, and the top 71 economics journal based on the Social Science Citation Index. The search terms we used include mixed methods terms used in prior studies (Hendren et al, 2018; Richwine et al, 2022) and two additional terms to focus on impact evaluations. Our search strategy, search terms, and full list of included journals are presented in Appendix A and Appendix B.…”

Political attacks and the undermining of the bureaucracy: The impact on civil servants' well‐being

“…Using a sequential mixed methods design (Hanson et al., 2005; Hendren et al., 2018; Richwine et al., 2022) we first conducted an exploratory qualitative study and then a confirmatory quantitative study aimed at corroborating and explaining the main findings of the qualitative phase (especially the relationship between the variables). More specifically, in the first study, we used interviews to explore how civil servants perceive political attacks and how these attacks affect them.…”

How street‐level dilemmas and politics shape divergence: The accountability regimes framework

“…The rationale of our mixed‐methods design is to explore both general relationships and the specific descriptions and explanations of (groups of) cases (Richwine et al, 2022; Seawright, 2016). The qualitative part builds on the quantitative part both in terms of sequence and inferences.…”