The MIXED framework: A novel approach to evaluating mixed‐methods rigor

Abstract

Evaluation of rigor in mixed-methods (MM) research is a persistent challenge due to the combination of inconsistent philosophical paradigms, the use of multiple research methods which require different skill sets, and the need to combine research at different points in the research process. Researchers have proposed a variety of ways to thoroughly evaluate MM research, but each method fails to provide a framework that is useful for the consumer of research. In contrast, the MIXED framework is meant to bridge the gap between an academic exercise and practical assessment of a published work. The MIXED framework (methods, inference, expertise, evaluation, and design) borrows from previously published frameworks to create a useful tool for the evaluation of a published study. The MIXED framework uses an experimental eight-item scale that allows for comprehensive integrated assessment of MM rigor in published manuscripts. Mixed methods are becoming increasingly prevalent in nursing and healthcare research requiring researchers and consumers to address issues unique to MM such as evaluation of rigor.

Appraising the quality of mixed methods research in nursing: A qualitative case study of nurse researchers’ views

“…In other words, no criteria referred to substantive aspects of the nursing discipline or reflected discipline‐specific ways of understanding the research process. This peculiarity is not unique to our study; previous nursing publications on the quality of MMR (Beck & Harrison, ; Eckhardt & Devon, ; Halcomb & Hickman, ; Morse & Niehaus, ) have reported quality criteria that are cross‐disciplinary in nature.…”

“…Accordingly, and in line with the majority of the participants’ views, research quality is universalistic in nature (i.e., an agreed set of criteria is applied across research contexts) and standardized in form (i.e., quality is appraised with structured and close‐ended checklists). In fact, a recent example of this perspective in nursing can be found in Eckhardt and Devon (); the authors proposed the MIXED framework, defined as “a streamlined method of assessing quality in MM research” (p. 2), and throughout their article, the term “rigor”—which is characterized by a “quantitative bias”, according to Davies and Dodd ()—is used as a synonym of “quality”. In addition, although the same authors stated that it was not their intention to “provide an all‐inclusive framework” (p. 2) or “an objective score of MM rigor” (p. 12), they nevertheless adopted a universal perspective on MMR quality on the grounds of the lack of a “single standard” (p. 1).…”

“…To address these challenges, scholars from a variety of disciplines (Collins, Onwuegbuzie, & Johnson, ; Creswell, Klassen, Plano Clark, & Smith, ; Eckhardt & Devon, ; O'Cathain, ) have suggested criteria and frameworks for appraising the quality of MMR studies. These scholars coincide in the view that as MMR involves several unique procedures, and, therefore, faces a series of particular threats, there is a case for developing and using quality criteria that are specific to MMR—in addition to those used in quantitative and qualitative research.…”

“…This observation was confirmed by Fàbregues and Molina‐Azorin (), who reviewed the literature on the quality of MMR published up to February 2016, finding that, from a total of 46 publications suggesting quality criteria or frameworks for MMR, only two were authored by nurse researchers (Halcomb & Hickman, ; Morse & Niehaus, ). Although two additional works in nursing (Beck & Harrison, ; Eckhardt & Devon, ) have been published since the review by Fàbregues and Molina‐Azorin (), the number of nursing contributions to the topic of the quality of MMR continues to be scarce, with only four publications to date. As underlined by Bressan et al.…”

“…As underlined by Bressan et al. () and Eckhardt and Devon (), the need to develop robust, clear, and comprehensive frameworks for appraising the quality of MMR in nursing is pressing.…”