Beyond a coefficient: an interactive process for achieving inter-rater consistency in qualitative coding

Abstract

Though qualitative research has become more prevalent in practice over the last 30 years, there is still considerable uncertainty among researchers regarding how to ensure inter-rater consistency when teams are tasked with coding qualitative data. In this article, we offer an explanation of a methodology that our qualitative team used to achieve systematic coding of our dataset in a way that preserved the contextual, subjective nature of the data, lent itself to the deductive and inductive creation of a layered codebook, and ensured consistent application of the codebook to varied types of data. This methodology prepared us to draw logical and substantiated conclusions during subsequent analyses; hence, the process serves as a welcome addition to the literature on consistently coding qualitative data in a manner that honors its defining characteristics.

Affordability, Feasibility, and Accessibility: Companion Animal Guardians with (Dis)Abilities’ Access to Veterinary Medical and Behavioral Services during COVID-19

“…Two research team members, with expertise in disaster and emergency management and disability social work respectively, coded all the interview transcripts independently and collaboratively elaborated themes through two rounds of data analysis, namely deductive (top-down) and inductive (bottom-up) strategies [65]. Combining both deductive and inductive strategies guarantees inter-rater consistency when the two members code the same data [66].…”

Mental health clinician training and experiences with utilization of advance statements in Victoria, Australia

“…1). We used this process to reach agreement between preliminary codes, and to ensure inter-rater reliability when completing further stages of data coding (Hemmler et al 2020).…”

The impact of COVID‐19 on alcohol and other drug nurses' provision of care: A qualitative descriptive study

“…Both authors independently became familiar with the data through reading transcripts and revisiting audio recordings, developing a preliminary coding matrix. To ensure inter-rater reliability and agreement in codes developed, preliminary themes were reviewed between researchers and coding decisions explained and justified following the methods of Hemmler et al, (2020). The authors then completed the following phases (searching, reviewing, defining and naming themes, and preparing the report) cooperatively to achieve consensus and agreement of themes and subthemes.…”