Challenges and opportunities in coding the commons: problems, procedures, and potential solutions in large-N comparative case studies

Abstract

Ongoing efforts to understand the dynamics of coupled socialecological systems and common pool resources have led to the generation of numerous datasets based on a large number of case studies. This data has facilitated the identification of important factors and fundamental principles thereby increasing our understanding of such complex systems. However, the data at our disposal are often not easily comparable, have limited scope and scale, and are based on disparate underlying frameworks which inhibit synthesis, metaanalysis, and the validation of findings. Research efforts are further hampered when case inclusion criteria, variable definitions, coding schema, and intercoder reliability testing are not made explicit in the presentation of research and shared among the research community. This paper first outlines challenges experienced by researchers engaged in a large-scale coding project; highlights valuable lessons learned; and finally discusses opportunities for future comparative case study analyses of social-ecological systems and common pool resources.

An iterative approach to case study analysis: insights from qualitative analysis of quantitative inconsistencies

“…Further, in two of the four cases, there was some disagreement among coders as to whether the case fit the definition of success. This further supports suggestions that case inclusion and exclusion criteria are important and necessary for replicable and robust case-study results (Ostrom et al 1989;Hruschka et al 2004;Ratajczyk et al 2016). For example, in case studies with a high relative abundance of resources, and low market pressures, resource users may not experience CPR dilemmas, and thus, coding for DPs may be irrelevant.…”

“…The work in this and other papers in this special feature set out to address these limitations. Baggio et al (2016) tackles the first, Ratajczyk et al (2016) the second, and this paper tackles the third issue. Ratajczyk et al (2016) describe the coding process at the CBIE.…”

“…Baggio et al (2016) tackles the first, Ratajczyk et al (2016) the second, and this paper tackles the third issue. Ratajczyk et al (2016) describe the coding process at the CBIE. The research team read and re-coded 69 of the 77 case studies from primary and secondary literature used in Cox et al (2010), including the same temporal periods, and tested for intercoder reliability within the research team and between research teams.…”

“…The research team read and re-coded 69 of the 77 case studies from primary and secondary literature used in Cox et al (2010), including the same temporal periods, and tested for intercoder reliability within the research team and between research teams. Our team defined success based on a coding of previously defined (Ostrom et al 1989) variables related to resource sustainability, collective action, and equity among users (see Ratajczyk et al 2016 for a detailed description). Baggio et al (2016) used this case coding to conduct visual analyses and Qualitative Comparative Analysis to examine the interrelationship between DPs and success.…”

Institutions and the performance of coupled infrastructure systems

“…We will close by introducing some research opportunities that emerge from potential applications of the CIS Framework. As discussed in Ratajczyk et al (2016) a critical challenge in coding case studies is a consistent mutual understanding of the qualitative information available. Although we discuss the results in this special issue from the perspective of CPR systems, we began using the CIS view during the project which provided a helpful consistent terminology for our team.…”