All research, e.g., qualitative or quantitative, is concerned with the extent to which analyses can adequately describe the phenomena it seeks to describe. In qualitative research, we use internal validity checks like intercoder agreement to measure the extent to which independent researchers observe the same phenomena in data. Researchers report indices of agreement to serve as evidence of consistency and dependability of interpretations, and we do so to make claims about the trustworthiness of our research accounts. However, few studies report the methods of how multiple analysts developed alignment in their interpretation of data, a process that undergirds accounts of consistency, dependability, and trustworthiness. In this article, we review the issues and options around achieving intercoder agreement. Drawing on our experience from a longitudinal, team-based research project that required rapid cycles of qualitative data analysis, we reflect on the challenges we had achieving high intercoder agreement (which refer to as the perils). It was through these challenges that we developed a method that helps to foster shared ways of seeing data, and thus alignment in our interpretations of phenomena in data. In this article, we present this method as a tool for dyadic and team-based qualitative data analysis to facilitate reliable and consistent high-inference interpretations of data with multiple analysts.
Video and co-design can be powerful tools to enrich problem-based learning experiences. We explore how a teacher and researcher engaged in co-design of a PBL experience focused on human-centered robotics as well as the resulting design. They explored the question “How can we design a robot that serves a need in our local community?” We highlight three aspects of the most recent iteration of our PBL curriculum that we have identified as central to its success. These three elements include: 1) co-design experiences that occurred before and during unit implementation, 2) the use of shared video viewing and analysis both in co-design and with student groups in the classroom, and 3) the bringing of local stakeholders into the classroom to work closely with students. These three aspects of our curriculum are positioned here as takeaways for researchers and educators working to design, implement, and study PBL.
Acknowledgments: This research was supported by the National Science Foundation through ITEST grant #1433414. Any opinions, findings, conclusions, or recommendations expressed in this report are those of the authors, and do not necessarily represent the official views, opinions, or policy of the National Science Foundation. A special thanks to Dr. Selma Šabanovic, principal investigator, and Dr. Matt Francisco for their contributions to this work.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.