Interpretation of student behavior in online learning platforms based on log data is complicated by not being able to directly observe the learner. In this paper, we attempt to identify data patterns that signal either guessing on assessment problems or disengaging from the task for students while working through homework modules in an introductory physics class by contrasting data from the general student population with those who completed homework modules in controlled, observed environments. We found that abnormally short problem solving attempts that were previously modeled as a single guessing or answer copying behavior actually consisted of two different types of "guessing" behavior: rapid and strategic guessing. Both types were associated with lower levels of self-confidence, but had different distribution among proctored and unproctored student populations. More importantly, the fraction of rapid guessing increased significantly after campus closure due to COVID-19, but the fraction of strategic guessing remained constant.
For citizen science games (CSGs) to be successful in advancing scientific research, they must effectively train players. Designing tutorials for training can be aided through developing a skill chain of required skills and their dependencies, but skill chain development is an intensive process. In this work, we hypothesized that free recall may be a simpler yet effective method of directly eliciting skill chains. We elicited 23 skill chains from players and developers and augmented our reflexive thematic analysis with 11 semi-structured interviews in order to determine how players and developers conceptualize skill trees and whether free recall can be used as an alternative to more resource-intensive cognitive task analyses. We provide three main contributions: (1) a comparison of skill chain conceptualizations between players and developers and across prior literature; (2) insights to the process of free recall in eliciting CSG skill chains; and (3) a preliminary toolkit of CSG skill-based design recommendations based on our findings. We conclude CSG developers should: give the big picture up front; embrace social learning and paratext use; reinforce the intended structure of knowledge; situate learning within applicable, meaningful contexts; design for discovery and self-reflection; and encourage practice and learning beyond the tutorial. Free recall was ineffective for determining a traditional skill chain but was able to elicit the core gameplay loops, tutorial overviews, and some expert insights.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.