Shared gaze visualizations, where the real-time gaze location of group members is shared with one another, have become increasingly studied over the last decade by HCI researchers. Shared gaze studies so far have found improved outcomes and better collaboration for peer collaborators. Less is known, however, about how gaze sharing may aid learners and instructors. In our study, an instructor teaches a learner how to assemble and program a simple microcontroller, communicating either through a webcam feed (webcam condition), a field-of-view video feed (HMC condition), or a field-of-view video feed with a gaze location pointer (gaze condition). We find that learning gain is highest in the gaze condition, especially for low achievers. Moreover, instructors predict learner post-test scores more accurately with gaze visualizations, suggesting gaze sharing can help instructors track the cognitive state of the learner. This effect was also most salient for low achievers. We find that in the HMC condition that only lacked this single dot, many of the benefits for both learning and teaching were lost. The paper concludes with discussions on how gaze visualization may support learning and teaching, and on the tool's limitations and conditions for usefulness.
This paper reviews 74 empirical publications that used high-frequency data collection tools to capture facets of small collaborative groups—i.e., papers that conduct Multimodal Collaboration Analytics (MMCA) research. We selected papers published from 2010 to 2020 and extracted their key contributions. For the scope of this paper, we focus on: (1) the sensor-based metrics computed from multimodal data sources (e.g., speech, gaze, face, body, physiological, log data); (2) outcome measures, or operationalizations of collaborative constructs (e.g., group performance, conditions for effective collaboration); (3) the connections found by researchers between sensor-based metrics and outcomes; and (4) how theory was used to inform these connections. An added contribution is an interactive online visualization where researchers can explore collaborative sensor-based metrics, collaborative constructs, and how the two are connected. Based on our review, we highlight gaps in the literature and discuss opportunities for the field of MMCA, concluding with future work for this project.
While the majority of social scientists still rely on traditional research instruments (e.g., surveys, self-reports, qualitative observations), multimodal sensing is becoming an emerging methodology for capturing human behaviors. Sensing technology has the potential to complement and enrich traditional measures by providing high frequency data on people’s behavior, cognition and affects. However, there is currently no easy-to-use toolkit for recording multimodal data streams. Existing methodologies rely on the use of physical sensors and custom-written code for accessing sensor data. In this paper, we present the EZ-MMLA toolkit. This toolkit was implemented as a website and provides easy access to multimodal data collection algorithms. One can collect a variety of data modalities: data on users’ attention (eye-tracking), physiological states (heart rate), body posture (skeletal data), gestures (from hand motion), emotions (from facial expressions and speech) and lower-level computer vision algorithms (e.g., fiducial/color tracking). This toolkit can run from any browser and does not require dedicated hardware or programming experience. We compare this toolkit with traditional methods and describe a case study where the EZ-MMLA toolkit was used by aspiring educational researchers in a classroom context. We conclude by discussing future work and other applications of this toolkit, potential limitations and implications.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.