Facial micro‐expression states as an indicator for conceptual change in students' understanding of air pressure and boiling points

Abstract

Utilizing facial recognition technology, the current study has attempted to predict the likelihood of student conceptual change with decision tree models based on the facial micro-expression states (FMES) students exhibited when they experience conceptual conflict. While conceptual change through conceptual conflicts in science education is a well-studied field, there is little research done on conceptual change through conceptual conflict in terms of students' facial expressions. As facial expressions are one of the most direct and immediate responses one can get during instruction and that facial expressions are often representations student's emotions, a link between students' FMES and learning was explored. Facial data was collected from 90 tenth graders. Only data from the 72 students who made incorrect predictions were analyzed in this study. The concept taught was the relationship between boiling point and air pressure. Through facial recognition software analysis and decision tree models, the current study found Surprised, Sad and Disgusted to be key FMES that could be used to predict student conceptual change in a conceptual conflict-based scenario.

The Influence of Academic Emotions on Learning Effects: A Systematic Review

“…Especially in scientific experiments, we find that when the experimental results are revealed, the dominant expression of students is first surprised, then negative, and the probability of knowledge change is higher [ 47 ]. Surprise, sadness, and disgust are also key facial expressions used to predict the change in students’ knowledge based on conflicting scenarios [ 48 ].…”

The Promise and Pitfalls of Self-report

“…This framework fits the current era as educational research is flooded with new (Horizontal) means of measuring students' cognitive processing (e.g., eye tracking; Chaulic et al 2020) and meta-cognitive processing (e.g., trace data; Rogiers, et al, 2020). This Horizontal drive for innovation of measurement continues to push into complex areas such as emotion (facial recognition ;Chiu, et al, 2019;Dingle, et al, 2016;skin conductance, Järvenoja, et al 2018;Lehikoinen et al, 2019) and motivation (neuroscience; Hidi, 2016;Mayer, 2017). The considerable momentum behind this drive for alternatives to self-report measurement arise in large part with a longstanding dissatisfaction with their intra-psychic nature and the general lack of Lateral development in these measures.…”

Relationships between Facial Expressions, Prior Knowledge, and Multiple Representations: a Case of Conceptual Change for Kinematics Instruction

“…In particular, in a counterintuitive scientific experiment demonstration, it was found that if the students' dominant expression was surprised followed by negative expressions when the result of the experiment was revealed, there will be a higher chance of conceptual change (Chiu, Yu, Liaw, & Lin, 2015). Surprised, sad, and disgusted were also identified as key facial expressions in the prediction of student conceptual change in a conceptual conflict-based scenario (Chiu et al, 2019). Based on the findings of the above literature, the current study has taken one step further by examining the meaning of different types of facial microexpression states (FMES), i.e., facial expressions that often last only a few micro-seconds and difficult for the naked eyes to capture, in terms of prior knowledge and multiple representation.…”

“…FaceReader's analysis is based on Ekman's seven universal facial expression states (Ekman, 1970), and the facial expressions were categorized into positive (happy), negative (sad, angry, scared, and disgusted), surprised, and neutral. It is a commercially available software that has been used in various disciplines and research (e.g., Chiu et al, 2014;Chiu et al, 2019;Lewinski, den Uyl, & Butler, 2014;Liaw et al, 2014;Maison & Pawłowska, 2017;Skiendziel, Rösch, & Schultheiss, 2019;Xie, Yu, Niu, & Li, 2019). The three key segments of students' facial expression videos mentioned above (i.e., E, T, and A) were the explanatory variables.…”

“…However, it is also time consuming, much like classroom observation (Ubben, Salisbury, & Daniel, 2019). Nevertheless, in recent years, students' facial microexpression states (FMES) were revealed as a viable real-time predictor of students' performance in a conceptual conflict-based science learning scenario (Chiu, Chou, Wu, & Liaw, 2014;Chiu, Liaw, Yu, & Chou, 2019;Matsumoto & Hwang, 2011). Such a revelation is offering researchers a much needed alternative approach when they wish to gather real-time and direct reactions from learners.…”