Factors affecting learning of vector math from computer-based practice: Feedback complexity and prior knowledge

Abstract

In experiments including over 450 university-level students, we studied the effectiveness and time efficiency of several levels of feedback complexity in simple, computer-based training utilizing static question sequences. The learning domain was simple vector math, an essential skill in introductory physics. In a unique full factorial design, we studied the relative effects of "knowledge of correct response" feedback and "elaborated feedback" (i.e., a general explanation) both separately and together. A number of other factors were analyzed, including training time, physics course grade, prior knowledge of vector math, and student beliefs about both their proficiency in and the importance of vector math. We hypothesize a simple model predicting how the effectiveness of feedback depends on prior knowledge, and the results confirm this knowledge-by-treatment interaction. Most notably, elaborated feedback is the most effective feedback, especially for students with low prior knowledge and low course grade. In contrast, knowledge of correct response feedback was less effective for low-performing students, and including both kinds of feedback did not significantly improve performance compared to elaborated feedback alone. Further, while elaborated feedback resulted in higher scores, the learning rate was at best only marginally higher because the training time was slightly longer. Training time data revealed that students spent significantly more time on the elaborated feedback after answering a training question incorrectly. Finally, we found that training improved student self-reported proficiency and that belief in the importance of the learned domain improved the effectiveness of training. Overall, we found that computer based training with static question sequences and immediate elaborated feedback in the form of simple and general explanations can be an effective way to improve student performance on a physics essential skill, especially for less prepared and low-performing students.

Framework and implementation for improving physics essential skills via computer-based practice: Vector math

“…The development of the materials for the online vector essential skills units is a continuation of our recent work on training students on vector skills in an education research lab setting [3]. Further, we are continuing work on essential skills units for a course on introductory materials science engineering [4,5].…”

“…Here we will base the practice and feedback format on results from our recent clinical study which investigates the relative effectiveness of various forms of feedback on the same population as the current study (students enrolled in calculus-based introductory physics) and the same topic, namely, vector math [3]. Following those results, the method used in the current study provides immediate feedback on the correctness of each response and also makes available a general explanation (via clicking a button if the student chooses to) of how to answer a particular question type.…”

“…First, most of the items were based on similar simple skills questions in textbooks and validated assessment instruments [3,48,49]. Note that the Test of Understanding Vectors [48] overlaps significantly with our vector skills assessment, but the former is aimed more at assessing conceptual understanding, and the latter is more aimed at assessing accuracy in vector math skills.…”

“…Note that the Test of Understanding Vectors [48] overlaps significantly with our vector skills assessment, but the former is aimed more at assessing conceptual understanding, and the latter is more aimed at assessing accuracy in vector math skills. Second, many of the items had also been pilot tested and refined through testing in our education research lab, including student interviews [3,27,28]. Third, pretest assessment scores were significantly correlated with grade in the course (r ¼ 0.36, 0.32; p 0 s < 0.001) for experiments 1 and 2, respectively.…”

Comparing student behavior in mastery and conventional style online physics homework

“…By enabling students to spend different amounts of time and effort on learning, it is proposed that significantly more students in a mastery learning condition will achieve the desired level of mastery. There have been several recent attempts at implementing mastery style online homework in introductory physics courses with encouraging outcomes [4,6,7].…”