To draw or not to draw? Examining the necessity of problem diagrams using massive open online course experiments

Abstract

Previous research on problem diagrams suggested that including a supportive diagram, one that does not provide necessary problem solving information, may bring little, or even negative, benefit to students' problem solving success. We tested the usefulness of problem diagrams on 12 different physics problems (6A/B experiments) in our massive open online course. By analyzing over 8000 student responses in total, we found that including a problem diagram that contains no significant additional information only slightly improves the first attempt correct rate for the few most spatially complex problems, and has little impact on either the final correct percentage or the time spent on solving the problem. On the other hand, in half of the cases, removing the diagram significantly increased the fraction of students' drawing their own diagrams during problem solving. The increase in drawing behavior is largely independent of students' physics abilities. In summary, our results suggest that for many physics problems, the benefit of a diagram is exceedingly small and may not justify the effort of creating one.

Evaluating the effectiveness of two methods to improve students' problem solving performance after studying an online tutorial

“…The current study demonstrated two such methods using OLM sequences and analysis of student log data. The first is a controlled AB experiment conducted in a single course, similar to the design in two previous studies [5,23]. The second method involves implementing improved instructional design in the same class for consecutive years.…”

Comparing student behavior in mastery and conventional style online physics homework

“…We employed a flipped online AB experiment design similar to two earlier studies [8,9]. Two versions of online homework were created for two chapters of a calculus-based introductory physics course, covering the topics of mechanical energy and linear momentum.…”

Role of diagrams in problem solving: An evaluation of eye-tracking parameters as a measure of visual attention

“…Question Interaction or a small positive effect of supportive diagrams on problem solving [7]. The reason why the diagram in question Q3 had a positive effect on students' percentages of correct answers could be that it emphasized the initial equal elevations of the bricks (see the Supplemental Material [30]).…”

“…More research is needed to determine the role of diagrams in problem solving. A recent large-scale study on usefulness of supportive diagrams has shown their small positive effect on students' scores [7].…”