Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National Network

Haudek, Kevin C.; Kaplan, Jennifer J.; Knight, Jennifer K.; Long, Tammy M.; Merrill, John; Munn, Alan; Nehm, Ross H.; Smith, Michelle K.; Urban‐Lurain, Mark

doi:10.1187/cbe.11-03-0019

Cited by 46 publications

(39 citation statements)

References 21 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we are currently developing question stems that integrate the ideas of these nine difficult questions and will use an online format to collect student open-ended responses. Such responses can be categorized with the help of computer programs to pick out patterns of student answers from large groups of students (Haudek et al 2011;Nehm et al 2011). Related studies have revealed unexpected trends in student responses that would not be captured by answers to multiplechoice questions.…”

Section: Genetics Instructors Can Help Students By Handling Misconcepmentioning

confidence: 99%

Using the Genetics Concept Assessment to Document Persistent Conceptual Difficulties in Undergraduate Genetics Courses

Smith

Knight

2012

Genetics

View full text Add to dashboard Cite

To help genetics instructors become aware of fundamental concepts that are persistently difficult for students, we have analyzed the evolution of student responses to multiple-choice questions from the Genetics Concept Assessment. In total, we examined pretest (before instruction) and posttest (after instruction) responses from 751 students enrolled in six genetics courses for either majors or nonmajors. Students improved on all 25 questions after instruction, but to varying degrees. Notably, there was a subgroup of nine questions for which a single incorrect answer, called the most common incorrect answer, was chosen by >20% of students on the posttest. To explore response patterns to these nine questions, we tracked individual student answers before and after instruction and found that particular conceptual difficulties about genetics are both more likely to persist and more likely to distract students than other incorrect ideas. Here we present an analysis of the evolution of these incorrect ideas to encourage instructor awareness of these genetics concepts and provide advice on how to address common conceptual difficulties in the classroom.

show abstract

Section: Genetics Instructors Can Help Students By Handling Misconcepmentioning

confidence: 99%

Using the Genetics Concept Assessment to Document Persistent Conceptual Difficulties in Undergraduate Genetics Courses

Smith

Knight

2012

Genetics

View full text Add to dashboard Cite

show abstract

“…Statistical classification, with or without human scoring, can organize student ideas into related groups. One goal of the AACR group is to develop these assessment resources (questions, categories and scoring models) for STEM disciplines [15], [16]. As this body of work grows, it will be made available to instructors to facilitate rapid feedback and tailor their lessons to address student misunderstandings.…”

Section: Discussionmentioning

confidence: 99%

“…We employ a feature-based approach that involves extracting and categorizing ideas from student responses, giving instructors insight into student thinking. This method can facilitate formative assessment and feedback in large-enrollment courses [15]. AACR has been exploring lexical analysis in evolution, genetics, cell metabolism and acid-base chemistry assessments [15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Examining student constructed explanations of thermodynamics using lexical analysis

Prevost

Haudek

Urban‐Lurain

et al. 2012

2012 Frontiers in Education Conference Proceedings

Self Cite

View full text Add to dashboard Cite

Thermodynamics can be challenging to students, thus improving thermodynamics instruction and assessment is an important area of science and engineering education research. Constructed response assessments can reveal the complexity of students' ideas about thermodynamics. We investigate the use of lexical analysis software for examining students' constructed responses using a group of three questions related to reaction thermodynamics. These questions were administered to students in a large enrollment undergraduate introductory science course and examined learning at two levels of Bloom's Taxonomy: comprehension and application.Our results show that students are able to identify correct statements about thermodynamics in a multiple choice context but fail to construct correct explanations using thermodynamic concepts. Lexical analysis revealed that students who provided correct explanations incorporated more correct concepts and/or made more connections among these concepts than did students with incorrect explanations. Lexical analysis provided insight into student understanding by revealing heterogeneous ideas that were masked in multiple choice versions of the assessment.

show abstract

“…LightSIDE is, at the time of this writing, free for download, which makes it a low-cost utility for this type of analysis [47]. LightSIDE (or its predecessor, SIDE) has been used in prior research that exploits machine learning for computerized assessment of student work [29,30,48,49]. Its choice is justified as a robust, time-efficient tool, well suited to this type of analysis, in some cases outperforming commercial software [48,49].…”

Section: Methodsmentioning

confidence: 99%

Automated analysis of short responses in an interactive synthetic tutoring system for introductory physics

Nakamura

Murphy

Christel

et al. 2016

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Computer-automated assessment of students' text responses to short-answer questions represents an important enabling technology for online learning environments. We have investigated the use of machine learning to train computer models capable of automatically classifying short-answer responses and assessed the results. Our investigations are part of a project to develop and test an interactive learning environment designed to help students learn introductory physics concepts. The system is designed around an interactive video tutoring interface. We have analyzed 9 with about 150 responses or less. We observe for 4 of the 9 automated assessment with interrater agreement of 70% or better with the human rater. This level of agreement may represent a baseline for practical utility in instruction and indicates that the method warrants further investigation for use in this type of application. Our results also suggest strategies that may be useful for writing activities and questions that are more appropriate for automated assessment. These strategies include building activities that have relatively few conceptually distinct ways of perceiving the physical behavior of relatively few physical objects. Further success in this direction may allow us to promote interactivity and better provide feedback in online learning systems. These capabilities could enable our system to function more like a real tutor.

show abstract

Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National Network

Cited by 46 publications

References 21 publications

Using the Genetics Concept Assessment to Document Persistent Conceptual Difficulties in Undergraduate Genetics Courses

Using the Genetics Concept Assessment to Document Persistent Conceptual Difficulties in Undergraduate Genetics Courses

Examining student constructed explanations of thermodynamics using lexical analysis

Automated analysis of short responses in an interactive synthetic tutoring system for introductory physics

Contact Info

Product

Resources

About