Best Practices for Administering Concept Inventories

Abstract

There is a plethora of concept inventories available for faculty to use, but it is not always clear exactly why you would use these tests, or how you should administer them and interpret the results. These research-based tests about physics and astronomy concepts are valuable because they allow for standardized comparisons among institutions, instructors, or over time. In order for these comparisons to be meaningful, you should use best practices for administering the tests. In interviews with 24 physics faculty 1 , we have identified common questions that faculty members have about concept inventories. We have written this article to address common questions from interviews and provide a summary of best practices for administering concept inventories.

Investigating students’ seriousness during selected conceptual inventory surveys

“…From the early days of RBAIs [31], researchers and instructors have raised concerns about whether students might not make a serious attempt at answering the questions on a conceptual-inventory RBAI, such as the FCI, CSEM, or BEMA correctly [23,[32][33][34][35]. In order for instructors and researchers to appropriately evaluate RBAI data, it is useful to know what proportion of students in a population are taking that RBAI seriously.…”

Classroom features supporting free-choice transfer

“…I was so proud of our group to turn out correct! -Maddy Physics Education Research has made great strides in improving students' understanding of physics content by operationalizing "understanding" [1], creating curricula that promote gains in understanding, and developing common design principles for that curriculum [2]. However, this improved understanding is of little value if students do not draw on their knowledge in non-instructional contexts.…”

Study of Pre- and Post-Course Knowledge Surveys in an Engineering Economy Course

“…Those types of tests also help distinguishing between learning and performance 6 . In addition, monitoring the results of pre and post concept inventories allows instructors to make comparisons among the effectiveness of their teaching over time and possibly in different environments and across different institutions 4 . Even though concept inventory tests are a good tool, it is suggested to not be used to replace exams, homework assignments or other assignments that demonstrate students' math skills 4 .…”

“…The question types used in assessment methods can be multiple-choice, short answer, open-ended, essays, matching, and true or false and can be incorporated in written reflections, quizzes, assignments, and others 2 . A commonly accepted assessment instrument used for both diagnostic and formative purposes is concept inventories 3 , which refer to any kind of research-based assessment techniques that measure conceptual understanding 4 . Using concept inventories helps instructors measure the effectiveness of their teaching 4 and determines if students have the correct understanding of important concepts on a topic.…”

“…A commonly accepted assessment instrument used for both diagnostic and formative purposes is concept inventories 3 , which refer to any kind of research-based assessment techniques that measure conceptual understanding 4 . Using concept inventories helps instructors measure the effectiveness of their teaching 4 and determines if students have the correct understanding of important concepts on a topic. This is particularly important and effective when students come to learn a subject with an established commonsense of the material 5 because that commonsense may not always be the right knowledge and it is better if the instructor is aware about the misconceptions from the start of the course.…”