Measuring Learning Gains in Chemical Education: A Comparison of Two Methods

Abstract

Evaluating the effect of a pedagogical innovation is often done by looking for a significant difference in a content measure using a pre–post design. While this approach provides valuable information regarding the presence or absence of an effect, it is limited in providing details about the nature of the effect. A measure of the magnitude of the pre–post change, commonly called learning gain, could provide this additional information to chemical education researchers. In this paper, we compare two methods of measuring learning gains using data from large-scale administrations of the Chemical Concepts Inventory at four universities. The intent of this study is to compare various measures of learning gain, not to contrast the teaching effectiveness at the four universities. In this gain analysis, we introduce a method based on Rasch modeling and discuss the advantages offered by this type of analysis over more commonly used measures of learning gain.

Reviewing affective, behavioural and cognitive learning gains in higher education

“…The "classic" way to measure cognitive learning gains is to assess changes in knowledge or conceptual learning using existing standardized tools at the start and end of a semester (Roohr, Liu, and Liu 2017). This method is particularly popular in the "harder" sciences, where acquisition and application of knowledge are the expected outcome of study (Gok 2012;Pentecost and Barbera 2013;Cahill et al 2014).…”

Introduction of Molecular Methods into a Food Microbiology Curriculum

“…Lecture and lab prescores in the PCR module (28.6% and 28.3%, respectively) lacked the large difference likely as (i) students were more likely exposed to PCR techniques in the prerequisite microbiology coursework and therefore scored higher on PCR lecture preassessments than PFGE prelecture assessments and (ii) the PCR lab assessment questions focused on applying primer design strategies which were not included in the lecture material presented before lab preassessments were given resulting in lower lab prescores (Table , Figure a and d). Mean student prescores from lecture in both units (28.6% PCR, 12.4% PFGE, Table , Figure a), were lower than prescores observed by others in entry‐level biology and chemistry lecture courses where mean prescores were observed at 36.6% and 41.7%, respectively (Smith and Knight ; Pentecost and Barbera ). These comparatively low prescores demonstrate students’ lack of knowledge concerning PCR and PFGE, and support the inclusion of the developed materials to give students working knowledge of molecular techniques important to the food industry.…”

Rasch analysis in physics education research: Why measurement matters

“…Lasry, Guillemette, and Mazur have also found (when analyzing more than 13 000 FCI student answers) that Hake's gain favors students with higher pretest results [51]. Pentecost and Barbera problematized the issue of normalized gain and its nonlinearity and suggested the use of Rasch gain instead [15]. It is obvious from these analyses that the normalized gain should be used with extreme caution.…”