Students’ epistemologies about experimental physics: Validating the Colorado Learning Attitudes about Science Survey for experimental physics

Abstract

Student learning in instructional physics labs represents a growing area of research that includes investigations of students' beliefs and expectations about the nature of experimental physics. To directly probe students' epistemologies about experimental physics and support broader lab transformation efforts at the University of Colorado Boulder (CU) and elsewhere, we developed the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS). Previous work with this assessment has included establishing the accuracy and clarity of the instrument through student interviews and preliminary testing. Several years of data collection at multiple institutions has resulted in a growing national data set of student responses. Here, we report on results of the analysis of these data to investigate the statistical validity and reliability of the E-CLASS as a measure of students' epistemologies for a broad student population. We find that the E-CLASS demonstrates an acceptable level of both validity and reliability on measures of, item and test discrimination, test-retest reliability, partial-sample reliability, internal consistency, concurrent validity, and convergent validity. We also examine students' responses using Principal Component Analysis and find that, as expected, the E-CLASS does not exhibit strong factors (a.k.a. categories).

Developing skills versus reinforcing concepts in physics labs: Insight from a survey of students’ beliefs about experimental physics

“…Some students' responses were also dropped from the data set based on their answer to a filtering question designed to eliminate students who had not read the item prompts (for more information see Ref. [21]). The final data set included N ¼ 4915 valid, matched responses.…”

“…A student's overall E-CLASS score was then given by the sum of their scores on each of the 30 items resulting in a possible range of scores of ½−30; 30 [21]. Statistics describing the validity and reliability of the E-CLASS for the student population included here have been reported elsewhere [20,21]. The distribution of scores on the E-CLASS is typically nonnormal, with students concentrated towards positive scores [21,23].…”

“…Statistics describing the validity and reliability of the E-CLASS for the student population included here have been reported elsewhere [20,21]. The distribution of scores on the E-CLASS is typically nonnormal, with students concentrated towards positive scores [21,23]. To account for this, we determine statistical significance based on the nonparametric Mann-Whitney U test [24] unless otherwise stated.…”

“…As a measure of effect size and practical significance, we also report Cohen's d [25] in cases where differences between distributions were statistically significant [26]. The E-CLASS targets a range of learning goals [8,20], some of which may not be relevant to a specific course, thus we have previously cautioned instructors against focusing exclusively on their students' average overall score when interpreting their results [21]. Instead, we suggest that instructors can and should focus also on individual items that they identify as being most relevant to their particular learning goals.…”

“…The assessment includes items targeting a wide range of learning goals in part because it was intended to be used in both introductory and advanced lab courses. E-CLASS was validated through student interviews and expert review [20], and was tested for statistical validity and reliability using responses from students at multiple institutions and at multiple course levels [21].…”