Evaluation of the influence of wording changes and course type on motivation instrument functioning in chemistry

Abstract

Increased understanding of the importance of the affective domain in chemistry education research has led to the development and adaptation of instruments to measure chemistry-specific affective traits, including motivation. Many of these instruments are adapted from other fields by using the word ‘chemistry’ in place of other disciplines or more general ‘science’ wording. Psychometric evidence is then provided for the functioning of the new adapted instrument. When an instrument is adapted from general language to specific (e.g.replacing ‘science’ with ‘chemistry’), an opportunity exists to compare the functioning of the original instrument in the same context as the adapted instrument. This information is important for understanding which types of modifications may have small or large impacts on instrument functioning and in which contexts these modifications may have more or less influence. In this study, data were collected from the online administration of scales from two science motivation instruments in chemistry courses for science majors and for non-science majors. Participants in each course were randomly assigned to view either the science version or chemistry version of the items. Response patterns indicated that students respond differently to different wordings of the items, with generally more favorable response to the science wording of items. Confirmatory factor analysis was used to investigate the internal structure of each instrument, however acceptable data-model fit was not obtained under any administration conditions. Additionally, no discernable pattern could be detected regarding the conditions showing better data-model fit. These results suggest that even seemingly small changes to item wording and administration context can affect instrument functioning, especially if the change in wording affects the construct measured by the instrument. This research further supports the need to provide psychometric evidence of instrument functioning each time an instrument is used and before any comparisons are made of responses to different versions of the instrument.

Is This Science? Students’ Experiences of Failure Make a Research-Based Course Feel Authentic

“…, 2018b ). The omega reliability coefficient is equivalent to Cronbach’s alpha when factor loadings are equivalent but avoids bias introduced by Cronbach’s alpha when factor loadings are independent ( Komperda et al. , 2018a , b).…”

The role of motivation on secondary school students’ causal attributions to choose or abandon chemistry

“…Reliability of the CAQ. Given the congeneric nature of the structural equation models employed in the CFA, reliability was analyzed using the statistic omega (Komperda et al, 2018). Regarding the CAQ for PC students, the omegas were: 0.914 for attributions to the teacher and classroom methodology, 0.858 for the attributions to the students' perceived difficulty of the subject, 0.774 for the attributions to the subject's relationship with mathematics, 0.753 for the attributions to the effect of media, 0.858 for the attributions to the effect of the family, and 0.702 for the attributions to the effect of friends.…”

“…Fourth, given the gender gap found in previous investigations in the field (Stokking, 2000;Brotman and Moore, 2008;Sheldrake et al, 2017;Pérez-Bitrián, 2018, 2019;Ardura and Galán, 2019), future research on the effect of sex on students' causal attributions should be tackled. Finally, the factor structure of the SMQII has recently been questioned (Komperda et al, 2018). For this reason, future investigations using different samples should address this problem.…”

Design and Validation of an Instrument to Measure Students’ Interactions and Satisfaction in Undergraduate Chemistry Laboratory Classes

“…For scale reliability, the analysis process was as follows: firstly, results from the factor analysis were analysed to check whether the items were unidimensional. Consequently, an overall internal consistency value or a single-administration reliability coefficient for each interaction scale was calculated (Komperda, Hosbein, and Barbera 2018). Secondly, to determine which coefficient was appropriate in this study, a single-factor CFA model for each interaction scale was analysed using the congeneric and tau-equivalent models (Komperda, Pentecost, and Barbera 2018).…”

“…Furthermore, while small changes in the wordings of instruments can influence the results, the applicability of this instrument needs to be further validated in other institutions and with a broader range of students (Komperda, Hosbein, et al 2018). Additionally, in this study, quantitative data collection was prioritised; it is suggested that further qualitative data collection and analysis method be performed to test the trustworthiness of the description of interactions in individual science and engineering laboratories (Luckay and Laugksch 2015).…”