Practices and Theories: How Can Machine Learning Assist in Innovative Assessment Practices in Science Education

Abstract

As cutting-edge technologies, such as machine learning (ML), are increasingly involved in science assessments, it is essential to conceptualize how assessment practices are innovated by technologies. To partially meet this need, this article focuses on ML-based science assessments and elaborates on how ML innovates assessment practices in science education. The article starts with an articulation of the "practice" nature of assessment both of learning and for learning, identifying four essential assessment practices: identifying learning goals, eliciting performance, interpreting observations, and decision-making and action-taking. I then extend a three-dimensional framework for innovative assessments, including construct, functionality, and automaticity, and based on which to conceptualize innovative assessments in three levels: substitute, transform, and redefine. Using the framework, I elaborate on how the 10 articles included in this special issue, Applying Machine Learning in Science Assessment: Opportunity and Challenge, advanced our knowledge of the innovations that ML brought to science assessment practices. I contend that the 10 articles exemplify a great deal of effort to transform the four components of assessment practices: ML allows assessments to target complex, diverse, and structural constructs, and thus better approaching the threedimensional science learning goals of the Next Generation Science Standards (NGSS Lead States, 2013); ML extends the approaches used to eliciting performance and collecting evidence; ML provides a means to better interpreting observations and using evidence; ML supports immediate and complex decision-making and action-taking. I conclude this article by pushing the field to consider the underlying educational theories that are needed for innovative assessment practices and the necessities of establishing a "romance" between assessment practices and the relevant educational theories, which I contend are the prominent challenges to forward innovative and ML-based assessment practices in science education.

Distilling vector space model scores for the assessment of constructed responses with bifactor Inbuilt Rubric method and latent variables

“…Chapelle and Voss (2017) remarked that the technological advances in language testing and other natural language-processing evaluations need to show their comparability with other classic psychoeducational tests to improve the current approaches to language assessment (note a similar rationale behind how the term "validity" changed for language assessment in Chapelle, 1999). While there is an important relation between technological advances and language assessment (e.g., Chapelle & Voss, 2016, 2021, it needs to continuously improve the design of computer-assisted language tests and the ways to demonstrate their validity. In this regard, natural language processing (NLP) research and other advances in language testing systematically lack empirical tests of measurement models.…”

“…First, different human raters read the instructional texts and generated ideal summaries. These ideal summaries were then used to extract common and necessary topics from each instructional text by consensus (these conceptual axes have been previously validated in Martínez-Huertas et al, 2018, 2021. Thereafter, an assessment rubric with different conceptual axes was created for each instructional text.…”

“…This method endows the dimensions of the vector space with semantic meanings determined a priori by the designer of the rubric. This method further makes vectors more than a meaningless set of coordinates as it generates comparable and interpretable coordinates in the vector space (Jorge-Botana et al, 2019;Martínez-Huertas et al, 2018, 2021Olmos et al, 2014Olmos et al, , 2016.…”

Can AI be racist? Color‐evasiveness in the application of machine learning to science assessments

“…Created by a series of seemingly invisible and unconscious human decisions, ML's seemingly benign artifice is shot through with critical assumptions about language, race, and power. CHEUK | 827 ML's promise to "revolutionize" assessments by offering efficiency, reliability and "unbiased" interpretations of students' thinking is at odds with how it polices linguistic borders and place learners into proficiency bins (Zhai, 2021). A closer look reveals ML science assessment practices as reflecting a narrow band of what constitutes as appropriate "language of science," privileging academic language over students' everyday nonspecialist vernacular in how they engage with science ideas.…”

“…They are subsequently penalized for making sense of the world around them with their nonnormative yet rich linguistic practices. While champions of these abstract ML black boxes point to gains in efficiency and automaticity (Zhai et al, 2020;Zhai, 2021), it would appear that what is really efficient in the move…”