Thinking beyond the score: Multidimensional analysis of student performance to inform the next generation of science assessments

Cardozo‐Gaibisso, Lourdes; Kim, Seohyun; Buxton, Cory A.; Cohen, Adam S.

doi:10.1002/tea.21611

Cited by 22 publications

(6 citation statements)

References 36 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, developing assessments to measure not only content knowledge, but also the skills, practices, and competencies that are crucial aspects of science learning, would help us understand better whether the differences in students’ interactions has an effect on various outcome measures. Thus far, aligning and assessing the three‐dimensional learning envisioned by the NGSS has been difficult (Cardozo‐Gaibisso, Kim, Buxton, & Cohen, 2019; Fulmer, Tanas, & Weiss, 2018). Particularly, few studies have examined how physical and virtual labs might differentially impact students’ learning of science practices and ability to plan and conduct experiments.…”

Section: Discussionmentioning

confidence: 99%

Supporting middle school students’ science talk: A comparison of physical and virtual labs

et al. 2020

View full text Add to dashboard Cite

Research exploring students’ learning from physical and virtual labs has suggested that on the whole, students learn science content just as well, if not better from virtual labs as they do from physical labs. However, the affordances of physical labs might support the learning of specific skills and competencies that are just as crucial for learning science. In this study, we examined students’ discussions as they worked on physical and virtual labs to better understand how they learned from each, and the kinds of learning that each type of lab supported. One hundred and fifteen 6th grade students from three science teachers’ classes participated in this study. We examined audio data from all available groups as they engaged in physical and virtual labs (n = 14 groups; physical, n = 8 groups; virtual, n = 6 groups). We found that students conducting physical labs engaged in a significantly higher proportion of talk related to setting up apparatus and taking measurements and calculating outputs. Students who performed virtual labs, on the other hand, engaged in significantly more discussions about making predictions and understanding patterns of relationships between variables, and interpreting science phenomena. While students in the Virtual condition engaged in discussions that were more focused on the relationships between science ideas, students in the Physical condition learned science practices related to planning and carrying out investigations that are equally valuable. Our findings suggest that learning from one experimental modality may complement and supplement the relative weaknesses of the other, indicating a need for strategically combining the two. Implications and future directions are discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

Supporting middle school students’ science talk: A comparison of physical and virtual labs

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Rubric-based scores provide useful information regarding students’ knowledge status with respect to the objectives being measured on the test. There is also information about students’ thinking and reasoning as reflected in their answers, however, that can be missed by the rubric-based scores alone ( Cardozo-Gaibisso et al, 2020 ). For example, each topic could represent a set of possible misconceptions ( Shin et al, 2019 ) or writing style.…”

Section: Discussionmentioning

confidence: 99%

Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment

et al. 2021

Self Cite

View full text Add to dashboard Cite

Selected response items and constructed response (CR) items are often found in the same test. Conventional psychometric models for these two types of items typically focus on using the scores for correctness of the responses. Recent research suggests, however, that more information may be available from the CR items than just scores for correctness. In this study, we describe an approach in which a statistical topic model along with a diagnostic classification model (DCM) was applied to a mixed item format formative test of English and Language Arts. The DCM was used to estimate students’ mastery status of reading skills. These mastery statuses were then included in a topic model as covariates to predict students’ use of each of the latent topics in their written answers to a CR item. This approach enabled investigation of the effects of mastery status of reading skills on writing patterns. Results indicated that one of the skills, Integration of Knowledge and Ideas, helped detect and explain students’ writing patterns with respect to students’ use of individual topics.

show abstract

“…Consistent with this call, research has begun to document the affordances of science assessment for cultivating MLs' rich meaning‐making potential. Specifically, studies have examined the potential of codesigning assessment tasks with teachers (e.g., Buxton et al, 2019), conducting multilayered analyses of MLs' responses to those tasks (e.g., Cardozo‐Gaibisso et al, 2020), and interrogating the language ideologies underlying teachers' formative assessment practices in science classrooms (e.g., Lemmi et al, 2019). In this section, we highlight our emerging research on science assessment with MLs that examines innovative approaches to classroom‐based assessment: (a) multimodal, dynamic assessment and (b) translanguaging assessment codesign.…”

Section: Emerging Research In Science Education With Mlsmentioning

confidence: 99%

Science education with multilingual learners: Equity as access and equity as transformation

et al. 2023

View full text Add to dashboard Cite

Research in science education with multilingual learners (MLs) has expanded rapidly. This rapid expansion can be situated within a larger dialogue about what it means to provide minoritized students with an equitable education. Whereas some conceptions of equity focus on ensuring all students have access to the knowledge, practices, and language normatively valued in K‐12 schools (equity as access), increasingly prominent conceptions focus on transforming those knowledge, practices, and language in ways that center minoritized students and their communities (equity as transformation). In this article, we argue that conceptions of equity provide a useful lens for understanding emerging research in science education with MLs and for charting a research agenda. We begin by tracing how conceptions of equity have evolved in parallel across STEM and multilingual education. Then, we provide an overview of recent developments from demographic, theoretical, and policy perspectives. In the context of these developments, we provide a conceptual synthesis of emerging research by our team of early‐career scholars in three areas: (a) learning, (b) assessment, and (c) teacher education. Within each area, we unpack the research efforts in terms of how they attend to equity as access while pushing toward equity as transformation. Finally, we propose a research agenda for science education with MLs that builds on and extends these efforts. We close by offering recommendations for making this research agenda coherent and impactful: (a) being explicit about our conceptions of equity, (b) paying attention to the interplay of structure and agency, and (c) promoting interdisciplinary collaboration.

show abstract

Thinking beyond the score: Multidimensional analysis of student performance to inform the next generation of science assessments

Abstract: This is the author manuscript accepted for publication and has undergone full peer review but has not been through the copyediting, typesetting, pagination and proofreading process, which may lead to differences between this version and the Version of Record. Please cite this article as

Cited by 22 publications

References 36 publications

Supporting middle school students’ science talk: A comparison of physical and virtual labs

Supporting middle school students’ science talk: A comparison of physical and virtual labs

Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment

Science education with multilingual learners: Equity as access and equity as transformation

Contact Info

Product

Resources

About