The In-Task Assessment Framework for Behavioral Data

Kerr, Deirdre; Andrews, Jessica J.; Mislevy, Robert J.

doi:10.1002/9781118956588.ch20

Cited by 13 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, psychometricians might select different combinations of features represented in the proposed heuristic to accurately measure the complex integration of comprehension skills and practices required in particularized digital reading environments. In turn, recent advances in technology can be leveraged to capture and track detailed evidence of both content understanding and targeted comprehension processes in digital environments (see, e.g., Coiro et al, 2019; Kerr, Andrews, & Mislevy, 2017).…”

Section: A Multifaceted Heuristic To Characterize the Complexity Of Dmentioning

confidence: 99%

Toward a Multifaceted Heuristic of Digital Reading to Inform Assessment, Research, Practice, and Policy

Coiro

2020

Reading Research Quarterly

111

View full text Add to dashboard Cite

In this commentary, the author explores the tension between almost 30 years of work that has embraced increasingly complex conceptions of digital reading and recent studies that risk oversimplifying digital reading as a singular entity analogous with reading text on a screen. The author begins by tracing a line of theoretical and empirical work that both informs and complicates our understanding of digital literacy and, more specifically, digital reading. Then, a heuristic is proposed to systematically organize, label, and define a multifaceted set of increasingly complex terms, concepts, and practices that characterize the spectrum of digital reading experiences. Research that informs this heuristic is used to illustrate how more precision in defining digital reading can promote greater clarity across research methods and advance a more systematic study of promising digital reading practices. Finally, the author discusses implications for assessment, research, practice, and policy.

show abstract

Section: A Multifaceted Heuristic To Characterize the Complexity Of Dmentioning

confidence: 99%

Toward a Multifaceted Heuristic of Digital Reading to Inform Assessment, Research, Practice, and Policy

Coiro

2020

Reading Research Quarterly

111

View full text Add to dashboard Cite

show abstract

“…Specifically, there are challenges with respect to conceptualizing what skills make up CPS and making sense of the large streams of log data. D. Kerr, Andrews, and Mislevy (2016) have developed an ECD-based framework that provides technological support for instantiating the student, evidence, and task models when seeking to assess proficiency in complex skills derived from high-granularity log data. The in-task assessment framework (I-TAF) reconceptualizes what it means to define the construct of interest, extract evidence from learner performances, and link the evidence to the construct of interest when the evidence is high-granularity log data of individuals' actions and discourse rather than standard item response data.…”

Section: Evidence-centered Design In Tracking Of Actions and Conversamentioning

confidence: 99%

Advancing the Science of Collaborative Problem Solving

Graesser

Fiore

Greiff

et al. 2018

Psychol Sci Public Interest

223

171

View full text Add to dashboard Cite

Collaborative problem solving (CPS) is an important 21st century skill that is increasingly recognized as being critical to efficiency, effectiveness, and innovation in the modern global economy (Fiore, Graesser, & Greiff, 2018; Organisation for Economic Cooperation and Development [OECD], 2017a). CPS has attracted interest in international assessments; national assessments of middle and high school students; and training in colleges, industry, and the military (Care,

show abstract

“…The assessment development team consisted of a very diverse group of subject‐matter experts including educational and occupational researchers, a community college department head, and a psychometrician. They utilized the in‐task assessment framework (I‐TAF; Andrews‐Todd & Kerr, 2019), which promotes the use of an ontology (Kerr et al, 2016), to graphically depict the conceptual structure of a content domain and the associated relationships therein. The team developed the BEK assessment after evaluating its alignment with the content knowledge needed to complete the existing simulation‐based electronics task.…”

Section: Methodsmentioning

confidence: 99%

The Development of a Content Assessment of Basic Electronics Knowledge

Steinberg

Andrews‐Todd

Forsyth

et al. 2020

ETS Research Report Series

View full text Add to dashboard Cite

This study discusses the development of a basic electronics knowledge (BEK) assessment as a pretest activity for undergraduate students in engineering and related fields. The 28 BEK items represent 12 key concepts, including properties of serial circuits, knowledge of electrical laws (e.g., Kirchhoff's and Ohm's laws), and properties of digital multimeters. This paper first discusses a psychometric evaluation of the BEK assessment to understand its basic measurement properties and to examine various group‐level differences based on demographic, institutional, and instructor characteristics. Subsequently, the relationship between BEK scores on the 23 retained items and performance on an existing complex collaborative simulation‐based electronics task is discussed. Results demonstrated that basic content knowledge alone may not be sufficient for students to demonstrate knowledge of electronics skills on more complex tasks. The research also carries great importance given ongoing concerns about improving the overall state and diversity of the engineering workforce and its associated pipeline to meet the demands of the national economy.

show abstract

The In-Task Assessment Framework for Behavioral Data

Cited by 13 publications

References 26 publications

Toward a Multifaceted Heuristic of Digital Reading to Inform Assessment, Research, Practice, and Policy

Toward a Multifaceted Heuristic of Digital Reading to Inform Assessment, Research, Practice, and Policy

Advancing the Science of Collaborative Problem Solving

The Development of a Content Assessment of Basic Electronics Knowledge

Contact Info

Product

Resources

About