On the Roles of External Knowledge Representations in Assessment Design

Abstract

Abstract:People use external knowledge representations (KRs) to create, identify, depict, transform, store, share, and archive information. Learning to work with KRs is central to becoming proficient in virtually every discipline. As such, KRs play central roles in curriculum, instruction, and assessment. We describe five key roles of KRs in assessment:1. An assessment is itself a KR, which makes explicit the knowledge that is valued, ways it is used, and standards of good work.2. The analysis of any domain in which learning is to be assessed must include the identification and analysis of the KRs in that domain.3. Assessment tasks can be structured around the knowledge, relationships, and uses of domain KRs.4. "Design KRs" can be created to organize knowledge about a domain in forms that support the design of assessment.5. KRs in the discipline of assessment design can guide and structure domain analyses (re #2), task construction (re #3), and the creation and use of design KRs (re #4).The third and fourth roles are developed in greater detail, through an "evidence-centered" design perspective that reflects the fifth role. Recurring implications of technology that leverage the impact of KRs in assessment are highlighted, including task design supports and automated task construction and scoring. Ideas are illustrated with "generate examples" tasks and simulation tasks for computer network design and troubleshooting.

Examining the influences of epistemic beliefs and knowledge representations on cognitive processing and conceptual change when learning physics

“…One aspect of the learning content that has received little attention in the epistemic beliefs literature to date is knowledge representation. Mislevy and colleagues (Mislevy et al, 2010) define knowledge representation, broadly speaking, as the way in which information about the world is represented, differentiating between internal knowledge representation, or the way in which we represent knowledge in our brains, and external knowledge representation, or "a physical or conceptual structure that depicts entities and relationships in some domain, in a way that can be shared among different individuals or the same individual at different points in time" (Mislevy et al, 2010, p. 4). Moreover, in their framework outlining various aspects of the epistemic climate, Haerle and Bendixen (2008) highlight textbooks, curricula, and assessments as examples of knowledge representations that could be explored in the context of epistemic beliefs research.…”

What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities

“…Additionally, multiple forms of knowledge representations such as diagrams, animations, maps, graphs could play a crucial role in the design of CBAs (Mislevy et al, 2010) and have been frequently incorporated into CBAs for item presentation. These multiple external representations (MERs) have been widely used for teaching and learning (see reviews in Ainsworth, 1999;Wu & Puntambekar, 2012).…”

Science in Writing: Learning Scientific Argument in Principle and Practice

“…These are typically represented in written argumentation. Such knowledge representations are essential tools for understanding, knowledge making and knowledge communication (Greeno & Hall, 1997;Bransford et al, 2000;Pellegrino et al, 2001;Chung et al, 2006;Mislevy, 2006a;Mislevy et al, 2007). These are characteristically social processes, requiring less memory work than a capacity to work with collaborating experts, searching sources, documenting and citing material used, and distinguishing one's own reasoning from the reasoning of others (Meltzoff et al, 2009).…”