A stratified study of students’ understanding of basic optics concepts in different contexts using two‐tier multiple‐choice items

Chu, Hye‐Eun; Treagust, David F.; Chandrasegaran, A. L.

doi:10.1080/02635140903162553

Cited by 71 publications

(47 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Developing TTMC items draws on literature on students' alternative conceptions that have been conducted over the past 20 years or so. For example, Chu and Treagust (2009) developed the Light Propagation Diagnostic Instrument (LPDI) based on prior qualitative research studies on students' conceptions mainly about fundamental geometric optics (Andersson and Kärrqvist, 1983;Fetherstonaugh and Treagust 1992;Galili and Hazan 2000;La Rosa et al 1984;Langley et al 1997). Thus, using TTMC items affords teachers access to results of this literature for uncovering students' pre-instructional understanding and reasoning about the phenomena or concepts.…”

Section: Development and Analysis Of Two-tier Itemsmentioning

confidence: 99%

“…The two tiers in two-tier items act together to uncover students' understanding of core concepts because the student must choose a seemingly "factual" knowledge response for the first tier (Taber and Tan 2011), and then choose for the second tier what reasoning about the concept they used to arrive at the first-tier response. A large body of research across contexts has applied two-tier items to uncover students' understanding of scientific concepts as broad ranging as optics, scientific reasoning, and scientific knowledge integration and in various settings such as the US, UK, Korea, Singapore, and Australia (Chu and Treagust 2009;Johnson and Tymms 2011;Liu et al 2011;Taber and Tan 2011;Tsui and Treagust 2009).Despite the breadth of this prior research, there is still relatively little attention over how best to analyze two-tier responses and uncover how students respond to the two tiers. In particular, through rigorous measurement approaches, it is possible to examine two of the fundamental notions about two-tier items and students' responses: (1) whether the traits assessed by the first and second tiers are distinguishable yet related, and (2) whether the second tier is indeed more difficult than the first tier.…”

mentioning

confidence: 99%

“…That national project found many commonalities in results with two-tier item studies in Western contexts, but also explored how students' misconceptions could relate to specific structure and meanings of Chinese words as used in the textbooks that were unique to the setting. In another example from Singapore and with parallel study in Korea, Chu et al (2009) developed LPDI items Fig. 1 Sample two-tier multiple choice questions specifically to assess students' stable or context-independent conceptions in two different situations: light propagation during the day and at night.…”

Section: Types and Uses Of Two-tier Itemsmentioning

confidence: 99%

“…The first data set comes from a study of secondary students' understanding of fundamental concepts in optics-light propagation and visibility (Chu and Treagust 2009). The original work focused on whether and how students' understanding of the content was dependent upon the particular context of the items.…”

Section: Data Set 1 -Opticsmentioning

confidence: 99%

mentioning

confidence: 99%

See 4 more Smart Citations

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Fulmer

Chu

Treagust

et al. 2015

Asia Pac. Sci. Educ.

Self Cite

View full text Add to dashboard Cite

Two-tier multiple-choice (TTMC) items are used to assess students' knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2. But are the knowledge and reasoning involved in these tiers really distinguishable? Are the tiers equally challenging for students? The answers to these questions influence how we use and interpret TTMC instruments. We apply the Rasch measurement model on TTMC items to see if the items are distinguishable according to different traits (represented by the tier), or according to different content sub-topics within the instrument, or to both content and tier. Two TTMC data sets are analyzed: data from Singapore and Korea on the Light Propagation Diagnostic Instrument (LPDI), data from the United States on the Classroom Test of Scientific Reasoning (CTSR). Findings for LPDI show that tier-2 reasoning items are more difficult than tier-1 knowledge items, across content sub-topics. Findings for CTSR do not show a consistent pattern by tier or by content sub-topic. We conclude that TTMC items cannot be assumed to have a consistent pattern of difficulty by tier-and that assessment developers and users need to consider how the tiers operate when administering TTMC items and interpreting results. Researchers must check the tiers' difficulties empirically during validation and use. Though findings from data in Asian contexts were more consistent, further study is needed to rule out differences between the LPDI and CTSR instruments.Keywords: Science education, Two-tier items, Rasch measurement models, Optics, Scientific reasoning Assessing student learning-of scientific concepts, practices, or habits of mind-is one of the central topics for research and development in science education. Such assessments can serve as formative or diagnostic tools for planning instruction and working with students, or as summative tools for gauging the effectiveness of our instructional practices, curriculum materials, or teacher education efforts. However, we observe an ongoing tension in science education assessment between our ability to construct conventional test items (e.g., multiple choice questions) that can be highly reliable but are perceived to be incapable of providing richer insights into students' conceptions and ways of thinking. Research addressing this includes efforts to make better sense of how students' responses to test items can be understood from a broader view of conceptual understanding, ability, or skill Neumann et al. 2011).One solution proposed that can address this tension are two-tier items (Treagust 1988). The two tiers in two-tier items act together to uncover students' understanding of core concepts because the student must choose a seemingly "factual" knowledge response for the first tier (Taber and Tan 2011), and then choose for the second tier what reasoning about the concept they used to arrive at the first-tier response. A large body of research across contexts has applied two-tier items to uncover students' understanding of scientific ...

show abstract

Section: Development and Analysis Of Two-tier Itemsmentioning

confidence: 99%

mentioning

confidence: 99%

Section: Types and Uses Of Two-tier Itemsmentioning

confidence: 99%

Section: Data Set 1 -Opticsmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Fulmer

Chu

Treagust

et al. 2015

Asia Pac. Sci. Educ.

Self Cite

View full text Add to dashboard Cite

show abstract

Assessing Understanding of the Energy Concept in Different Science Disciplines

Park

Liu

2016

Science Education

View full text Add to dashboard Cite

Energy is one of the most central and richly connected ideas across all science disciplines. The purpose of this study was to develop a measurement instrument for assessing students’ understanding of the energy concept within and across different science disciplines. To achieve this goal, the Inter‐Disciplinary Energy concept Assessment (IDEA) was developed through a pilot test, consisting of 49 items presented in pairs: multiple‐choice questions followed by open‐ended justification questions. The IDEA was administered to 356 college students in the United States. The partial credit Rasch model was applied to establish evidence of validity and reliability of measures for the IDEA. Results showed that the IDEA can produce reliable and valid measures of student understanding of the energy concept in different science disciplines. No statistical difference was found between the average difficulties of items across disciplines. In addition, the four science disciplines were positively correlated in terms of student understanding of the energy concept. When item difficulties were analyzed by science content topics, it was found that students have difficulty in understanding the energy concept in atomic structure, wave, electric and magnetic energy, and modern physics content areas. This result suggests that students’ difficulty in understanding the energy concept is compounded with specific science contents.

show abstract

Item feature effects in evolution assessment

2010

View full text Add to dashboard Cite

Despite concerted efforts by science educators to understand patterns of evolutionary reasoning in science students and teachers, the vast majority of evolution education studies have failed to carefully consider or control for item feature effects in knowledge measurement. Our study explores whether robust contextualization patterns emerge within particular evolutionary reasoning contexts, and the implications of these patterns for instruction, assessment, and models of cognition. We test four hypotheses regarding item feature effects on undergraduate biology majors' evolutionary reasoning using a sample of 1,200 open response explanations of evolutionary change across items differing in context and scale but standardized by taxon and trait. Evolutionary explanations were atomized into a series of scientific and naïve biological elements and tallied among prompts and their features. We documented clear, significant, and predictable item feature effects on evolutionary explanations. Tasks involving evolutionary trait loss elicited a significantly greater number of naïve biological elements than evolutionary trait gain tasks in all contexts, including: within species comparisons, between species comparisons, animal prompts, and plant prompts. Tasks involving between species evolutionary comparisons, regardless of gain or loss, animal or plant, always produced significantly more naïve biological explanatory elements than within species comparisons. For items prompting explanation of trait gain, the use of the core concepts of natural selection were not influenced by the hierarchical level of the task (within or between species). Explanations of trait gain were also the least sensitive to scale and context. Core concepts of natural selection were always deployed less frequently in cases of evolutionary trait loss (within and between species, in animals and plants). We discuss a series of implications of these findings for curriculum, instruction, and assessment. © 2010 Wiley Periodicals, Inc., Inc. J Res Sci Teach 48: 237–256, 2011

show abstract

A stratified study of students’ understanding of basic optics concepts in different contexts using two‐tier multiple‐choice items

Cited by 71 publications

References 13 publications

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model

Assessing Understanding of the Energy Concept in Different Science Disciplines

Item feature effects in evolution assessment

Contact Info

Product

Resources

About