Despite concerted efforts by science educators to understand patterns of evolutionary reasoning in science students and teachers, the vast majority of evolution education studies have failed to carefully consider or control for item feature effects in knowledge measurement. Our study explores whether robust contextualization patterns emerge within particular evolutionary reasoning contexts, and the implications of these patterns for instruction, assessment, and models of cognition. We test four hypotheses regarding item feature effects on undergraduate biology majors' evolutionary reasoning using a sample of 1,200 open response explanations of evolutionary change across items differing in context and scale but standardized by taxon and trait. Evolutionary explanations were atomized into a series of scientific and naïve biological elements and tallied among prompts and their features. We documented clear, significant, and predictable item feature effects on evolutionary explanations. Tasks involving evolutionary trait loss elicited a significantly greater number of naïve biological elements than evolutionary trait gain tasks in all contexts, including: within species comparisons, between species comparisons, animal prompts, and plant prompts. Tasks involving between species evolutionary comparisons, regardless of gain or loss, animal or plant, always produced significantly more naïve biological explanatory elements than within species comparisons. For items prompting explanation of trait gain, the use of the core concepts of natural selection were not influenced by the hierarchical level of the task (within or between species). Explanations of trait gain were also the least sensitive to scale and context. Core concepts of natural selection were always deployed less frequently in cases of evolutionary trait loss (within and between species, in animals and plants). We discuss a series of implications of these findings for curriculum, instruction, and assessment. © 2010 Wiley Periodicals, Inc., Inc. J Res Sci Teach 48: 237–256, 2011

To improve assessments of academic achievement, test developers have been urged to use an ''assessment triangle'' that starts with research-based models of cognition and learning [NRC (2001) Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press]. This approach has been successful in designing high-quality reading and math assessments, but less progress has been made for assessments in content-rich sciences such as biology. To rectify this situation, we applied the ''assessment triangle'' to design and evaluate new items for an instrument (ACORNS, Assessing Contextual Reasoning about Natural Selection) that had been proposed to assess students' use of natural selection to explain evolutionary change. Design and scoring of items was explicitly guided by a cognitive model that reflected four psychological principles: with development of expertise, (1) core concepts facilitate long-term recall, (2) causally-central features become weighted more strongly in explaining phenomena, (3) normative ideas co-exist but increasingly outcompete naive ideas in reasoning, and (4) knowledge becomes more abstract and less specific to the learning situation. We conducted an evaluation study with 320 students to examine whether scores from our new ACORNS items could detect gradations of expertise, provide insight into thinking about evolutionary change, and predict teachers' assessments of student achievement. Findings were consistent with our cognitive model, and ACORNS was revealing about undergraduates' thinking about evolutionary change. Results indicated that (1) causally-central concepts of evolution by natural selection typically co-existed and competed with the presence of naïve ideas in all students' explanations, with naïve ideas being especially prevalent in low-performers' explanations; (2) causally-central concepts were elicited most frequently when students were asked to explain evolution of animals and familiar plants, with influence of superficial features being strongest for low-performers; and (3) ACORNS scores accurately predicted students' later achievement in a college-level evolution course. Together, findings illustrate usefulness of cognitive models in designing instruments intended to capture students' developing expertise. ß

Studies of students' thinking about natural selection have revealed that the scenarios in which students reason evoke different types, magnitudes, and arrangements of knowledge elements and misconceptions. Diagnostic tests are needed that probe students' thinking across a representative array of evolutionary contexts. The ACORNS is a diagnostic test that treats different evolutionary contexts as unique scenarios worthy of focused assessment and targeted instruction. Our investigations revealed that ACORNS scores produce valid and reliable inferences about students' thinking about natural selection. We urge biology teachers at all educational levels to begin assessing and attending to their students' reasoning across a broader array of evolutionary contexts, as competency in one context is often not indicative of competency in another.

No abstract

We propose a new model of the factors influencing acceptance of evolutionary theory that highlights a novel variable unexplored in previous studies: the feeling of certainty (FOC). The model is grounded in an emerging understanding of brain function that acknowledges the contributions of intuitive cognitions in making decisions, such as whether or not to accept a particular theoretical explanation of events. Specifically, we examine the relationships among religious identity, level of education, level of knowledge, FOC, and level of evolutionary acceptance to test whether our proposed model accurately predicts hypothesized pathways. We employ widely used measures-the CINS, MATE, and ORI-in addition to new variables in multiple regression and path analyses in order to test the interrelationships among FOC and acceptance of evolutionary theory. We explore these relationships using a sample of 124 pre-service biology teachers found to display comparable knowledge and belief levels as reported in previous studies on this topic. All of our hypothesis tests corroborated the idea that FOC plays a moderating role in relationships among evolutionary knowledge and beliefs. Educational research into acceptance of evolutionary theory will likely benefit from increased attention to non-conscious intuitive cognitions that give rise to feeling of knowing or certainty. ß 2011 Wiley Periodicals, Inc. J Res Sci Teach 49: 2012