Applying measurement standards to evolution education assessment instruments

Abstract

Over the past 25 years a number of instruments have been published that attempt to measure understanding and acceptance of evolution. Science educators have been administering these instruments and reporting results, however, it is not clear these instruments are being used appropriately. The goal of this paper is to review these instruments, noting the original criteria and population for which evidence of validity and reliability was assessed, and to survey other publications that report their use, examining each for evidence of validity and reliability with subsequent populations. Our hope is that such a comprehensive review will engage researchers and practitioners in a careful examination of how they intend to use a particular instrument and whether it can provide an accurate and meaningful assessment of the desired outcomes. We encourage the community to administer evolution education assessments with the consideration of an instrument's measurement support and past use with similar populations. We also encourage researchers to add additional evidence of validity and reliability for these instruments, especially if modifications have been made to the instrument or if its use has been extended to new populations.

Towards common ground in measuring acceptance of evolution and knowledge about evolution across Europe: a systematic review of the state of research

“…To date there is no clear consensus in the evolution education community about the nature and the extent of this relationship (e.g., Barnes et al 2019;Dunk et al 2019;Glaze and Goldston 2015). The application of different measuring instruments (e.g., Barnes et al 2019;Mead et al 2019; as well as the different use of terms concerning the key constructs (Konnemann et al 2012; Smith and Siegel 2016) may be the main reasons for inconsistent results in this research area. This is a crucial issue for science education, since studies on attitudes and knowledge about evolution as well as their relationship lead to conclusions regarding the teaching of evolution (e.g., for Turkey Annaç and Bahçekapili 2012).…”

“…Besides this aspect of content validity, comparative investigations require appropriate evidence for validation in the local context of the single studies (AERA 2014). Since Nehm and Schonfeld (2008) raised the issue of measuring knowledge about natural selection and the subsequent debate (Anderson et al 2010;Nehm and Schonfeld 2010), the discourse concerning measurement issues in evolution education accelerated and has been addressed continuously within the last years (e.g., Anderson et al 2010;Barnes et al 2019;Beniermann 2019;McCain and Kampourakis 2018;Mead et al 2019;Novick and Catley 2012;. In the introduction to a special issue devoted to the topic of evolution assessment, Nehm and Mead (2019) have recently underlined the importance of drawing greater attention to research on the measurement and assessment of knowledge, attitudes and skills that are central to evolution education, thus calling for further research efforts in this area.…”

“…However, the vast majority of evolution education studies have failed to carefully consider or control for context effects of items in assessment tasks (Son andGoldstone 2009, but see Nehm et al 2012). Fifth, Mead et al (2019) pointed out the importance of measurement standards for instruments measuring evolutionary knowledge and acceptance. They reviewed 13 different evolution education assessment instruments with respect to the evidence supporting their validity and reliability.…”

“…They reviewed 13 different evolution education assessment instruments with respect to the evidence supporting their validity and reliability. Mead et al (2019) revealed validity and reliability issues for some often-used instruments. Additionally, most instruments were validated for only one specific population.…”

Acceptance and knowledge of evolutionary theory among third-year university students in Spain

“…We estimated the reliability of MATE and KEE data by using Cronbach's alpha statistics, as has been suggested to be done routinely for this kind of study [28,33,34]. In addition, we…”