Since about 90% of schools, colleges, and departments of education are currently using portfolios of one form or another as decision-making tools for standards-based decisions regarding certification or licensure (as well as NCATE accreditation), it is appropriate to explore the legal and psychometric aspects of this assessment device. The authors demonstrate that portfolios being used in a high-stakes context are technically testing devices and therefore need to meet psychometric standards of validity, reliability, fairness, and absence of bias. These standards, along with federal law, form the cornerstone for legal challenges to high-stakes decisions when students are denied a diploma or license based on the results of the assessment. The conclusion includes a list of requirements and caveats for using portfolios for graduation and certification decisions in a standards-based environment that help institutions reduce exposure to potential litigation.
Purpose Understanding and navigating the differences in standards, and the roots and rationales underlying accreditation reviews, is necessary for all institutions that seek multiple accreditations. The purpose of this paper is to demonstrate a method to assist institutional-level leaders and assessment practitioners analyze and align these differences in various national or international agency requirements, to develop a framework for assessment and data collection. The proposed method is demonstrated by using multiple accreditors’ standards from the USA. Design/methodology/approach Guided by a set of process questions, a review and content analysis of national standards and 12 accreditation agency requirements from the USA was conducted using Web-based, documentary sources. An operational definition of institutional quality was derived based on the core themes that emerged. Examples of evidence matched to each core theme were outlined to suggest an assessment framework. The 12 US agency requirements were compared and contrasted with the core themes and validated. Findings In the USA, recognition requirements set by two national bodies, the US Department of Education and Council of Higher Education Accreditation, drive the standards applied by various agencies that accredit institutions and programs. Six themes emerged from their requirements, serving as a core framework for designing institutional assessment systems. The themes are student achievement and continuous improvement; curriculum quality; faculty; facilities, equipment and supplies; fiscal and administrative capacity; and student support services, admissions and information-gathering systems. While the 12 sampled accreditation agencies generally used these core themes, divergences were found in how they treated the themes in published requirements. Practical implications Where multiple US or other accreditations are sought, the approach recommended could facilitate the work of institutional accreditation leaders and practitioners in establishing assessment systems that reduce redundancy while also maximizing efficiency in assessment and data collection. Originality/value There is little guidance in the literature on how institutional leaders and practitioners confronting the challenges of accreditation can negotiate multiple, and sometimes conflicting, sets of requirements. This paper demonstrates a possible solution strategy. Outside the general utility of the demonstrated method, the findings and core assessment framework produced could be useful for institutions seeking accreditation through the agencies in the study sample, in both the USA and overseas.
Validity and reliability are a major focus in teacher education accreditation by the Council for Accreditation of Educator Preparation (CAEP). CAEP requires the use of “accepted research standards,” but many faculty and administrators are unsure how to meet this requirement. The Standards of Educational and Psychological Testing abbreviated Joint Standards, are the implied accepted research standards. The purpose of this study was to test empirically the extent to which the Joint Standards could serve this purpose. The primary outcome of this study is a pragmatic set of implementation strategies organized into assessment specifications connecting CAEP requirements to the Joint Standards. An example set of assessment specifications is provided. Use of this process can improve assessments in any academic discipline for any accreditation agency, become a faculty professional development tool, raise assessment literacy, and potentially impact instruction of future educators.
This study explored whether differences in teacher candidate dispositions exist between Chinese and American students, while continuing validation of the updated Beliefs About Teaching Scale (BATS2). BATS2 incorporated the Rasch model of item response theory on Thurstone dichotomous items to measure commitment to the Interstate Teacher Assessment and Support Consortium Standards (InTASC) along the levels of the Bloom/Krathwohl affective taxonomy. This research is a unique combination of theories and practice – standards-based items, affective taxonomy, and modern measurement theory. Differential group function (DGF), applied in a mixed methods design, confirmed national differences, indicating differential commitment to standards and items. For standards and items that showed a difference in the two groups, literature and cultural context supporting those differences was identified to frame the qualitative portion of this study. For example, US teachers were more averse to assessment, clearly the result of the focus on standardized testing that is so resented in the US; the Chinese were less compelled to master content, which is less imperative in early childhood programs (the sample in this study). Results can be used in considering training needs and making instructional design more likely to be impactful for US institutions training Chinese natives and for Chinese institutions updating programs based on international input.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.