Constituting a metacognitive strategy, system competence or systems thinking can only assume its assigned key function as a basic concept for the school subject of geography in Germany after a theoretical and empirical foundation has been established. A measurement instrument is required which is suitable both for supporting students and for the evaluation of methodical‐didactic measures. Such a tool is theoretically anchored in an empirically validated geography‐didactic and cognition‐psychological competence model, providing a differentiated representation of both the internal structure of a competency and the proficiency levels. The starting point of this foundation was the development of a normative‐theoretically derived model of geographic system competence. Its empirical validation was performed in different phases aimed at operationalising the competence model by means of test problems. In order to analyse the factor structure of the theoretical model, various item response models were estimated. The item levels of difficulty expected in the competence model were related to the empirical levels of difficulty and predicted by means of ordinary least squares regression to verify the model for proficiency levels. The two‐dimensional competence model – with the two dimensions ‘system organisation and behaviour’ and ‘system‐adequate intention to act’ – exhibits a better fit in reference to the model fit criteria than the one‐dimensional and three‐dimensional models. The correlations between the expected and empirical item difficulties are positive. Items that should be more difficult according to the competence model are actually shown to be more difficult. These findings suggest the reliability and validity of this new measurement instrument for diagnosing and promoting geographical system competence. It has to be implemented in practice as the next step.

In this article, the change in examinee effort during an assessment, which we will refer to as persistence, is modeled as an effect of item position. A multilevel extension is proposed to analyze hierarchically structured data and decompose the individual differences in persistence. Data from the 2009 Program of International Student Achievement (PISA) reading assessment from N ¼ 467,819 students from 65 countries are analyzed with the proposed model, and the results are compared across countries. A decrease in examinee effort during the PISA reading assessment was found consistently across countries, with individual differences within and between schools. Both the decrease and the individual differences are more pronounced in lower performing countries. Within schools, persistence is slightly negatively correlated with reading ability; but at the school level, this correlation is positive in most countries. The results of our analyses indicate that it is important to model and control examinee effort in low-stakes assessments.

Dieses Dokument steht unter folgender Creative Commons-Lizenz: http://creativecommons.org/licenses/by-nc-nd/4.0/deed.de -Sie dürfen das Werk bzw. den Inhalt unter folgenden Bedingungen vervielfältigen, verbreiten und öffentlich zugänglich machen: Sie müssen den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. Dieses Werk bzw. dieser Inhalt darf nicht für kommerzielle Zwecke verwendet werden und es darf nicht bearbeitet, abgewandelt oder in anderer Weise verändert werden.

No abstract

He, Jia; Buchholz, Janine; Klieme, Eckhard Effects of anchoring vignettes on comparability and predictive validity of student self-reports in 64 cultures formal und inhaltlich überarbeitete Version der Originalveröffentlichung in: formally and content revised edition of the original source in:Journal of cross-cultural psychology 48 (2017) 3, S. 319-334, 10.1177 We grant a non-exclusive, non-transferable, individual and limited right to using this document. This document is solely intended for your personal, non-commercial use. Use of this document does not include any transfer of property rights and it is conditional to the following limitations: All of the copies of this documents must retain all copyright information and other information regarding legal protection. You are not allowed to alter this document in any way, to copy it for public or commercial purposes, to exhibit the document in public, to perform, distribute or otherwise use the document in public. AbstractAnchoring vignettes are item batteries especially designed for correcting responses that might be affected by incomparability. This article investigates the effects of anchoring vignettes on the validity of student self-report data in 64 cultures. Using secondary data analysis from the 2012 Programme for International Student Assessment (PISA), we checked the validity of ratings on vignette questions, and investigated how rescaled item responses of two student scales, Teacher Support and Classroom Management, enhanced comparability and predictive validity. The main findings include that (a) responses to vignette questions represent valid individual and cultural differences; in particular, violations in these responses (i.e., misorderings) are related to low socioeconomic status and low cognitive sophistication; (b) the rescaled responses tend to show higher levels of comparability; and (c) the associations of rescaled Teacher Support and Classroom Management with math achievement, Student-Oriented Instruction, and Teacher-Directed Instruction are slightly different from raw scores of the two target constructs, and the associations with rescaled scores seem to be more in line with the literature. Namely, the associations among all self-report Likerttype scales are weaker with rescaled scores, presumably reducing common method variance, and both rescaled scale scores are more positively related to math achievement. The country ranking also changes substantially; in particular, Asian cultures top the ranking on Teacher Support after rescaling. However, anchoring vignettes are not a cure-all in solving measurement bias in crosscultural surveys; we discuss the technicality and directions for further research on this technique.