OECD's Approach to Measuring Global Competency: Powerful Voices Shaping Education

Abstract

Background/Context Adding global competency to the Organisation of Economic Cooperation and Development (OECD) Program of International Student Assessment (PISA) suite heralds the world's first large-scale attempt at gauging education systems’ development of students’ global competency. Given the contested definitions and more than 150 extant instruments whose creators purport to measure global competency or related constructs, it is important to interrogate how influential and privileged global voices such as OECD portray and promote the construct. This new aspect of the PISA battery has the potential to reach 15 year olds in 80 countries that account for more than 80% of the world economy. Purpose/Focus of Study This paper is the first of a series of policy studies aimed at mapping OECD's global competency measure that will occur at significant periods of time within the implementation process. This initial study examines OECD's Global Competency for an Inclusive World (GCIW) promotional document to reveal its construction of global competency within discourse and assessment design. Research Design The study employs an uncommon mix of interpretive and relational methods. Critical discourse analysis (CDA) captures “how” global competency is portrayed and interrogates implications of OECD's language use and power differentials. Social network analysis (SNA) captures “who” is influencing the policy text and examines the connectivity among the authors cited as authorities on the subject of global competency. Greene, Caracelli, and Graham's (1989) convergence, complementarity, and contradiction framework is used to triangulate qualitative and quantitative datasets. Discussion and recommendations are framed and filtered through a policy implementation lens with five core policy threads: (a) people, (b) place, (c) philosophies, (d) processes, and (e) power, which authors refer to as the 5Ps. Findings CDA and SNA findings converge around the people, power, and places most central to OECD's approach to measuring global competency. The CDA shows that OECD's construction of the globally competent student reflects a rather narrow philosophical view of the term. The SNA demonstrates the power of particular academic networks and people that have informed this particular construction of global competency. CDA and SNA tell complementary stories about OECD's philosophies and processes. There were minimal contradictions between analytical methods, but the document under review seemed at odds with its own claims at times. For example, the PISA global competency measure seems to privilege particular social and economic ideologies, exercising power through its language in ways that oppose the very global competency definition that OECD seems to espouse. Conclusions By investigating the policy threads (5Ps) embedded in GCIW's production, the authors of the current study find OECD to have entertained a somewhat limited conversation in developing its definitional and measurement frameworks for assessing global competency. The ensuing critique highlights power differentials and inequalities within the GCIW document, revealing political, social, and technical issues. The current study concludes by challenging policymakers to seek a wider range of voices to inform policy directions as OECD and other influential organizations continue to refine their understanding of global competency, a 21st century imperative that is yet-to-be fully understood. The current study also offers recommendations such as continuing critiques of global policy texts and measures from inception through implementation, ensuring to capture both implications and impacts.

Research in Global Learning: Methodologies for global citizenship and sustainable development education

“…Overall, the findings have methodological implications that highlight the importance of directly engaging with pupils and teachers across and within national settings when assessing the validity or quality of standardised measures, as this can illuminate varying connotations or interpretations of items. The social and political ideologies endorsed, favoured and promoted by the test have been explored and critiqued in several recent studies (Auld and Morris 2019;Grotlüschen 2018;Ledger et al 2019); however, these studies concentrated on critical readings of the questions by scholars, rather than the direct understandings of pupils. Consulting with experts, scholars or policymakers from different national settings is insufficient, as they can be unfamiliar with the experiences of particular groups and hold uniform views of their population.…”