Comparative Coh-Metrix Analysis of Reading Comprehension Texts: Unified (Russian) State Exam in English vs Cambridge First Certificate in English

Abstract

The article summarizes the results of the comparative study of Reading comprehension texts used in B2 level tests: Unified (Russia) State Exam in English (EGE) and Cambridge First Certificate in English (FCE). The research conducted was mainly focused on six parameters measured with the Coh-Metrix, a computational tool producing indices of the linguistic and discourse representations of a text: narrativity, syntactic simplicity, word concreteness, referential cohesion, deep cohesion, Flesh Reading Ease. The research shows that the complexity of EGE texts caused by lower than in FCE texts cohesion is balanced with a simpler than in FCE texts syntax and higher narrativity thus resulting in about the same text complexity of the two sets of texts studied. EGE and FCE texts demonstrate correspondence to grade six and very similar Means of Flesh Reading Ease (FCE Mean is 71.06; EGE Mean is 78.25) which fit the band FAIRLY EASY.

WITHDRAWN/ RETRACTED: English-medium instruction in Siberia:

“…However, in Russia it is not possible to make admissions decisions based on English testing except by requiring a foreign language at the Unified State Exam; this will be required from most SAS entrants as of the fall of 2021 (in Biology and IT, prospective students rarely prioritise English in high school). Introducing the new EGE requirement will not solve many of the problems discussed in this note, however, as for example its reading comprehension tasks are around the FCE (First Certificate in English) level (Solnyshkina et al, 2014), so below what would normally be required to start an EMI undergraduate degree.…”

Natural language processing and discourse complexity studies

“…TERA рассчитывает и визуализирует интегральные индексы пяти кластеров параметров: повествовательность, синтаксическая простота, конкретность слова, референциальная связность и глубокая связность, востребованных методистами и тестологами при отборе текстов для соответствующей аудитории читателей (см. Solnyshkina, Harkova & Kiselnikov 2014). Сoh-Metrix же анализирует тексты по более чем 200 параметрам и предназначена преимущественно для исследовательских целей (McNamara & Graesser 2012).…”

“…Достижения современной парадигмы дискурсивной комплексологии наилучшим образом реализованы в автоматическом анализаторе Coh-Metrix, предназначенном для оценки языковой сложности и тестирования когнитивных моделей восприятия. Инструмент прошел проверку временем и валидирован в ряде исследований (Hall et al 2006, Graesser et al 2014, Solnyshkina et al 2014, Солнышкина, Кисельников 2015. Coh-Mеtrix создавалась на корпусе TASA (Touchstone Applied Science Associates), репрезентативность которого обеспечена не только его объемом (37 000 текстов), но и тем, что он «содержит примерно такое же количество и качество текстов, которые среднестатистический студент колледжа прочел за всю свою жизнь» (Jones et al).…”

Readability of Reading Passages in English Textbooks and the Thai National Education English Test: A Comparative Study

“…Readability is generally measured by using readability formulas. Flesch Reading Ease and Flesch-Kincaid Grade Level are the two most common and practical methods used in estimating difficulty level (Solnyshkina et al: 2014). These two formulas assess the difficulty level based on the word and sentence length in the target reading text (Flesch, 1948).…”

“…However, reading passages are not only found in teaching materials. In a testing context, reading passages are mainly used in reading comprehension tests, and their linguistic characteristics are regarded as a neglected area (Solnyshkina, Harkova, & Kiselnikov, 2014). Relatively few studies have been published in the field of language testing, especially those focusing on a comparison of the readability levels of teaching and testing materials.…”