Setting Evidence‐Based Language Goals

Abstract

The purpose of this project was to identify target language benchmarks for the German program at Michigan State University (MSU) based on national and international guidelines and previous research, to assess language skills across course levels and class sections in the entire German program, and to adjust the language benchmarks as needed based on the study results and make recommendations for the curriculum. The goal of the study was to identify cost‐ and time‐effective valid and reliable assessment tools for program‐wide language assessment. During the 2012 spring semester, all students enrolled in a German class at MSU (N = 320) participated in this study. At the beginning of the semester, students completed two different language tests both advertised as being aligned with the Common European Framework of Reference (CEFR). The research was guided by the following two research questions: (1) What are the mean and median language levels of students enrolled in German classes at MSU as determined by (a) a language learning software test, and (b) a language test adapted from Goethe‐Institut practice exams? and (2) Is there a correlation between the two tests and/or subsections of the tests? Benchmark levels were initially set as Novice High after the first year of language study, Intermediate Low after the second year, Intermediate High after the third year, and Advanced Low after the fourth year. Students mostly met the benchmark levels, though not equally at each program level and not equally across skills, with the higher scores in productive skills rather than receptive skills. While there was a correlation between the tests and course levels and also between many of the test portions, the tests did not yield the same CEFR ratings, which suggests that at least one of the instruments was not as valid and reliable as hoped. After reviewing the results, the benchmarks were revised to Intermediate Low after the first year, Intermediate Mid after the second year, Intermediate High after the third year, and Advanced Low after the fourth year with the recommendation to put more emphasis on receptive skills in the curriculum.

The Seal of Biliteracy: Successes and challenges to implementation

“…Overall, the findings confirm that world language departments need to implement valid and reliable assessment of students’ learning at various points during their program of study and use those data to set benchmarks (Goertler, Kraemer, & Schenker, ). Furthermore, districts could consider supporting data teams (Schildkamp, Poortman, & Handelzalts, ) charged with identifying trends in reading, writing, listening, and speaking data and then rewriting world language curricula to remedy patterns of deficiency, thus promoting positive washback.…”

Language Testing: Current Practices and Future Developments

“…Goertler, Kraemer, and Schenker () and Tschirner () summarized the studies undertaken to gauge oral proficiency levels of U.S. college students from their freshman year to graduation (see also Heidrich & Kraemer, , in this issue). These studies showed that the average speaking proficiency level of college students after one year of instruction was NH or IL; after two years, it was IL or IM; after three years, it was IM or IH; and after four years, it was IH without and IH or AL with a study abroad period (see Table ).…”

Factors that Influence Proficiency Attainment in a College Language Program: A German Case Study

“…Since then, language proficiency focused research has looked at student achievement in high school settings (e.g., Shrum & Glisan, ), while research in higher education has often revolved around proficiency in one modality (e.g., Glisan, Swender, & Surface, ; Swender, ) or in one or two languages (e.g., Magnan, ; Sanders, ). Even though some of this research lacks the breadth of modalities and languages of larger‐scale studies, it still provides insight into proficiency achievement of students since Carroll's study; it has informed many curricular decisions and proficiency target levels throughout the years (see Goertler, Kraemer, & Schenker, , for a summary table of research on level achievement). Additional research related to proficiency focused on how other factors, such as the relationship between anxiety and oral proficiency ratings (see Young, ), impacted student learning.…”