Abstract:This paper utilizes a case-study design to discuss global aspects of massive open online course (MOOC) assessment. Drawing from the literature on open-course models and linguistic gatekeeping in education, we position freeform assessment in MOOCs as both challenging and valuable, with an emphasis on current practices and student resources. We report on the findings from a linguistically-diverse pharmacy MOOC, taught by a native English speaker, which utilized an automated essay scoring (AES) assignment to enga… Show more
“…Future studies should continue to utilize different samples to test the validity of the automated VAIL to verify that the current results consistently replicate. In light of recent findings that suggest automated scoring techniques differentially benefit native English speakers (Reilly et al, 2016), it would be advantageous to test the automated VAIL, and similar measures, in linguistically diverse samples to ensure validity. As previously mentioned the present study considered in-service teachers who were enrolled in coursework and/or coaching.…”
Although scalable programs, such as online courses, have the potential to reach broad audiences, they may pose challenges to evaluating learners' knowledge and skills. Automated scoring offers a possible solution. In the current paper, we describe the process of creating and testing an automated means of scoring a validated measure of teachers' observational skills, known as the Video Assessment of Instructional Learning (VAIL). Findings show that automated VAIL scores were consistently correlated with scores assigned by the hand scoring system. In addition, the automated VAIL replicated intervention effects found in the hand scoring system. The automated scoring technique appears to offer an efficient and reliable assessment. This study may offer additional insight into how to utilize similar techniques in other large-scale programs and interventions.
“…Future studies should continue to utilize different samples to test the validity of the automated VAIL to verify that the current results consistently replicate. In light of recent findings that suggest automated scoring techniques differentially benefit native English speakers (Reilly et al, 2016), it would be advantageous to test the automated VAIL, and similar measures, in linguistically diverse samples to ensure validity. As previously mentioned the present study considered in-service teachers who were enrolled in coursework and/or coaching.…”
Although scalable programs, such as online courses, have the potential to reach broad audiences, they may pose challenges to evaluating learners' knowledge and skills. Automated scoring offers a possible solution. In the current paper, we describe the process of creating and testing an automated means of scoring a validated measure of teachers' observational skills, known as the Video Assessment of Instructional Learning (VAIL). Findings show that automated VAIL scores were consistently correlated with scores assigned by the hand scoring system. In addition, the automated VAIL replicated intervention effects found in the hand scoring system. The automated scoring technique appears to offer an efficient and reliable assessment. This study may offer additional insight into how to utilize similar techniques in other large-scale programs and interventions.
“…Nonnative English-speaking students read more slowly than native speakers and are likely to play a video slowly to understand instructors' lessons (Reilly et al, 2016) and may require more time to learn the content, sometimes falling behind (Sanchez-Gordon & Luján-Mora, 2014). Nonnative English-speaking students tend to achieve lower scores than Englishspeaking students in MOOCs (Engle et al, 2015;Reilly et al, 2016) Another concern is nonnative English-speaking learners' participation in social interaction, such as online discussion (Colas et al, 2016). In an analysis of online forum participation in a business course offered in Coursera, Gillani and Eynon (2014) found that European and North American participants not only achieved higher grades in forum participation than those from Asia but were also more visible in online forums, participating in discussions more actively than other participants.…”
Section: Nonnative English-speaking Participants' Learning In Moocsmentioning
confidence: 99%
“…Several researchers have argued that MOOC developers and educators should consider their needs. Because most MOOCs are offered in English (Shah, 2015a), the language barrier is a concern (Colas, Sloep, & Garreta-Domingo, 2016;Reilly et al, 2016;Sanchez-Gordon & Luján-Mora, 2014). Nonnative English-speaking students read more slowly than native speakers and are likely to play a video slowly to understand instructors' lessons (Reilly et al, 2016) and may require more time to learn the content, sometimes falling behind (Sanchez-Gordon & Luján-Mora, 2014).…”
Section: Nonnative English-speaking Participants' Learning In Moocsmentioning
confidence: 99%
“…Because most MOOCs are offered in English (Shah, 2015a), the language barrier is a concern (Colas, Sloep, & Garreta-Domingo, 2016;Reilly et al, 2016;Sanchez-Gordon & Luján-Mora, 2014). Nonnative English-speaking students read more slowly than native speakers and are likely to play a video slowly to understand instructors' lessons (Reilly et al, 2016) and may require more time to learn the content, sometimes falling behind (Sanchez-Gordon & Luján-Mora, 2014). Nonnative English-speaking students tend to achieve lower scores than Englishspeaking students in MOOCs (Engle et al, 2015;Reilly et al, 2016) Another concern is nonnative English-speaking learners' participation in social interaction, such as online discussion (Colas et al, 2016).…”
Section: Nonnative English-speaking Participants' Learning In Moocsmentioning
confidence: 99%
“…Among the 33,378 learners signed up for the MOOC, 15,310 students responded to the survey, 56.9% of whom reported that English was not their first language. In addition, in Take Your Medicine: The Impact of Drug Development, a course offered through edX, 1,090 students completed open-ended writing assignments, 64.74% of whom reported themselves learners of as English as a second language (Reilly et al, 2016). In terms of the growing power of the internet, the number of nonnative English-speaking learners in MOOCs is expected to grow rapidly as the total number of these courses dramatically increases (Shah, 2015a).…”
The goal of this study was to gain in-depth understanding about nonnative English-speaking students' lived experiences with massive open online courses (MOOCs) in a regular college classroom. Phenomenological methodology was used to examine those experiences in 24 Korean college students. Individual interviews, an open-ended online survey, observation notes, online weekly journal entries, and social media constituted the data sources. Findings show that students' lived experiences included (a) wonder and interest, (b) novel learning and teaching practices, (c) preference for video style, (d) learning strategies, (e) motivation to learn, and (f) need for face-to-face interaction. Implications for integrating a MOOC into a regular college course are also presented.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.