The MOOClet Framework: Unifying Experimentation, Dynamic Improvement, and Personalization in Online Courses

Reza, Mohi; Kim, Juho; Bhattacharjee, Ananya; Rafferty, Anna N.; Williams, Joseph Jay

doi:10.1145/3430895.3460128

Cited by 7 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Twenty‐three studies regarded automated feedback intervention delivered directly to learners. Among them, two studies propose computer agents to deliver feedback, while the conditions that trigger the feedback are decided by the course instructors (Reza et al, 2021; Yılmaz et al, 2021).…”

Section: Resultsmentioning

confidence: 99%

“…The textual support regarded text messages, hints and prompts, tips and personalized links or recommender systems and systems for automated corrections on provided solutions (Almeda et al, 2018; Caballe et al, 2014; Ferschke et al, 2015; Frick et al, 2022; Lafifi et al, 2020; Lan et al, 2015; Meku‐Fotso et al, 2020; Reza et al, 2021; Singelmann et al, 2019; Teusner et al, 2018; Wang et al, 2017; Yılmaz et al, 2021; Yu et al, 2021). Almeda et al (2018) proposed sending reminders with course‐related material and praising the top‐level discussion forum commenters.…”

Section: Resultsmentioning

confidence: 99%

“…Still, the role of instructors remains critical; instructors' active presence and support are valued positively by the learners and relate to retention (Gregori et al, 2018). In two studies, the automated and personalized feedback aimed at enhancing self-regulation was designed by the instructors (Reza et al, 2021;Yılmaz et al, 2021).…”

Section: Discussionmentioning

confidence: 99%

Section: Type Of Contributionmentioning

confidence: 99%

“…Twenty-five publications aimed at providing support to instructors by generating information about learners' progress.Twenty-three studies regarded automated feedback intervention delivered directly to learners. Among them, two studies propose computer agents to deliver feedback, while the conditions that trigger the feedback are decided by the course instructors(Reza et al, 2021;Yılmaz et al, 2021).Attending I.Q.2.3, 22 publications focused on shaping feedback interventions for all participants without reporting a specific cohort as a target of the designed interventions. Twelve publications focused on learners disengaged and at risk of dropping out Vinker and Rubinstein (2022).…”

mentioning

confidence: 99%

See 4 more Smart Citations

Delving into instructor‐led feedback interventions informed by learning analytics in massive open online courses

Topali

Chounta

Martínez‐Monés

et al. 2023

Computer Assisted Learning

View full text Add to dashboard Cite

Background: Providing feedback in massive open online courses (MOOCs) is challenging due to the massiveness and heterogeneity of learners' population. Learning analytics (LA) solutions aim at scaling up feedback interventions and supporting instructors in this endeavour. Paper Objectives: This paper focuses on instructor-led feedback mediated by LA tools in MOOCs. Our goal is to answer how, to what extent data-driven feedback is provided to learners, and what its impact is.Methods: We conducted a systematic literature review on the state-of-the-art LAinformed instructor-led feedback in MOOCs. From a pool of 227 publications, we selected 38 articles that address the topic of LA-informed feedback in MOOCs mediated by instructors. We applied etic content analysis to the collected data. Results and Conclusions:The results revealed a lack of empirical studies exploring LA to deliver feedback, and limited attention on pedagogy to inform feedback practices. Our findings suggest the need for systematization and evaluation of feedback. Additionally, there is a need for conceptual tools to guide instructors' in the design of LA-based feedback.Takeaways: We point out the need for systematization and evaluation of feedback. We envision that this research can support the design of LA-based feedback, thus contributing to bridge the gap between pedagogy and data-driven practice in MOOCs.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Type Of Contributionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Delving into instructor‐led feedback interventions informed by learning analytics in massive open online courses

Topali

Chounta

Martínez‐Monés

et al. 2023

Computer Assisted Learning

View full text Add to dashboard Cite

show abstract

e-FeeD4Mi: Automating Tailored LA-Informed Feedback in Virtual Learning Environments

Ortega‐Arranz

Topali

Asensio-Pérez

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning to Give Useful Hints: Assistance Action Evaluation and Policy Improvements

Schmucker,

Pachapurkar,

Bala

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We describe a fielded online tutoring system that learns which of several candidate assistance actions (e.g., one of multiple hints) to provide to students when they answer a practice question incorrectly. The system learns, from large-scale data of prior students, which assistance action to give for each of thousands of questions, to maximize measures of student learning outcomes. Using data from over 190,000 students in an online Biology course, we quantify the impact of different assistance actions for each question on a variety of outcomes (e.g., response correctness, practice completion), framing the machine learning task as a multi-armed bandit problem. We study relationships among different measures of learning outcomes, leading us to design an algorithm that for each question decides on the most suitable assistance policy training objective to optimize central target measures. We evaluate the trained policy for providing assistance actions, comparing it to a randomized assistance policy in live use with over 20,000 students, showing significant improvements resulting from the system’s ability to learn to teach better based on data from earlier students in the course. We discuss our design process and challenges we faced when fielding data-driven technology, providing insights to designers of future learning systems.

show abstract

The MOOClet Framework: Unifying Experimentation, Dynamic Improvement, and Personalization in Online Courses

Cited by 7 publications

References 13 publications

Delving into instructor‐led feedback interventions informed by learning analytics in massive open online courses

Delving into instructor‐led feedback interventions informed by learning analytics in massive open online courses

e-FeeD4Mi: Automating Tailored LA-Informed Feedback in Virtual Learning Environments

Learning to Give Useful Hints: Assistance Action Evaluation and Policy Improvements

Contact Info

Product

Resources

About