How Difficult is the Task for you? Modelling and Analysis of Students' Task Difficulty Sequences in a Simulation-Based POE Environment

Abstract

Task difficulty (TD) reflects students' subjective judgement on the complexity of a task. We examine the TDs data of 236 undergraduate students in a simulation-based Predict-Observe-Explain (POE) environment using three different labels easy, medium and hard. Generally, the students who perceive the tasks to be easy or hard perform poorly at the transfer task than the students who perceive the tasks to be medium or moderately difficult. Sequences of students' TDs are analysed which consist of a set of several judgements, collected once for each task in a POE sequence. The analysis suggests that given a sequence of TDs, difficulty level hard followed by a hard may lead to poorer learning outcomes at the transfer task. By contrast, difficulty level medium followed by a medium may lead to better learning outcomes at the transfer task. In terms of the TD models, we identify student behaviours that can be reflective of their perceived difficulties. Generally, the students who report that the tasks are easy, adopt a trial-and-error behaviour where they spend lesser time and make more attempts on tasks. By comparison, the students who complete the tasks in a longer time by making more attempts are likely to report that the following task is hard. For the students who report medium TDs, mostly these students seem to reflect on tasks where they spend a long time and require fewer attempts for task completions. Additionally, these students provide longer texts for explaining their hypothesis reasoning. Understanding how student behaviours and TDs manifest over time and how they impact students' learning outcomes is useful, especially when designing for real-time educational interventions, where the difficulty of the tasks could be optimised for students. It can also help in designing and sequencing the tasks for the development of effective teaching strategies that can maximise students' learning.

The Neglected 15%: Positive Effects of Hybrid Human-AI Tutoring Among Students with Disabilities

“…1). Although the latter may not provide direct insight into the impact of the tutoring treatment, it aligns with past AIED work [18] in highlighting the challenges involving diverse learners when applying AI-based learning interventions.…”

“…Addressing biases and avoiding discriminatory practices in the collection and use of learning analytics is crucial to fostering inclusivity [5]. For instance, past work found nuances in growth patterns among students with disabilities, emphasizing the need to set student's individualized growth patterns by providing nuanced expectations of growth rather than merely setting lower expectations based on pre-defined group measures [18]. AIED must work together to provide an inclusive ecosystem that empowers all students to thrive.…”

Curriculum Analytics of Course Choices:

“…Later, one-sample (two-tailed) t-tests were conducted on the calculated L values to measure whether each transition was significantly more or less likely than chance for each group. Then, Benjamini-Hochberg (BH) post-hoc correction was applied to control for false positives since the analysis involves multiple comparisons (Nawaz, Srivastava, Yu, et al, 2022).…”

Question Dosage in MOOCs

“…Questions were drawn from the Motivated Strategies for Learning Questionnaire (MSLQ) by (Pintrich & De Groot, 1991) and were slightly modified for the domain 1 . In addition, learners' weekly perception surveys based on prior work by (Nawaz et al, 2022) were also collected, where each week, the learners were asked how confident and challenged they felt that week about the learning content. Lastly, a demographics survey was also conducted, asking learners about their age, gender, employment status, education level, prior experience with MOOCs, English proficiency, country of residence and background knowledge in Microeconomics.…”