Design of a curriculum analytics tool to support continuous improvement processes in higher education

Hilliger, Isabel; Aguirre, Camila; Miranda, Constanza; Celis, Sergio; Pérez‐Sanagustín, Mar

doi:10.1145/3375462.3375489

Cited by 24 publications

(27 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Causes may be associated with factors where the institution may or may not be involved, and therefore, they should be studied in more depth [54][55][56]. Third, the need to extend learning analytics initiatives at the program level, supporting curricular improvement processes [1,57].…”

Section: Implications For Managers and Policy Makersmentioning

confidence: 99%

See 1 more Smart Citation

Curricular Analytics to Characterize Educational Trajectories in High-Failure Rate Courses That Lead to Late Dropout

et al. 2021

View full text Add to dashboard Cite

Late dropout is one of the most pressing challenges currently facing higher education, and the process that each student follows to arrive at that decision usually involves several academic periods. This work presents a curricular analytics approach at the program level, to analyze how educational trajectories of undergraduate students in high-failure rate courses help to describe the process that leads to late dropout. Educational trajectories (n = 10,969) of high-failure rate courses are created using Process Mining techniques, and the results are discussed based on established theoretical frameworks. Late dropout was more frequent among students who took a stopout while having high-failure rate courses they must retake. Furthermore, students who ended in late dropout with high-failure rate courses they must retake had educational trajectories that were on average shorter and less satisfactory. On the other hand, the educational trajectories of students who ended in late dropout without high-failure rate courses they must retake were more similar to those of students who graduated late. Moreover, some differences found among ISCED fields are also described. The proposed approach can be replicated in any other university to understand the educational trajectories of late dropout students from a longitudinal perspective, generating new knowledge about the dynamic behavior of the students. This knowledge can trigger improvements to the curriculum and in the follow-up mechanisms used to increase student retention.

show abstract

Section: Implications For Managers and Policy Makersmentioning

confidence: 99%

“…In recent years, both researchers and practitioners have been applying learning analytics to support curriculum understanding and improvement [1,2]. Worldwide participation in higher education has grown around the world [3].…”

Section: Introductionmentioning

confidence: 99%

Curricular Analytics to Characterize Educational Trajectories in High-Failure Rate Courses That Lead to Late Dropout

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Nowadays, there is a stream of research that realises this vision. Data-driven "curriculum analytics" approaches use available data to derive metrics to characterise a programme's curriculum (Ochoa, 2016) or to improve the learning programme provision (Hilliger et al, 2020). Nguyen et al (2018) analysed the timing of students' engagement against the instructors' learning designs, and found misalignment of students' actual engagement and that planned out by the instructor.…”

Section: Learning Analytics To Improvement Education Practicementioning

confidence: 99%

Data-driven Exploration of Engagement with Workplace-based Assessment in the Clinical Skills Domain

Piotrkowicz

Wang

Hallam

et al. 2021

Int J Artif Intell Educ

View full text Add to dashboard Cite

The paper presents a multi-faceted data-driven computational approach to analyse workplace-based assessment (WBA) of clinical skills in medical education. Unlike formal university-based part of the degree, the setting of WBA can be informal and only loosely regulated, as students are encouraged to take every opportunity to learn from the clinical setting. For clinical educators and placement coordinators it is vital to follow and analyse students’ engagement with WBA while on placements, in order to understand how students are participating in the assessment, and what improvements can be made. We analyse digital data capturing the students’ WBA attempts and comments on how the assessments went, using process mining and text analytics. We compare Year 1 cohorts across three years, focusing on differences between primary vs. secondary care placements. The main contribution of the work presented in this paper is the exploration of computational approaches for multi-faceted, data-driven assessment analytics for workplace learning which includes:(i) a set of features for analysing clinical skills WBA data, (ii) analysis of the temporal aspects ofthat data using process mining, and (iii) utilising text analytics to compare student reflections on WBA. We show how assessment data captured during clinical placements can provide insights about the student engagement and inform the medical education practice. Our work is inspired by Jim Greer’s vision that intelligent methods and techniques should be adopted to address key challenges faced by educational practitioners in order to foster improvement of learning and teaching. In the broader AI in Education context, the paper shows the application of AI methods to address educational challenges in a new informal learning domain - practical healthcare placements in higher education medical training.

show abstract

“…In the past decade, educational data mining (Romero and Ventura, 2020) and academic analytics (Campbell and Oblinger, 2007) have helped the field of education move through a data revolution (Knight and Shum, 2017; Joksimovic et al , 2019). In the context of curriculum analytics (Hilliger et al , 2020), Varouchas et al (2018) have suggested a theoretical framework based on three key elements of process, engagement and content. Similarly, Gottipati and Shankararaman (2014) proposed a framework based on stakeholders, objectives, data and techniques.…”

Section: Purposementioning

confidence: 99%

“…Many tools have been used to attempt to manage, deliver and assess a curriculum, and integration of these activities should be a goal of curriculum analytics software (Hilliger et al , 2020). In medical education, in particular, multiple instruments have been developed for assessment of teaching quality and outcomes (Schiekirka et al , 2015) and assessing curriculum coverage and student activities in experiential settings (Olmos and Corrin, 2012).…”

Section: Purposementioning

confidence: 99%

Using technology to automate syllabus construction for programmatic, curricular, faculty and experiential assessment activities

Brill

Shirodkar

et al. 2021

IJEM

View full text Add to dashboard Cite

PurposeWith a growing need to assess multiple aspects of healthcare education, the goal of this study was to develop an innovative web-based application to streamline assessment processes and meet the increasingly complex role of the educational manager.Design/methodology/approachAARDVARC (Automated Approach to Reviewing and Developing Valuable Assessment Resources for your Curriculum) was created with the core function of standardizing course syllabi through the use of a web-based portal and the ability to query fields within the portal to collect multiple points of data. AARDVARC permits quick and efficient gathering of programmatic, curricular, faculty, teaching, preceptor and financial data to facilitate meaningful change and a shared responsibility of assessment. This software has allowed automatic completion of complex analytics each semester, including coverage of program outcomes, course learning objectives, teaching and assessment methods, course readings, topics covered in the curriculum, faculty teaching hours, experiential activities, coverage of disease states and scheduling of peer observation of teaching.FindingsThree years after its initial launch, AARDVARC is now used by 520 faculty, 60 staff, 44 preceptors and over 2,000 students across multiple health profession and science programs. Data analytics through AARDVARC have allowed the School to reimagine how assessment can be conducted and have provided a pathway for making evidence-based programmatic and curricular changes.Originality/valueThis original software has provided an innovative approach to conduct assessment that combines best practices in curriculum, assessment, data analytics and educational technology while improving the overall quality, speed, and efficiency of academic and business operations.

show abstract

Design of a curriculum analytics tool to support continuous improvement processes in higher education

Cited by 24 publications

References 4 publications

Curricular Analytics to Characterize Educational Trajectories in High-Failure Rate Courses That Lead to Late Dropout

Curricular Analytics to Characterize Educational Trajectories in High-Failure Rate Courses That Lead to Late Dropout

Data-driven Exploration of Engagement with Workplace-based Assessment in the Clinical Skills Domain

Using technology to automate syllabus construction for programmatic, curricular, faculty and experiential assessment activities

Contact Info

Product

Resources

About