Conformance Analysis of Student Activities to Evaluate Implementation of Outcome-Based Education in Early of Pandemic using Process Mining

Abstract

The learning management system has a core component of a system event log that contains data on activities carried out by students and lecturers in the system. Educational process mining is a field in educational data mining that is concerned with finding, analyzing, and improving the overall educational process based on information hidden in educational data sets and event logs. The learning process in student lectures through the learning management system will produce a process flow according to the event data. In one semester in the information technology-based study program, the subjects taken are data from programming and non-programming courses in the 5th semester of the information systems department, namely Data Warehouse and Business Intelligence (DWBI) and Enterprise Architecture (EA). The Data Warehouse and Business Intelligence course is chosen because the main role in the graduate profile is as a data engineer. While the Enterprise Architecture course is chosen because being an IT Consultant requires knowledge of EA. Each course has different measured learning outcomes and each course has a different pattern in obtaining learning outcomes. To get a pattern for each learning achievement, an analysis of learning patterns, Bloom’s taxonomy level, and CLO pass scores was carried out using process mining. Course Learning Outcomes (CLO) is a competency standard or minimum qualification criteria for graduates’ abilities which include attitudes, knowledge, and skills assigned to courses. The existence of a bloom level indicates the level of expected learning achievement, where the higher the bloom level, the higher the expected ability. The mining process is carried out using Disco and PROM 5.2. The modeling process uses a heuristic miner algorithm because of its ability to express the main behavior recorded in the event log well. Heuristic miner algorithm can also take into account the frequency of the relationship between activities in the log to determine causal dependencies. The results of this study indicate that there is a difference between those that pass the course learning outcomes and those that do not pass. The passed CLO is indicated by the realization value of passing the course exceeding the threshold of 85.50%, while the failed CLO is indicated by the realization value of course graduation that is less than the threshold. In addition, control-flow, the frequency of activities that are often carried out indicate the appropriate learning path and are carried out by students to achieve a minimal assessment of course learning outcomes. In the Enterprise Architecture course, the activity that has the highest frequency in CLO1 is Attempt Quiz, while in CLO6 is View Course. In the Data Warehouse and Business Intelligence course, the activity that has the highest frequency in CLO3 is View Course, while in CLO4 is Attempt Quiz. The initial activity of the learning pattern produced in the two courses begins always with the view course activity. The highest bloom level in the Data Warehouse and Business Intelligence course is C6 Creation, while in the Enterprise Architecture course is C5 Evaluation. Thus, it can be said that Data Warehouse and Business Intelligence courses have a higher level of difficulty than Enterprise Architecture. Previously, in the DWBI course there was one CLO that failed in its implementation. With this research, it is hoped that this research can have a positive impact on adding new insights regarding the use of event logs in the field of education, so implementation of outcome-based education can be used as a benchmark for student learning to succeed in the course which include attitudes, knowledge, and skills.