A design science research methodology for developing a computer-aided assessment approach using method marking concept

Genemo, Hussein; Miah, Shah Jahan; McAndrew, Alasdair

doi:10.1007/s10639-015-9417-1

Cited by 24 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…DBR is considered a DSR realization in the education sector to conduct research to develop and evaluate an BDAS as an IT and DSR artefact. DSR complements DBR and provides multi-paradigm perspectives to construct fundamental knowledge by researching social pragmatisms (30)(31)(32).…”

Section: Integrated Design Science Research Methodologymentioning

confidence: 99%

Designing and Evaluating a Big Data Analytics Approach for predicting students’ success factors

Fahd

Miah

2022

Preprint

View full text Add to dashboard Cite

Reducing student attrition in tertiary education plays a significant role in the core mission and financial well-being of an educational institution. The availability of big data source from Learning Management System (LMS) can be analysed to help with the attrition issues. This study aims to use an integrated Design science research (DSR) methodology to develop and evaluate a Big Data Analytical Solution (BDAS) embedded in Educational Decision Support System as an educational artefact. The BDAS as DSR artefact harnesses the outcome of the application of Artificial Intelligence(AI) based approaches on the dataset collected from student interaction with LMS to train a predictive model to predict potential students at risk. Identifying students at risk helps to take timely intervention in the learning process to improve student academic progress to increase the retention rate. To evaluate the performance of the predictive model, we compare the accuracy of the collection of representational Artificial Intelligence algorithms in the literature. The BDAS aims not to replace any existing practice but to support educators to implement a variety of pedagogical practices to improve students’ academic performance.

show abstract

Section: Integrated Design Science Research Methodologymentioning

confidence: 99%

Designing and Evaluating a Big Data Analytics Approach for predicting students’ success factors

Fahd

Miah

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the need to ask questions that can be marked by computer leads to reduction in validity, as this tends to focus questions on procedural aspects (Broughton, Hernandez-Martinez and Robinson, 2017). The final answer input into a computer is not necessarily enough to establish partial credit (Rønning, 2017), a normal part of assessment in mathematics (Genemo, Miah & McAndrew, 2016), leading to a solution with a minor error potentially being marked as completely wrong (Greenhow, 2015). Some systems attempt to address this by breaking questions into parts, steps or sub-questions, prioritising procedural aspects and so reducing validity (Quinney, 2010).…”

Section: Summative Assessment: Validity Reliability and Efficiencymentioning

confidence: 99%

Partially-automated individualized assessment of higher education mathematics

Rowlett

2020

International Journal of Mathematical Education in Science and

View full text Add to dashboard Cite

A partially-automated method of assessment is proposed, in which automated question setting is used to generate individualised versions of a coursework assignment, which is completed by students and marked by hand. This is designed to be (a) comparable to a traditional written coursework assignment in validity, in that complex and open-ended tasks can be set with diverse submission formats that would not be suitable for written examination or automated marking; and, (b) comparable to e-assessment in terms of reduction of academic misconduct, with individualisation acting as a barrier to copying and collusion.This method of assessment is implemented in practice. Evaluation focuses on expert second-marking, student feedback and analysis of marks, and aims to establish that the partially-automated method can be useful in practice. The partially-automated method proposed appears to be capable of adapting a coursework assignment to make it less sensitive to copying and collusion (and therefore more reliable) while maintaining its validity, though leading to reduced efficiency for the marker. This paper therefore contributes the introduction of a novel approach to assessment which offers a way to bring automated individualisation to the assessment of higher order skills in higher education mathematics.

show abstract

“…Therefore, the problem of assessing the students' full workings, while evaluating MSQs using CAA for Mathematics, can be classified as a wicked problem. In MMC, a student's ability to produce a final result can be determined by assessing the sequence of steps used to produce the steps and final answers (Genemo et al, 2015).…”

Section: Issues Of Assessing Multi-step Questionsmentioning

confidence: 99%

A Design Science Research Methodology for Expert Systems Development

Miah

Genemo

2016

AJIS

Self Cite

View full text Add to dashboard Cite

The knowledge of design science research (DSR) can have applications for improving expert systems (ES) development research. Although significant progress of utilising DSR has been observed in particular information systems design -such as decision support systems (DSS) studies -only rare attempts can be found in the ES design literature. Therefore, the aim of this study is to investigate the use of DSR for ES design. First, we explore the ES development literature to reveal the presence of DSR as a research methodology. For this, we select relevant literature criteria and apply a qualitative content analysis in order to generate themes inductively to match the DSR components. Second, utilising the findings of the comparison, we determine a new DSR approach for designing a specific ES that is guided by another result -the findings of a content analysis of examination scripts in Mathematics. The specific ES artefact for a case demonstration is designed for addressing the requirement of a 'wicked' problem in that the key purpose is to assist human assessors when evaluating multi-step question (MSQ) solutions. It is anticipated that the proposed design knowledge, in terms of both problem class and functions of ES artefacts, will help ES designers and researchers to address similar issues for designing information system solutions.

show abstract

A design science research methodology for developing a computer-aided assessment approach using method marking concept

Cited by 24 publications

References 37 publications

Designing and Evaluating a Big Data Analytics Approach for predicting students’ success factors

Designing and Evaluating a Big Data Analytics Approach for predicting students’ success factors

Partially-automated individualized assessment of higher education mathematics

A Design Science Research Methodology for Expert Systems Development

Contact Info

Product

Resources

About