How Do Mathematics Teachers in Higher Education Look at E-assessment with Multiple-Choice Questions

Azevedo, José; Oliveira, Ema Patrícia; Beites, Patrícia Damas

doi:10.5220/0006324801370145

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CTT and IRT were used in order to analyze the quality of the MCQ included in a bank of questions, previously developed and implemented in Moodle, that is used to assess students through randomly generated tests (Azevedo, 2015; Azevedo, Oliveira and Beites, 2017; Azevedo, Torres, Lopes and Babo, 2017). In global terms, the purpose is to evaluate the quality of the questions through the use of metrics, bearing in mind that one intends the tests carried out by students to be as uniform and fair as possible.…”

Section: Methodsmentioning

confidence: 99%

“…LA is focused on the educational challenge, monitoring the learning processes for decision making (Ferguson, 2012; Greller and Drachsler, 2012). According to the Society for Learning Analytics Research, “learning analytics is the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purpose of understanding and optimizing learning and the environments in which it occurs” (Azevedo, Oliveira and Beites, 2017; Azevedo, Torres, Lopes and Babo, 2017, pp. 641-642).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Learning Analytics to evaluate the quality of multiple-choice questions

Azevedo

Oliveira

Beites

2019

IJILT

View full text Add to dashboard Cite

Purpose The purpose of this paper is to find appropriate forms of analysis of multiple-choice questions (MCQ) to obtain an assessment method, as fair as possible, for the students. The authors intend to ascertain if it is possible to control the quality of the MCQ contained in a bank of questions, implemented in Moodle, presenting some evidence with Item Response Theory (IRT) and Classical Test Theory (CTT). The used techniques can be considered a type of Descriptive Learning Analytics since they allow the measurement, collection, analysis and reporting of data generated from students’ assessment. Design/methodology/approach A representative data set of students’ grades from tests, randomly generated with a bank of questions implemented in Moodle, was used for analysis. The data were extracted from a Moodle database using MySQL with an ODBC connector, and collected in MS ExcelTM worksheets, and appropriate macros programmed with VBA were used. The analysis with the CTT was done through appropriate MS ExcelTM formulas, and the analysis with the IRT was approached with an MS ExcelTM add-in. Findings The Difficulty and Discrimination Indexes were calculated for all the questions having enough answers. It was found that the majority of the questions presented values for these indexes, which leads to a conclusion that they have quality. The analysis also showed that the bank of questions presents some internal consistency and, consequently, some reliability. Groups of questions with similar features were obtained, which is very important for the teacher to develop tests as fair as possible. Originality/value The main contribution and originality that can be found in this research is the definition of groups of questions with similar features, regarding their difficulty and discrimination properties. These groups allow the identification of difficulty levels in the questions on the bank of questions, thus allowing teachers to build tests, randomly generated with Moodle, that include questions with several difficulty levels in the tests, as it should be done. As far as the authors’ knowledge, there are no similar results in the literature.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Using Learning Analytics to evaluate the quality of multiple-choice questions

Azevedo

Oliveira

Beites

2019

IJILT

View full text Add to dashboard Cite

show abstract

“…The evaluation of a student is traditionally done by written examinations in educational institutes. In recent years researchers have proposed different approaches to generate examination questions to analyze the efficacy of the existing learning method [5]. Among different approaches, questions prepared by matching the intended learning outcomes (ILO) of a particular course module are proven to be effective to measure the overall learning progress of a student.…”

Section: Introductionmentioning

confidence: 99%

Cognitive Evaluation of Examinees by Dynamic Question Set Generation based on Bloom’s Taxonomy

Dutta

Chatterjee

Dey

et al. 2023

IETE Journal of Research

View full text Add to dashboard Cite

Educational data mining (EDM) is an emerging topic in recent years steered by data mining and machine learning techniques to enhance students' overall learning experience and academic progress. In recent years EDM techniques are frequently used to improve assessment systems but the evaluation procedure is majorly marks driven. Developing an evaluation system to distinguish candidates, based on their ability to answer cognitively difficult questions is a challenging task. In this study, a unique methodology is proposed to dynamically rank the candidates to develop an outcome-based online examination system that will properly evaluate a candidate's cognitive competencies. The questions are segmented into different cognitive groups based on classical Bloom's educational taxonomy. The Jenks Natural Breaks Optimization technique is used here to segment the questions and as a result, distinct question clusters based on different cognitive levels are obtained. Students are evaluated with different questions from these cognitive groups and ranking is done for individual candidates considering both the marks of the questions and his/her ability to solve questions from different difficulty levels.

show abstract