Supporting complex assessment processes

Villamañe, Mikel; Álvarez, Ainhoa

doi:10.1145/3144826.3145368

Cited by 2 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A promising research line is being able to automatically adjust those weights based on raters' past performance (Kulkarni et al, 2013) or by using a weight adjustment or optimization system (Villamañe, Álvarez, & Larrañaga, 2017). The first one is that the use of personalized students' weights from students' engagement information can improve peer grading.…”

Section: Discussionmentioning

confidence: 99%

“…So as a future direction in assessment approaches in MOOCs (Alcarria, Bordel, de Andrs, & Robles, 2018), we recommend using personalized or adaptive or configurable weights when aggregating student's peer grading scores. A promising research line is being able to automatically adjust those weights based on raters' past performance (Kulkarni et al, 2013) or by using a weight adjustment or optimization system (Villamañe, Álvarez, & Larrañaga, 2017). The second one is that performance-based weighted median scores can increase the correlation with potential instructor's grades.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Improving essay peer grading accuracy in massive open online courses using personalized weights from student's engagement and performance

García-Martínez

Cerezo

Bermúdez

et al. 2018

Computer Assisted Learning

View full text Add to dashboard Cite

Most massive open online courses (MOOC) use simple schemes for aggregating peer grades, taking the mean or the median, or compute weights from information other than the instructor's opinion about the students' knowledge. To reduce the difference between the instructor and students' aggregated scores, some proposals compute specific weights to aggregate the peer grades. In this work, we analyse the use of students' engagement and performance measures to compute personalized weights and study the validity of the aggregated scores produced by these common functions, mean, and median, together with two others from the information retrieval field, the geometric and harmonic means. To test this procedure, we have analysed data from a MOOC about Philosophy. The course had 1,059 students registered, and 91 participated in a peer review process that consisted in writing an essay and rating three of their peers using a rubric. We compared the aggregation scores obtained using weighted and nonweighted versions of the functions. Our results show that the correlation between the aggregated scores and the instructor's grades can be improved in relation to peer grading, when using the median and weights are computed according to students' performance in chapter tests.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Improving essay peer grading accuracy in massive open online courses using personalized weights from student's engagement and performance

García-Martínez

Cerezo

Bermúdez

et al. 2018

Computer Assisted Learning

View full text Add to dashboard Cite

show abstract

“…7) Committee: Variable or customized evaluation committees of different sizes or compositions should be allowed for each assessable item. Depending on the context of the assessment, sometimes different evaluation committees assess the assessable items from each course [21]- [23]. For example, the course's assessable items might be distributed among different lecturers.…”

Section: Requirements To Support Complex Continuous Assessment Processesmentioning

confidence: 99%

CASA: An Architecture to Support Complex Assessment Scenarios

2020

View full text Add to dashboard Cite

The evaluation or assessment of student performance and knowledge is a central element of most instructional design models, as it provides the information required to take remediation actions and improve the learning process. However, the assessment contexts present such a diverse range of cases due to course, teacher and student idiosyncrasies that it is difficult to support all the possibilities via software. To deal with this problem, this paper presents an architecture that satisfies the requirements to support complex assessment scenarios, those that potentially involve different evaluators using diverse assessment methods, tools and techniques at the same time. An analysis of the literature and the assessment scenarios of courses from Computer Engineering grades was carried out to identify the requirements for systems that support such complex assessment scenarios and to infer the information needed to support them. This information was modeled and formalized in the Complex Assessment Scenarios Architecture (CASA) that comprises functional modules, knowledge bases and the relationships among them. Following this architecture, an assessment system (AdESMuS) was developed using agile methodologies. AdESMuS has been satisfactorily tested under different complex assessment scenarios proving to be able to support those scenarios' requirements. This paper presents CASA, the architecture for developing a system to support complex assessment scenarios along with the satisfactory case study of AdESMuS.INDEX TERMS Software architecture, data models, knowledge management, learning management systems, requirements management, software tools.

show abstract

Supporting complex assessment processes

Cited by 2 publications

References 7 publications

Improving essay peer grading accuracy in massive open online courses using personalized weights from student's engagement and performance

Improving essay peer grading accuracy in massive open online courses using personalized weights from student's engagement and performance

CASA: An Architecture to Support Complex Assessment Scenarios

Contact Info

Product

Resources

About