On Characterization of Norm-Referenced Achievement Grading Schemes toward Explainability and Selectability

Abstract

Grading is the process of interpreting learning competence to inform learners and instructors of the current learning ability levels and necessary improvement. For norm-referenced grading, the instructors use a conventionally statistical method, z score. It is difficult for such a method to achieve explainable grade discrimination to resolve dispute between learners and instructors. To solve such difficulty, this paper proposes a simple and efficient algorithm for explainable norm-referenced grading. Moreover, the rise of artificial intelligence nowadays makes machine learning techniques attractive to the norm-referenced grading in general. This paper also investigates two popular clustering methods, K-means and partitioning around medoids. The experiment relied on the data sets of various score distributions and a metric, namely, Davies–Bouldin index. The comparative evaluation reveals that our algorithm overall outperforms the other three methods and is appropriate for all kinds of data sets in almost all cases. Our findings however lead to a practically useful guideline for the selection of appropriate grading methods including both clustering methods and z score.

Hybrid data analytic technique for grading fairness

“…Bai and Chen (2006) automatically constructed grade membership functions for fuzzy reasoning to infer student scores. Our prior works (Banditwattanawong and Masdisornchote, 2020a, b, 2021) proposed a heuristic for norm-referenced grading and demonstrated for the first time how to apply K -means and partitioning around medoids (PAM) (Kaufmann and Rousseeuw, 1987) to accomplish such a purpose. The grading result qualities of the heuristic, z score, K -means and PAM were comparatively quantified by using the Davies-Bouldin index, which is not suitable for grading fairness evaluation.…”

“…K -means (Witten and Frank, 2016; Banditwattanawong and Masdisornchote, 2021) is a machine learning technique applicable to norm-referenced grading. K -means partitions n objects into k clusters, which are exclusively mapped to eligible unique grade symbols.…”

“…Z score ( z ) (Wadhwa, 2008; Banditwattanawong and Masdisornchote, 2021) is a statistical grading method, which converts raw scores to z scores, which are subsequently converted to t scores to normalize a score range to be from 0 to 100 whereas z scores can be negative. The t scores are further sorted, and minimum-to-maximum t score range is divided by the desired number of unique grade symbols to derive the same score interval.…”

“…The relative criteria can be determined by a conventionally statistical means (i.e. z score), a heuristic (Banditwattanawong and Masdisornchote, 2021), or machine learning methods (Banditwattanawong and Masdisornchote, 2021). Despite these available methods, a common question to norm-referenced grading is how to clarify the fairness of grade results to all stakeholders (e.g.…”

“…To mitigate both unfair grading problems or to meet both fair norm-referenced grading requirements, we initially define grading fairness and its measurement metric to be subsequently used as the objective function of our novelly proposed hybrid algorithm for fair norm-referenced grading. Given a score data set, the algorithm will return a set of corresponding grades with the maximum degree of fairness based on three candidate internal methods: z score, K -means and our previously proposed heuristic (Banditwattanawong and Masdisornchote, 2021).…”