M-SVR Model for a Serious Game Evaluation Tool

Abstract

Today, due to their interactive, participatory and entertaining nature, the Serious Games set themselves apart from other learning methods used in teaching. Much progress has been made in the design techniques and methods of Serious Games, but little in their evaluation. In order to fill this gap, we had proposed in our previous work an evaluation tool capable of helping practitioners to evaluate Serious Games in different training contexts. This evaluation tool for Serious Games is designed around four dimensions, namely the pedagogical, technological, ludic and behavioral dimensions, which are measured by clearly defined criteria. During this process, it was highlighted that the human factor (evaluator) influences considerably the result of the weightings through the choice to weight the evaluation dimensions of the Serious Games. In order to reduce this influence during the evaluation process and to keep the correlation between the variables of our evaluation system, we present in this paper, an improvement of our evaluation tool by equipping it with an intelligent supervised self-learning algorithm allowing self-regulation of the weights according to the context of use of the Serious Game to be evaluated. Thanks to the experimental verification of the optimization results, the root mean square error and the coefficient of determination are 0.016 and 98.59 percent respectively, indicating that the model has high precision which guaranteed better predictive performance. A comparison was made between this intelligent model and the models presented in our previous work; the results obtained indicated the same order of the four dimensions, and this by reducing the influence of the human factor during the Multi-Output Support Vector Regression weighting process.

A Comparative Analysis of Applied FMADM in a Serious Games Evaluation Model

“…The results of this study clearly indicate the justification for adopting the fuzzy AHP method in the Serious Games evaluation model. This finding was also confirmed by the study (Omari et al, 2021). For this reason, the application of fuzzy AHP has been strongly adopted in the literature (Kahraman et al, 2015).…”

Towards an Intelligent Model for Evaluating Serious Games

“…As such, we model the context of use of the serious game using the input vector X = {3,5,7,3,5,3}, which will be fed into the final model created by M-SVR to obtain the appropriate weights {y 1 , y 2 , y 3 , y 4 } for each evaluation dimension of the serious game, as illustrated in Table 4. The Table 4 shows that the results obtained from the intelligent weighting model indicate the same order of the four dimensions used in our two previous contributions [24,25]. It should be noted that the influence of the human factor, i.e., the evaluator, during the M-SVR weighting process is neglected.…”

“…Therefore, it is crucial to adjust all the criteria adopted according to the context of use of the serious game. In our two subsequent studies [24,25], we adopted methods based on the fuzzy multiattribute decision-making approach (FMADM) to address this issue. In our first study [24], we decided to use the fuzzy analytic hierarchy process (FuzzyAHP) method [26] to ensure the validity of evaluator preferences.…”

“…For the second study [25], we also used two other fuzzy multi-attribute decision-making methods-namely, Fuzzy TOPSIS [27] and Fuzzy ELECTRE [28]-to evaluate the same criteria in the same study context as in our first study. The results obtained by these FMADM methods led to a convergent ranking of the selected criteria/dimensions.…”

“…Step 6: Model deployment After training and fine-tuning the M-SVR model with the training data, the next step is to deploy the model in the serious game evaluation tool. In this step, we use the intelligent model created with the M-SVR algorithm in the same evaluation context as presented in our initial contributions [24,25], where the serious game is used purely for educational purposes and targets a university and scientific audience.…”