On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study

Abstract

In this study, we report on a Systematic Mapping Study (SMS) on how the quality of the quantitative instruments used to measure digital competencies in higher education is assured. 73 primary studies were selected from the published literature in the last 10 years in order to 1) characterize the literature, 2) evaluate the reporting practice of quality assessments, and 3) analyze which variables explain such reporting practices. The results indicate that most of the studies focused on medium to large samples of European university students, who attended social science programs. Ad hoc, self-reported questionnaires measuring various digital competence areas were the most commonly used method for data collection. The studies were mostly published in low tier journals. 36% of the studies did not report any quality assessment, while less than 50% covered both groups of reliability and validity assessments at the same time. In general, the studies had a moderate to high depth of evidence on the assessments performed. We found that studies in which several areas of digital competence were measured were more likely to report quality assessments. In addition, we estimate that the probability of finding studies with acceptable or good reporting practices increases over time.

Digital proficiency: Sorting real gaps from myths among higher education students

“…Therefore, studies using it and providing comparable results could be increasingly useful. However, only a few studies have explored higher education (HE) students' digital competence (Saltos‐Rivas et al., 2021), particularly using DigComp. Those that do so focus mainly on undergraduate students coming from Social Sciences programmes (Saltos‐Rivas et al., 2021).…”

“…However, only a few studies have explored higher education (HE) students' digital competence (Saltos‐Rivas et al., 2021), particularly using DigComp. Those that do so focus mainly on undergraduate students coming from Social Sciences programmes (Saltos‐Rivas et al., 2021). Additionally, such studies lack quality reliability and validity assessments and remain at HE students' self‐assessment of their digital competence, leaving a knowledge gap in relation to other related aspects (Saltos‐Rivas et al., 2021; Sillat et al., 2021): how does the proficiency level of students from different fields of education and training (FET) differ?…”

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

“…In the development of a quantitative instrument for assessment purposes, it is crucial to measure its quality [95][96][97]. However, recent reviews of tools for the assessment of DC concluded that the evidence provided is not enough [11,13,14]. In view of this, we designed our study by planning several studies throughout the different phases to obtain enough evidence to ensure the quality of the instruments.…”

“…In this context, assessing DC has become a topic of growing interest in recent years and relevant studies have examined the principal advances and limitations [8,[11][12][13][14]. However, despite employing different approaches, some issues still require further study; most of the assessment systems consist of self-assessments, do not cover the three components of DC (knowledge, skills and attitudes) and measure mainly low-order cognitive skills (according to the Bloom's taxonomy, the lower-order cognitive skills include remembering, understanding and applying, while the higher-order cognitive skills include analysing, evaluating and creating).…”

Level of Digital Teaching Competence on the Verge of the Post Pandemic

“…Thus, the use of devices such as cell phones, laptops and others increased their sales in just two years, being these a primary need at home to have a virtual education in the development of teaching-learning, the use of these devices significantly affected the academic performance of students as they have access to quality information and in abundance [25], forcing the teacher to do more research on the subject to be treated in the classroom [19]. Questionnaires on digital competencies were developed, designed, implemented and validated in different areas that had ICT as the first ally and showed difficulty in digital competencies [26] even mapping the educational institutions, the same results were obtained as in the higher level [27].…”