Exploring the structure of digital literacy competence assessed using authentic software applications

Reichert, Frank; Zhang, Deju; Law, Nancy; Wong, Gary K. W.; Torre, Jimmy de la

doi:10.1007/s11423-020-09825-x

Cited by 27 publications

(25 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It appears that the DL assessed may not be independent of the technology used in the assessment, such as different digital devices, software packages, or testing platforms (cf. Reichert, Zhang et al., 2020). Moreover, technical equipment and infrastructure (e.g., internet connection) can be hurdles for remote assessment.…”

Section: Insights From the Pilot Studymentioning

confidence: 99%

Measuring Digital Literacy during the COVID‐19 Pandemic: Experiences with Remote Assessment in Hong Kong

Pan

Reichert

Torre

et al. 2022

Educational Measurement

Self Cite

View full text Add to dashboard Cite

Hong Kong was one of the first places hit with the COVID-19 outbreak in January 2020 and has since responded with several periods of full or partial school suspension, during which teaching and learning often occurred remotely using digital means. During this time, collecting research data and testing students have become more difficult than before. In this article, we report on a large-scale digital literacy (DL) performance assessment in Hong Kong that had to be conducted in a mix of modes-onsite, online supported, and online self-directed-during the pandemic. We discuss the administrative and methodological challenges of assessing DL, and explore preliminary data on the associations between student performance and the assessment mode adopted. We further discuss potential implications, including for high-stakes testing, and make recommendations for assessing DL remotely.

show abstract

Section: Insights From the Pilot Studymentioning

confidence: 99%

Measuring Digital Literacy during the COVID‐19 Pandemic: Experiences with Remote Assessment in Hong Kong

Pan

Reichert

Torre

et al. 2022

Educational Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…In view of this, we designed our study by planning several studies throughout the different phases to obtain enough evidence to ensure the quality of the instruments. Although the multidimensionality of the construct of DC has been identified in several studies, theoretical and empirical studies have reported contradicting results [32]. More specifically, Vuorikari et al [25] theoretically described the construct of DC in DigComp framework 2.0, but the dimensions identified have not been empirically confirmed or require further research.…”

Section: Discussionmentioning

confidence: 99%

“…However, theoretical and empirical studies have reported contradicting results. For example, Reichert et al analysed the most commonly used digital literacy frameworks and found that empirical evidence on the use of digital applications allows for distinguishing between a general digital literacy component and four applicationspecific components (the web-based information retrieval factor, the knowledge-based information retrieval factor, the word processing factor and the digital presentation factor) [32]. Jin et al found in their custom implementation based on DigComp that DC can be considered as a general one-dimensional construct [33].…”

Section: Reference Framework For the Evaluation Of DCmentioning

confidence: 99%

“…[83]. Therefore, we defined an initial solution based on key studies identified [24,86,87] and from other key studies identified in the literature review, such as [8,9,11,32]. Once the new design proposition was established, it was necessary to examine and improve it after the testing and analysis [83].…”

Section: Phase 2: Development Of Theoretical Framework Solutions Base...mentioning

confidence: 99%

See 1 more Smart Citation

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Bartolomé

Garaizar

2022

Sustainability

View full text Add to dashboard Cite

Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.

show abstract

“…Only few studies capture online information use in a realistic online environment (in real time) while gathering both process and performance data and assessing development over time using pre-post-measurements. Only very few studies applied intervention designs and investigated how online media use can be promoted in educational practice (Booth, 2011; for process-related studies, Tossell et al, 2015;Gadiraju et al, 2018;Yu et al, 2018;Goldhammer & Zehner, 2017; for instruments for assessing digital media competencies and for relevant concepts, such as digital literacy, see, e.g., Reichert et al, 2020). With regard to the promotion of competent media use, only few intervention studies were found; none focused on young professionals or practical training phases.…”

Section: Current State Of Researchmentioning

confidence: 99%

Performance Assessment and Digital Training Framework for Young Professionals' Generic and Domain-Specific Online Reasoning in Law, Medicine, and Teacher Practice

Zlatkin‐Troitschanskaia

Brückner²,

Nagel³

et al. 2021

JoSPoE

View full text Add to dashboard Cite

In the digital age, the Internet is increasingly considered a major information source. This is especially true for informal, e.g., post-university, learning. Evidentially, young professionals are increasingly using online sources as an information and learning tool. Critical reasoning from online information for learning and professional processes in the domains of medicine, law, and teaching is considered a highly relevant competence facet. For example, staying up to date on a multitude of matters, e.g., published in articles and guidelines, as is the case in the medical field, can be challenging when the required competencies to use online media are absent (e.g., Allen et al. 2005, O'Carroll et al. 2015). Current research on students in higher education indicates substantial deficits in their critical online reasoning skills, also among graduates. However, online information seeking and corresponding competencies of young professionals in job-specific educational processes have not been researched yet. There is a lack of both valid domain-specific assessments for different professions and learning tools that can effectively foster the competent use of online information in practice among young professionals. Our research presented here is part of the collaborative BRIDGE project, which is conducted under the umbrella of the program "Research for the Design of Educational Processes under the Conditions of Digital Change." This study is based on our previous work on the assessment of generic skills in higher education in the international projects CLA+, iPAL, and CORA as well as on experiences with job-specific performance assessments from the research programs KoKoHs and ASCOT+, which measured professional competence. To validly measure critical online reasoning among young professionals from three domains — medicine, law, and teacher training —we develop new computer-based online performance assessments and corresponding training tools. The specific aim is to analyze to what extent they improve in using online information with greater reflection when creating job-specific documents after an online training based on process and performance data (using innovative approaches, such as text mining and educational data mining). In this paper, we showcase the conceptual and assessment framework of the newly developed innovative tools to measure and promote generic and domain-specific online reasoning among young professionals in medicine, law, and teacher education. Based on this framework, we discuss how these crucial professional competence facets can be validly measured and effectively fostered in practice.

show abstract

Exploring the structure of digital literacy competence assessed using authentic software applications

Abstract: Take-down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Cited by 27 publications

References 31 publications

Measuring Digital Literacy during the COVID‐19 Pandemic: Experiences with Remote Assessment in Hong Kong

Measuring Digital Literacy during the COVID‐19 Pandemic: Experiences with Remote Assessment in Hong Kong

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Performance Assessment and Digital Training Framework for Young Professionals' Generic and Domain-Specific Online Reasoning in Law, Medicine, and Teacher Practice

Contact Info

Product

Resources

About