Toward automated computer-based visualization and assessment of team-based performance.

Abstract

A considerable amount of research has been undertaken to provide insights into the valid assessment of team performance. However, in many settings, manual and therefore labor-intensive assessment instru ments for team performance have limitations. Therefore, automated assessment instruments enable more flexible and detailed insights into the complex processes influencing team performance. The central objective of this study was to advance knowledge in automated assessment of team-based performance using a language-oriented approach. Fifty-six teams of learners (N = 224) in 3 experimental conditions solved 2 tasks in an online learning environment. They were analyzed with the Automated Knowledge Visualization and Assessment (AKOVIA) methodology. AKOVIA integrates a natural language-oriented algorithm and enables a structural and semantic compression of individual-and team-based knowledge representations. Findings indicate initial evidence of the feasibility and validity of the fully automated methodology. A framework for integrating research and methodology development is suggested for improving educational technology innovations such as computer-based assessment environments in international large-scale assessments.

Exploring learning how to learn in a team-based engineering education

“…As we develop the automated team-based assessment methodology further, this may help to overcome labour intensive manual analysis of team-based outcomes and provide opportunities for near real-time interventions. AKOVIA has been successfully implemented for the analysis of team-based performance with over 50 teams (Ifenthaler, 2014).…”

Application exercise design for team‐based learning in online courses

“…Assessment possibilities depend upon how team participation is captured, and what emphasis is placed on the required task in grading rubrics. Possible assessment includes instructor feedback of individual student contributions, instructor feedback on synchronous activities of a team captured via audio or video (Ifenthaler, 2014), written reasoning of team specific choices (Weinberger & Fischer, 2006) particularly for larger class sizes. Two opportunities for summative or formative instructor assessment are also suggested in Figure 4.…”

Assisting students’ writing with computer-based concept map feedback: A validation study of the CohViz feedback system

“…In the reliability study, we investigated the reliability of CohViz concept maps compared to concept maps based on propositional segmentations by human expert raters on a corpus of 100 expository texts of novice writers collected from previous experimental studies. To compare the concept maps, we used structural indicators commonly used in research on concept maps [32][33][34][35]. We assumed that CohViz concept maps yield similar results in the number of concepts, relations, and fragments compared to the human expert raters.…”