Introducing COSMOS: a Web Platform for Multimodal Game-Based Psychological Assessment Geared Towards Open Science Practice

Abstract

We have established the COgnitive Science Metrics Online Survey (COSMOS) platform that contains a digital psychometrics toolset in the guise of applied games measuring a wide range of cognitive functions. Here, we are outlining this online research endeavor designed for automatized psychometric data collection and scalable assessment: once set up, the low costs and expenditure associated with individual psychometric testing allow substantially increased study cohorts and thus contribute to enhancing study outcome reliability. We are leveraging gamification of the data acquisition method to make the tests suitable for online administration. By putting a strong focus on entertainment and individually tailored feedback, we aim to maximize subjects' incentives for repeated and continued participation. The objective of measuring repeatedly is obtaining more revealing multitrial average scores and measures from various operationalizations of the same psychological construct instead of relying on single-shot measurements. COSMOS is set up to acquire an automatically and continuously growing dataset that can be used to answer a wide variety of research questions. Following the principles of the open science movement, this data set will also be made accessible to other publicly funded researchers, given that all precautions for individual data protection are fulfilled. We have developed a secure hosting platform and a series of digital gamified testing instruments that can measure theory of mind, attention, working memory, episodic long-and short-term memory, spatial memory, reaction times, eye-hand coordination, impulsivity, humor appreciation, altruism, fairness, strategic thinking, decision-making, and risk-taking behavior. Furthermore, some of the game-based testing instruments also offer the possibility of using classical questionnaire items. A subset of these gamified tests is already implemented in the COSMOS platform, publicly accessible and currently undergoing evaluation and calibration as normative data is being collected. In summary, our approach can be used to accomplish a detailed and reliable psychometric characterization of thousands of individuals to supply various studies with large-scale neurocognitive phenotypes. Our game-based online testing strategy can also guide recruitment for studies as they allow very efficient screening and sample composition. Finally, this setup also allows to evaluate potential cognitive training effects and whether improvements are merely task specific or if generalization effects occur in or even across cognitive domains.

Effective Gamification of the Stop-Signal Task: Two Controlled Laboratory Experiments (Preprint)

“…An individual's true ability will not be represented if they are not engaged and fully attentive, which can lead to inaccurate interpretations of cognitive task performance data [17]. To improve engagement with cognitive tasks, researchers have looked to games [25,26], with Aeberhard et al [27] noting that "leveraging gamification to repeatedly obtain behavioral samples paves the way for next-generation high-throughput psychometric toolset." However, caution must be taken when introducing game elements to cognitive tasks owing to the risk of muddying the measurement of the targeted cognitive process [28].…”

Construction of a Web Game for the Teaching-Learning Process of Electronics during the COVID-19 Pandemic

“…Technological tools enable students to acquire the requisite knowledge from the courses they attend, irrespective of their location (Aeberhard et al, 2019;Demir, 2020;Hawamdeh & Soykan, 2021;Hebert et al, 2021). The incorporation of web applications within the educational field favors the active role of students both inside and outside of the classroom (Burnett et al, 2020;Lim & Newby, 2020;Marchetti, 2021).…”

“…In particular, the use of web games in the educational field increases students' motivation and academic performance (Aeberhard et al, 2019;Hannig et al, 2012;Salas-Rueda et al, 2020). In an Introduction to Statistics course, the WGODS game helped to improve the assimilation and development of students' mathematical skills (Salas-Rueda et al, 2020).…”

“…At the RWTH Aachen University Medical School, students used a web game called eMedOffice in order to facilitate their learning process, and reportedly developed skills on the use of medical components (Hannig et al, 2012). Similarly, the COSMOS game helped to facilitate the skills development of learners in the field of psychology (Aeberhard et al, 2019).…”