Lessons learned in adapting a lab-based measure of working memory capacity for the web

Abstract

The purpose of this project was to document the challenges in adapting a measure of complex working memory span for delivery over the web. Automated operation span (AOSPAN;Unsworth, Heitz, Schrock, & Engle, 2005) is one of the most widely used tests for measuring working memory capacity. However, AOSPAN is a computer-based test that runs locally and thus is only suitable for lab-based studies. We have recently created a web-based version of AOSPAN that runs on a much wider variety of platforms (e.g., mobile phones and tablets). Conclusions and recommendations are applicable to researchers who are thinking of conducting studies online and developing web-based measures of which are traditionally used in the laboratory. OBJECTIVESThe purpose of this project is to document recent developments in online studies and uses for web-based applications, specifically webSPAN. The presentation summarized previous problems with data collected in an online environment, possible solutions to those challenges through design recommendations, and the lessons learned from developing the webSPAN. The presentation was followed by interactive demonstrations with audience members.

Building, Hosting and Recruiting: A Brief Introduction to Running Behavioral Experiments Online

“…There are some aspects researchers should consider when starting out with running online studies or transferring lab-based experiments to online systems [59,75,76] (see Figure 2). To a certain extent, creating successful online experiments is similar to app development: one needs to think of a coherent framework and constantly worry about what the users are doing with the 'product' and whether they are using it as intended-without many opportunities for direct feedback.…”

Measuring individual differences in cognitive abilities in the lab and on the web

“…This modification made to the web-based version was motivated by technical reasons, specifically, the browser window may not register the participants’ response if the cursor is not over a certain area on the page, which in turn may cause problems of missing data. Previous research has found that participants in web-based research are particularly prone to err when using the keyboard to input their responses [60], which in this case might have affected the results of the comparison between lab-based and web-based versions of CVMT. Future research comparing performance between the lab and web-versions may benefit from collecting data through touch input instead, as this might help overcome potential technical difficulties caused by using mouse clicking for web-based data.…”

Building, hosting, recruiting: A brief introduction to running behavioral experiments online

“…There are some aspects researchers should consider when starting out with running online studies or transferring lab-based experiments to online systems [59,60]. To a certain extent, creating successful online experiments is similar to app development: one needs to think of a coherent framework and constantly worry about what the users are doing with the 'product' and whether they are using it as intended -without many opportunities for direct feedback.…”