Computerized Card Sort Training Tool: Is it Comparable to Manual Card Sorting?

Abstract

One way to determine training needs and to evaluate learning is to measure how trainees organize knowledge using a card sorting task. While card sorting is a valid tool for assessing knowledge organization, it can be work intensive and error-prone when it is manually administered. For this reason, we developed a software tool that computerizes the card sort task. We present a study that was conducted to determine whether the computerized version of card sorting is comparable to the manual sort. One-hundred eight participants completed two card sorts, either two manual sorts, one manual and one computerized sort, or two computerized sorts. No differences were found between the administration methods with respect to card sort accuracy, test-retest scores, and number of piles created. Differences between the two methods were found in administration time and length of the pile labels. These differences disappeared after one computerized administration. We conclude that a computerized card sorting task is just as effective at eliciting knowledge as a manual card sort.

Using Card Sorts for Understanding Website Information Architectures: Technological, Methodological and Cultural Issues

“…Less than 10% of cards were grouped differently and the groups which emerged were the same. This finding is in line with previous studies by Bussolon et al [3] and Harper et al [7]. However, both oncard and online sorts produced groups from English speaking participants which were substantially different from the a priori groups, although the latter were based on an analysis of a substantial number of websites in each of the two domains.…”

“…Harper et al [7] developed their own card sort program and then evaluated it by comparing the program with traditional oncard sorts. 108 participants undertook two open card sorts each, either oncard -oncard, online -oncard, oncard -online, or online -online.…”

“…There is a small but growing body of research and guidelines on how to vary websites for users from different cultures [2,7], although the guidelines do not mention variations in information architecture. Kralisch, Yeu and Jali [11] used card sorts to investigate the differences between British, German, Malaysian and Russian participants in their understanding of medical terms that might be used in health information websites.…”

TPL—KATS-card sort: A tool for assessing structural knowledge

“…Because of this, TPL-KATS-card sort software has an advantage over previously used manual card sort methods. In addition, Harper, Jentsch, Rhodenizer-VanDuyne, and Sanchez (2002) investigated the differences between administering card sorts manually and with TPL-KATS-card sort. After extensive comparisons, the researchers concluded that with respect to important psychometric measures, no differences exist between the two methods.…”

Is a Picture Worth a Thousand Words? Investigating Structural Knowledge with Textual and Pictorial Stimuli

“…This experiment was performed using the TPL-KATS card-sorting program (see Figure 1) This software has been found to be comparable to traditional manual card sorting methods by Harper et al (2002). Participants executed the card sort program in much the same way as the solitaire game that is found on most PCs.…”