The study of how individuals organize knowledge has been a popular endeavor for several decades. As a result, techniques have been developed to assess how individuals represent and organize knowledge internally. Although several conceptual knowledge elicitation methods have been developed and used to assess the organization of knowledge, their use is often labor intensive and time consuming. Presented here is a software tool that was developed to reduce the problems associated with manually administering the conceptual knowledge elicitation technique, or card sorting. The TPL-KATS-card sort software not only simplifies the administration of the task, but also adds features to the card-sorting task such as media insertion, time stamping, and instructorless administration. In the present article, an introduction to the card-sorting technique is provided, the new software tool is described, and the advantages of the software are detailed.
One way to determine training needs and to evaluate learning is to measure how trainees organize knowledge using a card sorting task. While card sorting is a valid tool for assessing knowledge organization, it can be work intensive and error-prone when it is manually administered. For this reason, we developed a software tool that computerizes the card sort task. We present a study that was conducted to determine whether the computerized version of card sorting is comparable to the manual sort. One-hundred eight participants completed two card sorts, either two manual sorts, one manual and one computerized sort, or two computerized sorts. No differences were found between the administration methods with respect to card sort accuracy, test-retest scores, and number of piles created. Differences between the two methods were found in administration time and length of the pile labels. These differences disappeared after one computerized administration. We conclude that a computerized card sorting task is just as effective at eliciting knowledge as a manual card sort.
Previous research has indicated that the structure of an individual's knowledge may be just as important as the quantity of knowledge. Given this, using assessment methods that elicit trainee's knowledge structures seems imperative for predicting performance. Unfortunately, incorporation of these methods has been hindered, in part, due to the complexity of methods used to derive a score and the belief that simpler methods of scoring will not provide accurate information about an individual's knowledge. Presented here is a study that investigated whether this claim was true for the structural knowledge elicitation method, concept mapping. Twenty-six participants were run through a same-day training and assessment session. Following, concept maps were scored using a simple method and a complex method. Results indicated that both scoring methods produced significantly higher scores for the trained group and significantly lower scores for the untrained group. In addition, there was a very strong, positive relationship between the two scoring methods. Finally, both methods produced a moderately high correlation with the paper-pencil assessment. Results and implications are discussed further within the paper.
Card sorting and concept mapping are two popular techniques used for assessing structural knowledge. However, manual administration of these tasks has been cumbersome in the past. This demonstration serves as an introduction to the TPL-KATS – Card Sort and Concept Map software tool developed at the Team Performance Lab at the University of Central Florida. TPL-KATS is presented as a possible solution to the problems of manual administration. The software contains computerized versions of both techniques, with automatic administration and scoring capabilities. This demonstration provides an introduction to the card sort and concept map tasks, and describes the basic need for this software tool. A full description of TPL-KATS – Card Sort and Concept Map software is presented, along with a demonstration of its advantages over current techniques.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.