Validation of the Psychological Research Inventory of Concepts

Abstract

The current set of three studies further evaluates the validity and application of the Psychological Research Inventory of Concepts (PRIC). In Study 1, we administered the PRIC to a sample of introductory psychology students and online (Mechanical Turk) participants along with measures assessing theoretically related concepts. We found evidence of concurrent validity by demonstrating that PRIC scores relate to education, cognitive effort, and general psychological knowledge. In Study 2, we demonstrated that advanced psychology majors score higher on the PRIC than college graduates who did not major in psychology, suggesting that the PRIC assesses domain-specific knowledge and may be useful in assessment of the undergraduate psychology major. Finally, in Study 3, we demonstrated that scores on the PRIC increase from the start to the end of a research methods course. Together, these studies provide further evidence that the PRIC may be a useful index of learning research methods and statistical knowledge important for the undergraduate psychology degree.

Development of a Research Methods and Statistics Concept Inventory

“…Although the idea of a concept inventory is to assess research methods and statistical knowledge beyond grades, it would still be useful to know if PRIC scores are associated with higher grades in research methods or statistics courses. Thus, we recognize that further work is needed to provide further validation of the PRIC, and we turn to answering these questions in subsequent studies (Veilleux & Chapman, 2017, this issue).…”

Methods of assessing scientific competency in health service psychology master’s programs.

“…The PRIC has the advantages of a relatively short administration period (20-item, multiple-choice), is simple to score, and is designed for multiple administrations. The PRIC has demonstrated concurrent, convergent, and discriminant validity (Veilleux & Chapman, 2017b). Additionally, the Psychological Critical Thinking Exam (PCTE) was developed by Lawson, (1999) and contains 14 claims about seven distinct phenomena that violate an important principle of psychological science (e.g., poor control groups, poor sampling techniques, concluding causality from correlational evidence).…”

The measurement of psychological literacy and employable skills self-efficacy in undergraduate psychology students.

“…The PRIC is a 20-item vignette-based measure assessing knowledge about psychological research methods and statistics (Veilleux & Chapman, 2017a, 2017b). Vignettes and corresponding items represented topics covered throughout the undergraduate psychology program.…”

“…Eldon (2019) collected data from 18 psychology students at the University of Southern Queensland and compared this to data collected from 25 undergraduate psychology students at Curtin University. Her particular focus was on the 13 psychological research methods and statistics items in the TOPL (we refer to these items as the TOPL research methods [TOPL-RM]), and these were compared to the Psychological Research Inventory of Concepts (PRIC; Veilleux & Chapman, 2017a, 2017b) and a Short Test for the Assessment of Basic Knowledge in Psychology (STBKIP; Peter et al, 2015). The students’ year of enrollment (first, second, third, or fourth) and their usual coursework grade (fail, pass, credit, distinction, high distinction) were captured as well as other demographic details.…”

“…The goal of the present study was to assess the degree to which students’ psychological literacy levels in the area of research methods can be explained by their self-rated confidence in four employability skill domains, controlling for their concurrently assessed domain knowledge. One domain from the TOPL (Roberts & Gasson, 2018) was selected to provide an assessment of psychological literacy and was included in this study along with the ESSES and the PRIC (Veilleux & Chapman, 2017a, 2017b), a 20-item vignette-based multiple-choice measure assessing knowledge about research methods and statistics in psychology.…”