The Inventory of Documented Accomplishments for Graduate Admissions: Results of a Field Trial Study of Its Reliability, Short‐term Correlates, and Evaluation

Abstract

This is the report of the third phase of a multi‐phase project designed to develop an inventory of documented accomplishments for graduate student selection. The material developed in earlier stages was pilot‐tested in Phase Three in 26 departments that represented the fields of English, biology, and psychology. The major purpose of this study was to investigate the instrument developed in Stage Two in terms of: 1) technical soundness, and 2) the feasibility of its use in the admissions process. This study represents the transition of the use of documented accomplishments to predict graduate school achievement from a research phase to a more operational phase during which the instrument would be tested in actual admissions situations. The departments were asked to administer the instrument to their newly enrolled graduate students. Straightforward descriptive and correlational analyses of the responses to the inventory were conducted to: (1) describe characteristics of the students and the instruments; and (2) identify the most reliable clusters of items and indices of accomplishment. Students were followed up at the end of their first year to assess their graduate school accomplishments and their relationship to previous attainments. This information provided data for analyses of the short‐term correlates of the measures. In addition, faculty were interviewed to determine the plausibility of the content of the instrument and the most accurate and operationally simple instructions and reporting formats. Students' reactions to the inventory were obtained by interview and by additional questions at the end of the inventory.

The Role of Noncognitive Constructs and Other Background Variables in Graduate Education

“…Several studies have demonstrated the relationship between prior accomplishments and academic experiences and later academic success. Baird and Knapp (1981) found that pre-graduate school accomplishments predicted a host of graduate school accomplishments, although not graduate school grades.…”

“…Self-report measures often included in graduate applications (e.g., relevant coursework) can also yield prior educational information. Baird (1979) created a prior accomplishments inventory for graduate school applicants that included literary and expressive, scientific and technical, artistic, and social service organizational activities (Baird and Knapp, 1981). Stricker (1983) created a documented accomplishments scale, designed to enable validation, that is organized into academic achievement, leadership, practical language, aesthetic expression, science, and mechanical domains.…”

Research on Validity Theory and Practice at ETS

“…The biographical inventory developed in earlier stages was evaluated in 26 graduate departments that represented the fields of English, biology, and psychology. Responses to the inventory were generally not related to graduate grades, but a number of inventory responses reflecting preadmission accomplishments were significantly related to accomplishments in graduate school (Baird and Knapp 1981). Lawrence Stricker and colleagues further refined measures of documented accomplishments (Stricker et al 2001).…”

Advancing Human Assessment: A Synthesis Over Seven Decades

“…This decline coincided with a drop in attention to personal qualities research more generally, following a shift in ETS management priorities from the very clear think tank orientation of the 1960s and 1970s to a greater focus on research to assist existing testing programs and the creation of new ones. That focus remained centered largely on traditional academic abilities, though limited research proceeded on creativity (Baird and Knapp 1981;Ward et al 1980). …”