Measuring non-cognitive predictors in high-stakes contexts: The effect of self-presentation on self-report instruments used in admission to higher education

Niessen, A. Susan M.; Meijer, Rob R.; Tendeiro, Jorge N.

doi:10.1016/j.paid.2016.11.014

Cited by 36 publications

(32 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers clearly call against using self-report measures in student selection (Murphy et al 2009;Niessen et al 2017;Patterson et al 2016;Wouters 2016). It was demonstrated that self-reported measures induce a self-presentation effect, or, in other words, applicants tend to fake their reports in the selection context, which represent a high-stake situation (Niessen et al 2017).…”

Section: Discussionmentioning

confidence: 99%

“…Namely, when it comes to assessment of personality, motivation, and attitudes, such tools as the interviews, recommendation letters, CVs and motivation letters are often used. Interviews, CVs and motivation letters represent self-reported measures and when applied in a high-stake situation of selective admission, they are susceptible to faking by the applicants (Niessen et al 2017). When interpreting recommendation letters, a measure which is not self-reported, the admission committee members sometimes indicate a need Bto read between the lines^or Bto detect an extra glow of enthusiasm about the student^ (Walpole et al 2002 p. 5), which indicates the susceptibility of this measure to biases as well.…”

Section: How Do Admission Committees Select?mentioning

confidence: 99%

See 1 more Smart Citation

How do admission committees select? Do applicants know how they select? Selection criteria and transparency at a Dutch University

2019

View full text Add to dashboard Cite

In this study, we investigated the currently applied selective admission criteria and tools of the two-year research master's programs of both the Graduate Schools of Life Sciences and Natural Sciences of Utrecht University (the Netherlands). In addition, we evaluated their transparency to applicants. Both admissions staff members and applicants participated. To determine admission criteria that are important for admission decisions, we ranked 51 admission criteria and, on their basis, combined into six domains: academic background, grades, cognitive ability, research background, personality and personal competencies, motivation factors. To evaluate transparency, we contrasted the perceptions of applicants with the actual importance of admission criteria, as reported by admission staff members. We found that admissions criteria related to personality and personal competencies are less important in admission decisions than criteria related to grades, academic background and motivation. The applicants find the admissions decisions transparent to a moderate degree. This study also revealed that selectors use criteria and tools both with and without predictive value for later graduate performance. Moreover, some of the currently applied admission instruments might be prone to admission biases. We advocate selectors to use admission criteria and tools that are evidence-based, resistant to admission biases, and transparent to the applicants.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: How Do Admission Committees Select?mentioning

confidence: 99%

How do admission committees select? Do applicants know how they select? Selection criteria and transparency at a Dutch University

2019

View full text Add to dashboard Cite

show abstract

“…Fourth, during both test administrations, the SJT was administered for research purposes only, which might reduce the generalizability of our findings to real selection settings. However, Niessen et al () found large score differences on several noncognitive measures between a research and an admission context, even though applicants were informed that the noncognitive measures were not used for selection. Additionally, the difference in extreme responding indicated that the applicants responded differently on T 2.…”

Section: Discussionmentioning

confidence: 99%

Faking on a situational judgment test in a medical school selection setting: Effect of different scoring methods?

Leng

Stegers‐Jager

Born

et al. 2019

Int J Selection Assessment

View full text Add to dashboard Cite

We examined the occurrence of faking on a rating situational judgment test (SJT) by comparing SJT scores and response styles of the same individuals across two naturally occurring situations. An SJT for medical school selection was administered twice to the same group of applicants (N = 317) under low‐stakes (T1) and high‐stakes (T2) circumstances. The SJT was scored using three different methods that were differentially affected by response tendencies. Applicants used significantly more extreme responding on T2 than T1. Faking (higher SJT score on T2) was only observed for scoring methods that controlled for response tendencies. Scoring methods that do not control for response tendencies introduce systematic error into the SJT score, which may lead to inaccurate conclusions about the existence of faking.

show abstract

“…Most existing assessments rely on self‐reports, and there are almost no studies that investigated the effectiveness of noncognitive predictors such as personality traits or study skills and habits in high‐stakes procedures. Due to faking, the generalizability of the validity of scores obtained in low‐stakes conditions on such instruments to high‐stakes admission conditions is limited (Niessen, Meijer, & Tendeiro, ; Peterson, Griffith, Isaacson, O'Connell, & Mangos, ).…”

Section: Female Underpredictionmentioning

confidence: 99%

Gender‐Based Differential Prediction by Curriculum Samples for College Admissions

Niessen

Meijer

Tendeiro

2019

Educational Measurement

Self Cite

View full text Add to dashboard Cite

A longstanding concern about admissions to higher education is the underprediction of female academic performance by admission test scores. One explanation for these findings is selection system bias, that is, not all relevant KSAOs that are related to academic performance and gender are included in the prediction model. One solution to this problem is to include these omitted KSAOs in the prediction model, many of these KSAOs are 'noncognitive' and “hard‐to‐measure” skills in a high‐stakes context. An alternative approach to capture relevant KSAOs is using representative performance samples. We examined differential prediction of first year‐ and third year academic performance by gender based on a curriculum‐sampling test that was designed as a small‐scale simulation of later college performance. In addition, we examined differential prediction using both frequentist and Bayesian analyses. Our results showed no differential prediction or small female underprediction when using the curriculum‐sampling tests to predict first year GPA, and no differential prediction for predicting third year GPA. In addition, our results suggest that more comprehensive curriculum samples may show less differential prediction. We conclude that curriculum sampling may offer a practically feasible method that yields minimal differential prediction by gender in high‐stakes operational selection settings.

show abstract

Measuring non-cognitive predictors in high-stakes contexts: The effect of self-presentation on self-report instruments used in admission to higher education

Cited by 36 publications

References 42 publications

How do admission committees select? Do applicants know how they select? Selection criteria and transparency at a Dutch University

How do admission committees select? Do applicants know how they select? Selection criteria and transparency at a Dutch University

Faking on a situational judgment test in a medical school selection setting: Effect of different scoring methods?

Gender‐Based Differential Prediction by Curriculum Samples for College Admissions

Contact Info

Product

Resources

About