Aim The aim was to explore the psychometric properties with respect to the internal consistency reliability of the subject‐specific questionnaire “Failing to Fail.” Design Cross‐sectional study. Methods Exploratory factor analysis with varimax rotation. A confirmatory factor analysis was used to examine the factor structure of the “Failing to Fail” scale. The sample included 336 Norwegian nurse mentors. Results The confirmatory factor analysis confirmed a five‐factor structure of the “Failing to Fail” scale with adequate model fit. The factors were named as: (a) Insufficient mentoring competence; (b) Insufficient support in the working environment; (c) Emotional process dominates the assessment; (d) Insufficient support from the university; and (e) Decision‐making detached from learning outcomes. The scale proved to be feasible to test whether mentors are Failing to Fail nursing students. The confirmatory factor analysis model supported the predictive validity of the “Failing to Fail” scale.