The McCarthy Screening Test, used to identify children with learning difficulties, yields a pass‐fail decision for each subtest. The reliability of these decisions is studied using several criterion‐referenced testing methods (N = 1,323). The several results are evaluated. It is suggested that dependability indices be adopted for clinical instruments such as the MST.