ObjectivesLifetime DSM‐5 diagnoses generated by the lay‐administered Composite International Diagnostic Interview for DSM‐5 (CIDI) in the World Mental Health Qatar (WMHQ) study were compared to diagnoses based on blinded clinician‐administered reappraisal interviews.MethodsTelephone follow‐up interviews used the non‐patient edition of the Structured Clinician Interview for DSM‐5 (SCID) oversampling respondents who screened positive for five diagnoses in the CIDI: major depressive episode, mania/hypomania, panic disorder, generalized anxiety disorder, and obsessive‐compulsive disorder. Concordance was also examined for a diagnoses of post‐traumatic stress disorder based on a short‐form versus full version of the PTSD Checklist for DSM‐5 (PCL‐5).ResultsInitial CIDI prevalence estimates differed significantly from the SCID for most diagnoses ( = 6.6–31.4, p = 0.010 < 0.001), but recalibration reduced most of these differences and led to consistent increases in individual‐level concordance (AU‐ROC) from 0.53–0.76 to 0.67–0.81. Recalibration of the short‐form PCL‐5 removed an initially significant difference in PTSD prevalence with the full PCL‐5 (from = 610.5, p < 0.001 to = 2.5, p = 0.110) while also increasing AU‐ROC from 0.76 to 0.81.ConclusionsRecalibration resulted in valid diagnoses of common mental disorders in the Qatar National Mental Health Survey, but with inflated prevalence estimates for some disorders that need to be considered when interpreting results.