In the recent literature, methods from extreme value theory (EVT) have frequently been applied for the estimation of tail risk measures. While previous analyses show that EVT methods often lead to accurate estimates for risk measures, a potential drawback lies in high standard errors of point estimates of these methods as only a fraction of the data set is used. Thus, the aim of this paper is to comprehensively study the impact of model risk on EVT methods when determining the Value-at-Risk and Expected Shortfall. We distinguish between first order effects of model risk, which consist of misspecification and estimation risk, and second order effects of model risk which refer to the dispersion of risk measure estimates. We show that EVT methods are less prone to first order effects of model risk, however, they exhibit a higher sensitivity towards second order effects of model risk. We find that this can lead to severe Value-at-Risk and Expected Shortfall underestimations and should be reflected in regulatory capital models.