We are grateful to the authors who provided their insightful commentaries [1][2][3][4][5], which we hope will lead to more appropriate uses of the NRI and IDI metrics and their parent measures, the maximum relative utility, and discrimination slope. Here, we highlight common themes, clarify certain issue, and point out where we differ with some of the authors.Together with the papers they reference [6][7][8], several of the authors re-iterate the importance of model calibration when using the NRI and IDI metrics. Emphasis is appropriately placed on using smooth calibration plots to assess calibration. As suggested by van Calster et al.[9] and further illustrated by the theoretical examples in [6,7], 'moderate' (or 'level 3') calibration should be required: For persons with the same predicted risk, the observed event rate equals the predicted risk. As pointed out in [1], there is room for further discussion on recalibration and how and when it should be performed. When evaluating added value of new markers, it is reasonable to assess it on a model that is calibrated in the moderate sense: if a decision is made to use the new markers, they would be incorporated into a model for which satisfactory calibration needs to be ascertained before it is recommended for general applications. At the same time, we recognize the decision-making conundrum where in some applications to new data, we may not know ahead of time the extent of potential miscalibration.A key conclusion from the work of Chipman and Braun [6] is that calibration must be sufficiently met for the population of inference. If an investigator wishes to report the IDI in a clinically relevant subgroup, sufficient calibration must be established for that subgroup. If this is not the case, Simpson's Paradox may be observed in comparing the subgroup specific IDI to the IDI for the whole sample. Even small departures from calibration may lead to Simpson's Paradox; this further emphasizes the sensitivity of the discrimination slope to model calibration [2]. Of note, these issues are not resolved by using the weighted discrimination slope or IDI. While the weighted IDI removed the issue of Simpson's Paradox in simulations, under some settings, it yielded conclusions that were inconsistent with other measures including change in AUC, Brier Score, and other R-squared metrics [6]. Therefore, while illustrative, we do not recommend using it as a new metric.An important feature of the R-squared metrics is that, unlike the AUC, they integrate both calibration and discrimination into one number. Because the AUC is equivalent to the rank-based Mann-Whitney statistic, it necessarily loses information that relates to the calibration of the risk estimates. Thus, the AUC is a more pure measure of discrimination. The discrimination slope, on the other hand, retains all the information. Yet because the slope is not a proper scoring rule, it is susceptible to misleading conclusions for poorly calibrated models [6,7].