BackgroundWhile many clinical prediction models (CPMs) exist to guide valvular heart disease treatment decisions, the relative performance of these CPMs is largely unknown. We systematically describe the CPMs available for patients with valvular heart disease with specific attention to performance in external validations.Methods and ResultsA systematic review identified 49 CPMs for patients with valvular heart disease treated with surgery (n=34), percutaneous interventions (n=12), or no intervention (n=3). There were 204 external validations of these CPMs. Only 35 (71%) CPMs have been externally validated. Sixty‐five percent (n=133) of the external validations were performed on distantly related populations. There was substantial heterogeneity in model performance and a median percentage change in discrimination of −27.1% (interquartile range, −49.4%–−5.7%). Nearly two‐thirds of validations (n=129) demonstrate at least a 10% relative decline in discrimination. Discriminatory performance of EuroSCORE II and Society of Thoracic Surgeons (2009) models (accounting for 73% of external validations) varied widely: EuroSCORE II validation c‐statistic range 0.50 to 0.95; Society of Thoracic Surgeons (2009) Models validation c‐statistic range 0.50 to 0.86. These models performed well when tested on related populations (median related validation c‐statistics: EuroSCORE II, 0.82 [0.76, 0.85]; Society of Thoracic Surgeons [2009], 0.72 [0.67, 0.79]). There remain few (n=9) external validations of transcatheter aortic valve replacement CPMs.ConclusionsMany CPMs for patients with valvular heart disease have never been externally validated and isolated external validations appear insufficient to assess the trustworthiness of predictions. For surgical valve interventions, there are existing predictive models that perform reasonably well on related populations. For transcatheter aortic valve replacement (CPMs additional external validations are needed to broadly understand the trustworthiness of predictions.

No abstract

Background and objectiveUrology residency match occurs through the American Urological Association (AUA), and hence information about the success of applicants in finding a match is not readily available. The average number of publications a successful urology applicant has when applying for residency is unknown. In light of this, we conducted this study to examine the number of PubMed-indexed research projects involving US senior medical students who successfully matched into the top 50 urology residency programs in the 2021, 2022, and 2023 match cycles. We also assessed these applicants based on their medical schools and gender. MethodsDoximity Residency Navigator was used to generate the top 50 residency programs as sorted by reputation. Newly matched residents were found using program Twitter accounts and residency program websites. PubMed was queried for peer-reviewed publications of incoming interns. ResultsThe average number of publications across all incoming interns in the three years was 3.65. The average number of urology-specific publications was 1.86 and that of first-author urology publications was 1.11. The median number of total publications for matched applicants was 2, and applicants with a total of five publications were in the 75th percentile for research productivity. ConclusionA successful applicant had two PubMed-indexed urology papers on average and also had a urology-specific first-author paper in the cycles we surveyed. There has been an increase in publications per applicant when comparing the results to previous application cycles, which can be attributed to post-pandemic changes.

Background: Clinical prediction models (CPMs) hold the potential to improve decision-making and individualize care for patients with cardiovascular disease (CVD); however, it is increasingly recognized that CPM performance may decrease when validated externally. Here we systematically describe published external validations of CPMs for patients with CVD, with specific attention to changes in CPM performance. Methods: A systematic review was conducted using a citation search to identify external validations of CPMs in the Tufts Predictive Analytics and Comparative Effectiveness (PACE) CPM Registry, a comprehensive database of CPMs for patients with CVD published between 1990 and May 2015. Information on CPM performance in the original derivation database as well as during subsequent external validations (through March 2017) was extracted. The percent change in discrimination was calculated as [(Validation AUC - 0.5) - (Derivation AUC - 0.5) / (Derivation AUC - 0.5) * 100]. Results: The Registry includes 1083 CPMs for CVD. 1555 external validations were identified. On average there were 1.4 validations/ de novo CPM (range 0-83). 758 (70%) of the CPMs have never been externally validated. The median external validation sample size was 1215 (IQR 430, 6509) and the median number of events 59 (21, 209). 88% (1368) of the external validations report area under the receiver operating characteristic curve (AUROC). 59% (917) report some measure of CPM calibration. The median external validation AUROC was 0.74 (0.69-0.81) and the median percent change in discrimination was -14.7% (-34.1%, +0.8%). 29.3% (455) of model validations showed CPM discrimination at or above the performance reported in the derivation dataset. Conclusion: While numerous CPMs exist for patients with CVD, most have never been externally validated. CPMs generally show substantially worse discrimination in external validations compared to the derivation datasets.