Administration of azole antifungals to tacrolimus-treated solid organ recipients results in a major drug-drug interaction characterized by increased exposure to tacrolimus. The magnitude of this interaction is highly variable but cannot currently be predicted. We performed a retrospective analysis of 126 solid organ recipients (95 lung, 31 kidney) co-treated with tacrolimus and voriconazole (n = 100) or posaconazole (n = 26). Predictors of the change in tacrolimus dose-corrected trough concentrations (C/D) between baseline and tacrolimus-azole co-therapy were assessed using linear mixed modeling. Patients were genotyped for relevant polymorphisms in CYP3A4, CYP3A5, MDR1, CYP2C19, POR, and UGT1A4. Tacrolimus C/D increased by a factor 5.0 ± 2.7 (range 1.0-20.2) for voriconazole and 4.4 ± 2.6 (range 0.9-18.0) for posaconazole, suggesting that a 66% dose reduction is insufficient for the majority of patients. Change in C/D was blunted in CYP3A5 expressors (estimated effect: -43%, p = 0.017) and affected by hematocrit (+8% per %, p = 0.004), baseline C/D (-14% per 100% increase, p < 0.001), and age (+1%, p = 0.008). However, the final model explained only 22% of interindividual variability in C/D change. In conclusion, CYP3A5 genotype and several clinical variables were identified as modulators of the tacrolimus-azole interaction, but these did not permit accurate predictions in individual patients.