T-cell receptors (TCRs) have emerged as a new class of therapeutics, most prominently for cancer where they are the key components of new cellular therapies as well as soluble biologics. Many studies have generated high affinity TCRs in order to enhance sensitivity. Recent outcomes, however, have suggested that fine manipulation of TCR binding, with an emphasis on specificity may be more valuable than large affinity increments. Structure-guided design is ideally suited for this role, and here we studied the generality of structure-guided design as applied to TCRs. We found that a previous approach, which successfully optimized the binding of a therapeutic TCR, had poor accuracy when applied to a broader set of TCR interfaces. We thus sought to develop a more general purpose TCR design framework. After assembling a large dataset of experimental data spanning multiple interfaces, we trained a new scoring function that accounted for unique features of each interface. Together with other improvements, such as explicit inclusion of molecular flexibility, this permitted the design new affinity-enhancing mutations in multiple TCRs, including those not used in training. Our approach also captured the impacts of mutations and substitutions in the peptide/MHC ligand, and recapitulated recent findings regarding TCR specificity, indicating utility in more general mutational scanning of TCR-pMHC interfaces.