The specificity of a T-cell receptor (TCR) repertoire determines personalized immune capacity. Existing methods have modelled the qualitative aspects of TCR specificity, while the quantitative aspects remained unaddressed. We developed a package, TCRanno, to quantify the specificity of TCR repertoires. Applying TCRanno to 4,195 TCR repertoires revealed quantitative changes in repertoire specificity upon infections, autoimmunity and cancers. Specifically, TCRanno found cytomegalovirus-specific TCRs in seronegative healthy individuals, indicating their past abortive infections instead of non-exposure to the virus. TCRanno discovered age-accumulated SARS-CoV2 specific TCRs in pre-pandemic samples, which may explain the aggressive symptoms and age-related severity of COVID-19. TCRanno also identified the encounter of Hepatitis B antigens as a trigger of systemic lupus erythematosus. TCRanno annotations showed capability in distinguishing TCR repertoires of healthy and cancers including melanoma, lung and breast cancers. TCRanno also demonstrated usefulness to single-cell TCRseq+gene expression data analyses by isolating T-cells with the specificity of interest.