Sarcopenia has become a key challenge for healthy aging in older adults. However, it remains unclear whether traditional Chinese medicine can effectively treat sarcopenia. This systematic review analyzes the current evidence for the effect of traditional Chinese medicine (TCM) on sarcopenia. We searched for articles regarding sarcopenia treated by TCM in Cochrane library, PubMed, SinoMed, Web of Science, Embase, and the China National Knowledge Infrastructure (from inception until 10 December 2021). Two researchers independently screened the literature in accordance with the inclusion and exclusion criteria designed by PICOS principles. The risk of bias was assessed by the Cochrane Risk of Bias (ROB) tool. The quality of evidence was assessed by the grading of recommendations, assessment, development, and evaluation (GRADE). Participants’ characteristics, interventions, and the relevant results of the included studies were extracted and synthesized in a narrative way. The total number of participants in the 21 included studies was 1,330. Most of the studies evaluated physical function (n = 20) and muscle strength (n = 18), and a small number of studies (n = 6) assessed muscle mass. Overall, it was found that TCM had a positive impact on muscle strength (grip strength, chair stand test) and physical function (6-m walking speed, timed up and go test, sit and reach) in patients with sarcopenia, inconsistent evidence of effects on muscle mass. However, the small sample size of the included studies led to imprecision in the results, and the presence of blinding of the studies, allocation concealment, and unreasonable problems with the control group design made the results low grade. Among these results, the quality of evidence for grip strength (n = 10) was of medium grade, and the quality of evidence related to the remaining indicators was of low grade. This systematic review showed that traditional Chinese Qigong exercises and Chinese herbal medicine have a positive and important effect on physical performance and muscle strength in older adults with sarcopenia. Future high-quality multicenter randomized controlled trials (RCTs) with large samples are needed to determinate whether acupuncture and other therapies are effective in treating sarcopenia.