Phytoremediation is becoming more prevalent globally. Literature on phytoremediation strategies in western China is relatively scarce. The present research sought to fill this gap by examining the effects of trace elements such as Cd, Cr, Pb, and Zn on growth, physiological traits, tolerances, and accumulation characteristics in 2-year-old saplings of Ligustrum obtusifolium. The gradient of trace element concentration was determined by adding exogenous trace elements to the soil in a pot experiment: CK (no exogenous trace element), T1, T2, and T3 (Cd 0, 2, 5, and 10 mg/kg; Cr 0, 300, 500, and 700 mg/kg; Pb 0, 400, 800, and 1200 mg/kg; and Zn 0, 300, 500, and 1000 mg/kg, respectively). The results indicated that Chla, Chlb, and total Chls significantly decreased (p < 0.05) in the leaves of L. obtusifolium, with the lowest value obtained during soil treatment with T3. Along with the increase in trace element concentration, the net increase in height, root biomass, aerial biomass, and total biomass was reduced significantly. The net growth of L. obtusifolium under Cr stress did not differ significantly from that of CK at T1; however, the net growth of L. obtusifolium under Cr stress was considerably reduced at T2 and T3. The antioxidant enzyme activity of L. obtusifolium increased under different trace element stresses and first increased and then decreased as trace element levels increased. It was found that the SOD, POD, CAT, and APX activity of L. obtusifolium peaks at T2 under Cd and Zn stress in contrast to a peak at T1 under Cr and Pb stress. The contents of trace elements in L. obtusifolium roots, stems, and leaves increased along with the increase in soil trace element levels. Cr, Pb, and Zn threshold values can be set at 300 mg/kg, 400 mg/kg, and 300 mg/kg, respectively. This is carried out by using a reference index of the biomass of L. obtusifolium decreased by 10%, while the Cd threshold value needs to be further studied. L. obtusifolium would be an appropriate plant for phytoremediation of Cr-polluted soil, compared to Cd, Pb, and Zn absorption and accumulation. It showed optimal antioxidant enzyme activity and transfer ability under soil Cr contents of 300 mg/kg, and the growth of L. obtusifolium was not restricted. Therefore, L. obtusifolium was particularly suitable for phytoremediation of Cr pollution in areas in western China.