Trace metal element concentrations in rice (Oryza sativa L.) have great effects on the health of human being. In this study, QTLs for the concentrations of Cd, Cu, Fe, Mn and Zn in brown rice were identified using a recombinant inbred lines (RIL) population. A total of 22 QTLs for the concentrations of Cd, Cu, Fe, Mn and Zn in brown rice were identified with 204 recombinant inbred lines (RILs), including qCd5 for pot cultivation (PC) with Cd stress and paddy field cultivation (FC) trials in Hangzhou, 9 for PC trial only, and 13 for FC trial only. The phenotypic variance explained by a single QTL ranged as 4.6-9.9% and 2.8-10.4% in PC and FC trials, respectively. Among all QTLs, qCu4-2 had the largest single effect (10.4% of the phenotypic variance), qCu7 had the lowest single effect (2.8% of the phenotypic variance). The positive alleles (increasing trace metal element concentrations) at 16 QTLs except qCd2, qCu7, qFe7, qMn7, qZn1-1 and qZn4 were inherited from the female parent Zhenshan 97 B (ZS97B). These QTLs distributed on chromosomes 1, 2, 4, 5, 7, 8, 10 and 11. Six QTLs were clustered in 1, 4 and 7 chromosomal regions. Especially, qCd5 was a novel QTL regulated low Cd accumulation in rice. The identification of QTLs provides good candidates for fine mapping and cloning, and could be utilized to improve rice safe and nutritional quality by MAS.