Rare earths, e.g. neodymium (Nd), praseodymium (Pr) and dysprosium (Dy), are abundant in the rare earth sintered magnet scrap (Nd-Fe-B scrap), but their recycling is tedious and costly due to the high content of impurity Fe. Herein, a novel approach was developed to effectively recycle rare earths from the scrap via an integrated acid dissolution and hematite precipitation method. The scrap contained 63.4% Fe, 21.6% Nd, 8.1% Pr and 3.9% Dy. It was dissolved in nitric, hydrochloric and sulfuric acids, separately. Nearly all impurity Fe in the scrap was converted to Fe3+ in nitric acid but was converted to Fe2+ in hydrochloric and sulfuric acids. After hydrothermal treatment, the rare earths in the three acids were almost unchanged. From nitric acid, 77.6% of total Fe was removed, but total Fe was not from the hydrochloric and sulfuric acids. By adding glucose, the removal of total Fe was further increased to 99.7% in nitric acid, and 97% of rare earths remained. The major mechanism underlying total Fe removal in nitric acid was the hydrolysis of Fe3+ into hematite, which was promoted by the consumption of nitrate during glucose oxidation. This method effectively recycled rare metals from the waste Nd-Fe-B scrap and showed great potential for industrial application.