The use of silica fume can significantly enhance mechanical properties of concrete given its beneficial filling and pozzolanic effects. In this study, a simple and effective double-side pullout testing method was adopted to characterize the interfacial bond properties, which include pullout load-slip relationship, bond strength, and pullout energy, of steel fiber-matrix in ultra-high strength cement-based material (UHSC) with 0-25% silica fume by the mass of binder. The effects of silica fume content on flowability, heat of hydration, compressive and flexural strengths, hydration products, and pore structure of matrix at different curing time were evaluated as well. Backscatter scanning electron microscopy (BSEM) and micro-hardness measurement were used to examine the quality of interfacial transition zone (ITZ) around the fiber. In terms of the results, the optimal silica fume content could be in the range of 15%-25%. UHSC mixtures with these dosages of silica fume showed significant improvement in pullout behavior. Its bond strength and pullout energy at 28 d could increase by 170% and 250% compared to the reference samples without any silica fume. The microstructural observation verified the findings on the macro-properties development. Formation of more and higher strength of hydration products and refinement of ITZ around the fiber ensured higher micro-hardness, and thus increased the bond to fiber.