The soft magnetic properties of Fe-based nanocrystalline alloys are determined by their grain size. In the present article, the (Fe0.4Co0.6)79Nb3B18 nanocrystalline alloys have been successfully prepared by isothermal annealing. The variation of soft magnetic properties as a function of annealing temperature and incubation time is investigated in detail. Two distinct crystallization behaviors were found for the (Fe0.4Co0.6)79Nb3B18 alloys. The initial nanocrystallization products comprise a mixture of α-Fe(Co), Fe2B, and Fe23B6-type crystalline metastable phases, and the final crystallization products are composed of α-Fe(Co), Fe2B, and Fe3B crystalline phases. The grain size decreases first and then increases with the increasing annealing temperature in the range of 764–1151 K, and a fine grain size with mean grain size of 12.7 nm can be achieved for alloys annealed at 880 K. As the annealing temperature increases from 764 K to 1151 K, the saturation magnetization increases first and then decreases without a significant increase of the coercivity. The alloys annealed at 880 K exhibit the optimized soft magnetic properties with high Ms of 145 emu g−1 and low Hc of 0.04 Oe.