This paper introduces a novel method of robust parameter estimation of infinite impulse response (IIR) system. When training signal contains strong outliers, the conventional squared error-based cost function fails to provide desired performance. Thus a computationally efficient robust cost functions are used here. It is a fact that the IIR system falls in local minima. Thus the gradien-based algorithm which is good for finite impulse response (FIR) system, can not be used for IIR. Therefore the parameters of the IIR system is estimated using modified particle swarm optimisation algorithm. The most used and analysed robust cost functions such as Hubers and saturation nonlinearity function are used in the optimisation algorithm. The simulation results show that the proposed robust algorithms are providing better performance than the Wilcoxon norm-based robust algorithm and conventional error squared-based PSO algorithms.