This paper proposes an adaptive fading Bayesian unscented Kalman filter (AF-BUKF) and explores its application for state estimation of unmanned aircraft systems (UASs). In the AF-BUKF, the state and noise densities are approximated as finite Gaussian mixtures, in which the mean and covariance for each component are recursively estimated using the UKF. To avoid the prohibitive computational complexity caused by the exponential growth of mixture components, a Gaussian mixture simplification algorithm is employed. Moreover, the AF-BUKF algorithm employs a novel adaptive fading strategy to recursively update the Gaussian components, so that the adverse effect of inexact knowledge of the state and measurement noise covariance can be mitigated. An AF-BUK Smoother (AF-BUKS) is also proposed by extending the AF-BUKF algorithm using the concept of optimal Bayesian smoothing and the Rauch-Tung-Striebel Smoother to improve estimation accuracy. Experimental results on simulated and real UAS data show that the proposed AF-BUKF/S algorithms can achieve better performance compared with the conventional methods. Thus, they can serve as attractive alternative approaches for nonlinear state estimation of UASs and other problems. INDEX TERMS Bayesian smoothing, nonlinear and non-Gaussian system, Gaussian mixture, unmanned aircraft systems, unscented Kalman filter