Visual object tracking, one of the most fundamental and challenging tasks with several applications, has attracted a great deal of research interest and has been widely applied in unmanned aerial vehicle (UAV) missions. Despite the efficient correlation filter (CF)-based approaches that have shown prominent performance in visual object tracking for different UAV applications, the model drift, caused by undesired boundary effects, filter degradation, and severe appearance variations, remains a challenging problem. To solve the aforementioned issue, we propose saliencyenhanced background-aware CFs with dual temporal regularization for UAV tracking. In more detail, the dual temporal regularization term is introduced to reinforce interframe continuity using current and previous frames' filter and sample information, respectively. As for the saliency enhancement term, it is integrated by multiplying the merged saliency map and the vectorized image patch element-wise to emphasize the noticeable object and suppress irrelevant background noise. Finally, a CF regression model is formulated and its closed-form solution is derived by utilizing the alternating direction method of multipliers (ADMM) algorithm. The saliency and temporal information in the tracking scene are fully considered to promote model discriminative ability and boost the overall tracking performance. Extensive experiments are conducted on three public UAV benchmark datasets UAV123@10 fps, DTB70, and UAVDT using 243 challenging sequences. The results demonstrate that the proposed method performs favorably against other state-of-the-art trackers while operating at a real-time speed on a single CPU, making it suitable for UAV tracking tasks.