Adaptive control can significantly improve tracking performance of robot manipulators subject to modeling errors in dynamics. In this letter, we propose a new framework combining the composite adaptive controller using a natural adaptation law and an extension of the adaptive variance algorithm (AVA) for controller blending. The proposed approach not only automatically adjusts the feedback action to reduce the risk of violating actuator constraints but also anticipates substantial modeling errors by means of an uncertainty measure, thus preventing severe performance deterioration. A formal stability analysis of the closed-loop system is conducted. The control scheme is experimentally validated and directly compared with baseline methods on a torque-controlled KUKA LWR IV+.