During the past 20 years, the development of HIV vaccines has come a long way. The focus has progressively changed from the traditional protein-based HIV vaccines that induce humoral immunity to the live recombinant viral vector-based HIV vaccines capable of eliciting both cellular and humoral immune responses. These new viral vector-based vaccines encoding multiple HIV antigens, delivered either alone or in heterologous prime-boost modalities elicited antigen-specific CTL responses in immunized hosts and protected animals from disease. The viral vector-based vaccines have proven to be potent vaccines in pre-clinical studies and foster the hope to put an end to the ever-increasing threat of the AIDS epidemic. Several unique features of viral vector-based HIV vaccines have contributed to their success, including their intrinsic immune-modulating properties, high transduction efficiency, and in vivo production of immunogens within the cell mimicking a natural infection without the associated health risks. In this review, we will discuss the characteristics of non-replicating viral vectors most commonly used for HIV vaccines with a particular focus on immune responses elicited by the vector particles alone and their effect on the potency of viral vector-based HIV vaccines.