Antimicrobial peptides (AMPs) are short oligopeptides that can penetrate the bacterial inner and outer membranes. Together with cell-penetrating peptides (CPPs), they are called membrane active peptides; peptides which can translocate across biological membranes. Over the last fifty years, attempts have been made to understand the molecular features that drive the interactions of membranes with membrane active peptides. This review examines the features of a membrane these peptides exploit for translocation, as well as the physicochemical characteristics of membrane active peptides which are important for translocation. Moreover, it presents examples of how these features have been used in recent years to create conjugates consisting of a membrane active peptide, called a “vector”, attached to either a current or novel antibiotic, called a “cargo” or “payload”. In addition, the review discusses what properties may contribute to an ideal peptide vector able to deliver cargoes across the bacterial outer membrane as the rising issue of antimicrobial resistance demands new strategies to be employed to combat this global public health threat.