Augmenting adaptive immunity is a critical goal for developing next-generation cancer therapies. T and B cells infiltrating the tumor dramatically influence cancer progression through complex interactions with the local microenvironment. Cancer cells evade and limit these immune responses by hijacking normal immunologic pathways. Current experimental models using conventional primary cells, cell lines, or animals have limitations for studying cancer-immune interactions directly relevant to human biology and clinical translation. Therefore, engineering methods to emulate such interplay at local and systemic levels are crucial to expedite the development of better therapies and diagnostic tools. In this review, we discuss the challenges, recent advances, and future directions toward engineering the tumor-immune microenvironment (TME), including key elements of adaptive immunity. We first offer an overview of the recent research that has advanced our understanding of the role of the adaptive immune system in the tumor microenvironment. Next, we discuss recent developments in 3D in-vitro models and engineering approaches that have been used to study the interaction of cancer and stromal cells with B and T lymphocytes. We summarize recent advancement in 3D bioengineering and discuss the need for 3D tumor models that better incorporate elements of the complex interplay of adaptive immunity and the tumor microenvironment. Finally, we provide a perspective on current challenges and future directions for modeling cancer-immune interactions aimed at identifying new biological targets for diagnostics and therapeutics.