Bispecific antibodies (BsAb) can induce long-term responses in refractory and relapsed B cell lymphoma patients. Nevertheless, response rates across patients are heterogenous and the factors determining quality and duration of responses are poorly understood. In order to identify key determinants of response to BsAb, we established a primary, autologous culture model allowing us to mimic treatment with CD3xCD19 and CD3xCD20 BsAb within the lymph node microenvironment ex vivo. T cell-mediated killing of lymphoma cells and proliferation of T cells varied significantly among patients but highly correlated between BsAb targeting CD20 or CD19. Ex vivo response to BsAb was significantly associated with expansion of T cells and secretion of effector molecules, such as granzyme B and perforin, but not with expression of T cell exhaustion (e.g. PD1, TIM3) or activation markers (e.g. CD25, CD69) or formation of intercellular contacts. In addition, we identified a distinct phenotype of regulatory T cells that was linked to ex vivo response independently from T cell frequency at baseline. High expression levels of Aiolos (IKZF1), ICOS and CXCR5 were positively associated with ex vivo response, whereas strong expression of Helios (IKZF2) had unfavorable impact on ex vivo response to BsAb. Furthermore, we demonstrated that lenalidomide, nivolumab and atezolizumab improved ex vivo response to BsAb by potentiating T cell effector functions. In summary, our ex vivo study identifies a distinct regulatory T cell phenotype as potential contributor to treatment failure of BsAb, and suggests drug combinations of high clinical relevance that could improve the efficacy of BsAb.