Pressure towards population aging in the demographic pyramid is not only due to sociological/personal choices but also due to subfertility or infertility. There are several chemicals and mixtures that impair male fertility. While experimental animal models are crucial to identify compounds that affect male fertility, it is essential to use reliable in vitro models to determine cellular targets and intracellular pathways that mediate chemical toxicity in the male reproductive system. In this review, we focused on the somatic Sertoli cell (SC) that, within the testis, is a major target for hormonal signaling and provides physical and nutritional support to developing germ cells. The different outcomes possible in each type of study: in vivo versus in vitro (either in primary or immortalized cell cultures) are analyzed. Herein, we intend to clarify the unique features that render SCs as excellent candidates for a robust in vitro model to study the deleterious effects of chemicals on male reproductive health. The sensitivity of SCs to toxicants/pharmaceuticals is discussed and, based on the literature reviewed we propose the in vitro study of SC physiology as a model to disclose deleterious effects of substances to male fertility.