Granulomas are clusters of immune cells. These structures can be formed in reaction to infection and display signs of necrosis, such as in tuberculosis. Alternatively, in several immune disorders, such as sarcoidosis, Crohn's disease and common variable immunodeficiency, non-caseating granulomas are formed without an obvious infectious trigger. Despite advances in our understanding of the human immune system, the pathogenesis underlying these non-caseating granulomas in chronic inflammatory diseases is still poorly understood. Here, we review the current knowledge about the immunopathogenesis of granulomas, and we discuss how the involved immune cells can be targeted with novel therapeutics.