Plant-based diets are associated with reduced risk of lifestyle-induced chronic diseases. The thousands of phytochemicals they contain are implicated in cellular-based mechanisms to promote antioxidant defense and reduce inflammation. While recommendations encourage the intake of fruits and vegetables, most people fall short of their target daily intake. Despite the need to increase plant-food consumption, there have been some concerns raised about whether they are beneficial because of the various ‘anti-nutrient’ compounds they contain. Some of these anti-nutrients that have been called into question included lectins, oxalates, goitrogens, phytoestrogens, phytates, and tannins. As a result, there may be select individuals with specific health conditions who elect to decrease their plant food intake despite potential benefits. The purpose of this narrative review is to examine the science of these ‘anti-nutrients’ and weigh the evidence of whether these compounds pose an actual health threat.