The diversity of tropical herbivorous insects has been explained as a direct function of plant species diversity. Testing that explanation, we reared 2857 flies from flowers and seeds of 24 species of plants from 34 neotropical sites. Samples yielded 52 morphologically similar species of flies and documented highly conserved patterns of specificity to host taxa and host parts. Widespread species of plants can support 13 species of flies. Within single populations of plants, we typically found one or more fly species specific to female flowers and multiple specialists on male flowers. We suggest that neotropical herbivorous insect diversity is not simply a function of plant taxonomic and architectural diversity, but also reflects the geographic distribution of hosts and the age and area of the neotropics.