Is there a prevalent pattern among interaction networks: nestedness or modularity? Must consumers always trade-off generalism for average performance in resource exploitation? These two questions have been addressed in various systems, with contradictory results. A recent integrative hypothesis combines both questions within a common theoretical framework, proposing that ecological specialization is structured by different prevailing processes in smaller and larger network units. This should produce both a compound interaction network, formed by internally nested modules, and a scale-dependence on the relationship between consumer performance and generalism. Here, we confirm both predictions in a large dataset on host-parasite interactions. We show that modules indeed constrain nestedness at the whole network level, and that the relationship between parasite generalism and performance on their hosts changed from negative at large to positive at small scales. Our results shed light on both debates, and provide some clues to their integration and solution.