Conversion of formerly continuous native habitats into highly fragmented landscapes can lead to numerous negative demographic and genetic impacts on native taxa that ultimately reduce population viability. In response to concerns over biodiversity loss, numerous investigators have proposed that traits such as body size and ecological specialization influence the sensitivity of species to habitat fragmentation. In this study, we examined how differences in body size and ecological specialization of two rodents (eastern chipmunk; Tamias striatus and white‐footed mouse; Peromyscus leucopus) impact their genetic connectivity within the highly fragmented landscape of the Upper Wabash River Basin (UWB), Indiana, and evaluated whether landscape configuration and complexity influenced patterns of genetic structure similarly between these two species. The more specialized chipmunk exhibited dramatically more genetic structure across the UWB than white‐footed mice, with genetic differentiation being correlated with geographic distance, configuration of intervening habitats, and complexity of forested habitats within sampling sites. In contrast, the generalist white‐footed mouse resembled a panmictic population across the UWB, and no landscape factors were found to influence gene flow. Despite the extensive previous work in abundance and occupancy within the UWB, no landscape factor that influenced occupancy or abundance was correlated with genetic differentiation in either species. The difference in predictors of occupancy, abundance, and gene flow suggests that species‐specific responses to fragmentation are scale dependent.