Understanding which environmental variables and traits underlie adaptation to harsh environments is difficult because many traits evolve simultaneously as populations or species diverge. Here, we investigate the ecological variables and traits that underlie Mimulus laciniatus' adaptation to granite outcrops compared to its sympatric, mesic-adapted progenitor, Mimulus guttatus. We use fine-scale measurements of soil moisture and herbivory to examine differences in selective forces between the species' habitats, and measure selection on flowering time, flower size, plant height, and leaf shape in a reciprocal transplant using M. laciniatus × M. guttatus F hybrids. We find that differences in drought and herbivory drive survival differences between habitats, that M. laciniatus and M. guttatus are each better adapted to their native habitat, and differential habitat selection on flowering time, plant stature, and leaf shape. Although early flowering time, small stature, and lobed leaf shape underlie plant fitness in M. laciniatus' seasonally dry environment, increased plant size is advantageous in a competitive mesic environment replete with herbivores like M. guttatus'. Given that we observed divergent selection between habitats in the direction of species differences, we conclude that adaptation to different microhabitats is an important component of reproductive isolation in this sympatric species pair.