We examine how first language (L1) Spanish listeners with varying levels of experience with English recalibrate their phonemic category boundaries following exposure to second language (L2), American English-accented Spanish. Specifically, we examine changes to voice onset time (VOT) boundaries, which are often positively-shifted when produced by American English-accented Spanish speakers (as compared to L1 Spanish speakers). Our results demonstrate that listeners make adjustments to their phonemic category boundaries following exposure to accented words with the critical sounds in onset position (e.g., “bailar” and “parir”, meaning “to dance” and “to give birth”, for the /b/ and /p/ phonemic categories). In many cases generalization of phonemic learning was also observed, such that boundaries for categories that were not presented in training were also adjusted. Surprisingly, however, there were cases in which boundaries for trained categories did not show adjustments; for example, listeners trained with items for all places of articulation (POAs) showed recalibration of their bilabial and alveolar boundaries but not their velar boundary. Also notable was the role of the Spanish listeners' experience with English: More experienced listeners showed more positively-shifted (English-like) boundaries in the pre-test session. This suggests that more experienced listeners may have rapidly identified the American English-accented Spanish and applied their English category boundaries accordingly. We conclude that listener accommodation of L2 accent is supported by a phonemic recalibration mechanism, and that experience with the L1 of an L2-accented speaker facilitates rapid recalibration of phonemic categories.