The current study examined the phonological and semantic contributions to the verbal short-term memory (VSTM) deficit in Down syndrome (DS) by experimentally manipulating the phonological and semantic demands of VSTM tasks. The performance of 18 individuals with DS (ages 11–25) and 18 typically developing children (ages 3–10) matched pairwise on receptive vocabulary and gender was compared on four VSTM tasks, two tapping phonological VSTM (phonological similarity, nonword discrimination) and two tapping semantic VSTM (semantic category, semantic proactive interference). Group by condition interactions were found on the two phonological VSTM tasks (suggesting less sensitivity to the phonological qualities of words in DS), but not on the two semantic VSTM tasks. These findings suggest that a phonological weakness contributes to the VSTM deficit in DS. These results are discussed in relation to the DS neuropsychological and neuroanatomical phenotype.