The importance of protein tyrosine phosphatases (PTPs) in the regulation of cellular signaling is well established. Malfunction of PTP activity is also known to be associated with cancer, metabolic syndromes, autoimmune disorders, neurodegenerative and infectious diseases. However, a detailed understanding of the roles played by the PTPs in normal physiology and in pathogenic conditions has been hampered by the absence of PTP-specific small molecule agents. In addition, the therapeutic benefits of modulating this target class are underexplored due to lack of suitable chemical probes. Potent and specific PTP inhibitors could significantly facilitate functional analysis of the PTPs in complex cellular signal transduction pathways and may constitute valuable therapeutics in the treatment of several human diseases. We will highlight the current challenges and opportunities in developing PTP-specific small molecule agents. We will also review available selective small molecule inhibitors developed for a number of PTPs, including PTP1B, TC-PTP, SHP2, Lyp, HePTP, CD45, PTPβ, PTPγ, PTPRO, VHR, MKP-1, MKP-3, Cdc25, YopH, mPTPA, and mPTPB.