The Src homology 2 (SH2) domain-containing phosphatase 2 (SHP-2) is a nonreceptor protein tyrosine phosphatase (PTP) involved in extracellular-regulated kinase (ERK) activation. Recent studies have shown that gain-of-function mutations in SHP-2 are associated with several diseases, including LEOPARD syndrome, Noonan syndrome, and juvenile myelomonocytic leukemia. In this study, we identified the novel SHP-2 inhibitor 3-(1-benzimidazolylmethyl)-6-p-tolyl-[1,2,4]triazolo[3,4-b][1,3,4]thiadiazole (MLS-001). SHP-2 activity was inhibited by MLS-001, whereas other types of PTPs, namely ACP1, CDC25A, DUSP3, DUSP14, DUSP18, DUSP22, DUSP23, DUSP26, and SSH3, were not. Furthermore, TCPTP and SHP-1 that are closely related to SHP-2 were not inhibited by the inhibitor. Kinetic studies with MLS-001 and SHP-2 revealed a competitive inhibition. The SHP-2 expressing cells treated with MLS-001 demonstrated reduced SHP-2 phosphatase activity, thereby suggesting that MLS-001 effectively passes through cell membranes. In addition, MLS-001 reduced SHP-2-mediated phosphorylation in the activation loop of ERK in cells. Therefore, MLS-001 could be a lead compound for developing a potent SHP-2 inhibitor.