Indigenous populations of Malaysia known as Orang Asli (OA) show huge morphological, anthropological, and linguistic diversity. However, the genetic history of these populations remained obscure. We performed a high-density array genotyping using over 2 million single nucleotide polymorphisms in three major groups of Negrito, Senoi, and Proto-Malay. Structural analyses indicated that although all OA groups are genetically closest to East Asian (EA) populations, they are substantially distinct. We identified a genetic affinity between Andamanese and Malaysian Negritos which may suggest an ancient link between these two groups. We also showed that Senoi and Proto-Malay may be admixtures between Negrito and EA populations. Formal admixture tests provided evidence of gene flow between Austro-Asiatic-speaking OAs and populations from Southeast Asia (SEA) and South China which suggest a widespread presence of these people in SEA before Austronesian expansion. Elevated linkage disequilibrium (LD) and enriched homozygosity found in OAs reflect isolation and bottlenecks experienced. Estimates based on Ne and LD indicated that these populations diverged from East Asians during the late Pleistocene (14.5 to 8 KYA). The continuum in divergence time from Negritos to Senoi and Proto-Malay in combination with ancestral markers provides evidences of multiple waves of migration into SEA starting with the first Out-of-Africa dispersals followed by Early Train and subsequent Austronesian expansions.