The Japanese Archipelago is widely covered with acidic soil made of volcanic ash, an environment which is detrimental to the preservation of ancient biomolecules. More than 10,000 Palaeolithic and Neolithic sites have been discovered nationwide, but few skeletal remains exist and preservation of DNA is poor. Despite these challenging circumstances, we succeeded in obtaining a complete mitogenome sequence from Palaeolithic human remains. We also obtained those of Neolithic (hunting-gathering Jomon and the farming Yayoi cultures) remains, and over 2,000 present-day Japanese. The Palaeolithic mitogenome is potentially an ancestral type of haplogroup M, suggesting it is not only connected to present-day Japanese but also present-day East Asians. There were no changes in the gene pool from the hunting-gathering (Jomon) to the farming cultures (Yayoi), and this is different from in Europe, where there was no genetic continuity between hunter-gatherers and farmers. We also found that a vast increase of population size happened and has continued since the Yayoi period, characterized with paddy rice farming. It means that the cultural transition, i.e. rice agriculture, had significant impact on the demographic history of Japanese population.