The size of continents is an essential parameter to understand the growth of the continental crust and the evolution of the solid Earth because it is subject to tectonism and mantle convection and affects the preservation of the crust. This article reviews the secular change in the size of continents on the early Earth, focusing on terrigenous clastic rocks, especially quartzose sandstones occurring on relatively large continents. The earliest continental crust in the Hadean or early Archean was produced with a width of ∼200–500 km, similar to modern oceanic island arcs along subduction zones or oceanic islands in hot spot regions by mantle plume heating. Through the collision and amalgamation of such primitive continental crusts, continental blocks over 500 km in width and length evolved and appeared by ca. 3.5 Ga. Through further amalgamation, during ca. 3.3–2.5 Ga, the Archean continents emerged with widths and lengths greater than 1,000 km, which were still smaller than those of modern continents. Continents with widths and lengths of nearly 10,000 km have existed since ca. 2.4 Ga (early Proterozoic). Further analyses of the composition and formation mechanism of clastic rocks will help reveal more quantitative secular changes in the sizes of continents.