The surficial cycling of Mg is directly coupled with the global carbon cycle, a predominant control of Earth’s climate. However, how Earth’s surficial Mg cycle evolved with time had been elusive. Magnesium isotope signatures of seawater (δ26Mgsw) track the surficial Mg cycle, which could provide crucial information on the carbon cycle in Earth’s history. Here, we present a reconstruction of δ26Mgsw evolution over the last 2 billion years using marine halite fluid inclusions and sedimentary dolostones. The two independent archives yield consistent evolutionary trends of δ26Mgsw for the past 430 million years, and the dolostone records extend the δ26Mgsw curve to 2 billion years ago. Modeling results of the net CO2 sequestration efficiency (EMg−CO2) by the surficial Mg cycle based on the δ26Mgsw record reveal a secular decline EMg−CO2 during the past 2 billion years, with the periods of low EMg−CO2 coinciding with ice ages in the Phanerozoic. Our work underlines a previously under-appreciated, but indispensable role of dolostones in regulating Earth’s climate on geologic time scales.