In the past decades, food consumption in China has undergone a rapid increase and a significant structure transition, as a result of population growth and economic development. The food system is increasingly threatening the environment by depleting water resources, deteriorating water bodies, aggravating climate change, degrading ecosystems, etc. It is significant to understand how food consumption affected the environment and how its impacts were driven in the historical period. This study reveals the environmental impacts of China’s food system from 1961 to 2017 from a consumption perspective by assessing water, carbon, and ecological footprints. The logarithmic mean Divisia index method was used to examine the drivers of the growing environmental footprints. The assessment results show that all three environmental footprints have had a drastic increase of more than two times during the studied period, which indicates the high environmental pressure posed by food consumption. We also found that, before the 1980s, the main driving forces of the increasing footprints were population and per capita energy intake. From 1984, the diet pattern started to take a positive effect and then became the dominant driver of the growing environmental footprints after the end of the 1990s.