Ninety percent of rice is produced in Asia, where it provides half of calories consumed, making it critical to food security. The 640 million rural agricultural workers of Asia are especially vulnerable to hot and humid weather, which impacts health and productivity; we show that heat hazard exposure increased significantly from 1980 to 2019. Around half of rice production occurs in locations and months where heat hazard is strongly correlated with global mean surface warming, based on climate simulations from the Coupled Model Intercomparison Project 6 (CMIP6), including >80% of production in Southeast Asia. Limiting global warming to 1.5 C rather than 2.0 C prevents a statistically significant reduction in labour capacity of 1% across all Asia and 2% across Southeast Asia, affecting the livelihoods of around 100 million people. Our results underscore regional disparities in labour productivity from climate change resulting in rising inequality and need for climate adaptation.