Indigenous peoples, who depend on their environment for their livelihoods and are often subject to poverty and socio-economic marginalisation, are some of the most vulnerable to climate change. While the rights of Indigenous peoples are recognised at international level, these are often not translated into adaptation responses at national and local levels. Using insights from theories of environmental and social justice in the case study analysis of Batwa community in Uganda, we assess how justice-related factors impact the adaptive capacities of Indigenous peoples and discuss how these can be taken into account when designing and implementing adaptation responses. The findings from our fieldwork reveal a multidimensional range of systemic injustices experienced by the Batwa, resulting from their continued social-economic, cultural and political marginalisation after their eviction from Uganda's forests. We also observe that there is a variety of projects happening locally in relation to 'adaptation' but not labelled as such, suggesting how Batwa's vulnerability to climate change is rooted in wider aspects of livelihoods and development. More importantly, we find that most projects tend to focus on distribution of material benefits, while less attention is paid to the more intricate issues of compensation, political discrimination and uneven participation. This depoliticised and compartmentalised approach suggests a slow and incomplete way of operationalising justice in climate adaptation. Hence, we call for sincere efforts to address recognition, rights, and disproportionate levels of disadvantage for Indigenous communities, including their constitutional recognition, financial redress and more opportunities for participation in decision-making.