The physical properties of space may be universal, but the way people conceptualize space is variable. In some groups, people tend to use egocentric space (e.g. left, right) to encode the locations of objects, while in other groups, people encode the same spatial scene using allocentric space (e.g. upriver, downriver). These different spatial frames of reference (FoRs) characterize the way people talk about spatial relations and the way they think about them, even when they are not using language. Although spatial language and spatial reasoning tend to covary, the root causes of this variation are unclear. Here we propose that variation in FoR use partly reflects the discriminability of the relevant spatial continua. In an initial test of this proposal in a group of indigenous Bolivians, we compared FoR use across spatial axes that are known to differ in discriminability. In both verbal and nonverbal tests, participants spontaneously used different FoRs on different spatial axes: On the lateral axis, where egocentric (left-right) discrimination is difficult, their spatial behavior and language was predominantly allocentric; on the sagittal axis, where egocentric (front-back) discrimination is relatively easy, they were predominantly egocentric. These findings challenge the claim that each language group can be characterized by a predominant spatial frame of reference. Rather, both spatial memory and language can differ categorically across axes, even within the same individuals. We suggest that differences in spatial discrimination can explain differences in both spatial memory and language within and across human groups.