Face recognition is a remarkable human ability, which underlies a great deal of our social behaviour. We can recognize family members, friends and acquaintances over a very large range of conditions, and yet the processes by which we do this remain poorly understood, despite decades of research. Although a detailed understanding remains elusive, face recognition is widely thought to rely on Ôconfigural processingÕ, specifically an analysis of spatial relations between facial features (so-called second-order configurations). In this paper, we challenge this traditional view, raising four problems: (i) configural theories are underspecified; (ii) large configural changes leave recognition unharmed; (iii) recognition is harmed by non-configural changes; (iv) in separate analyses of face-shape and face-texture, identification tends to be dominated by texture. We review evidence from a variety of sources and suggest that failure to acknowledge the impact of familiarity on facial representations may have led to an overgeneralization of the configural account. We argue instead that second-order configural information is remarkably unimportant for familiar face recognition.