Despite several decades of research, the epigenesis of behavioural and brain lateralization is still elusive, although its knowledge is important in understanding developmental plasticity, function and evolution of lateralization, and its relationship with developmental disorders. Over the last decades, it has become clear that behavioural lateralization is not restricted to humans, but a fundamental principle in the organization of behaviour in vertebrates. This has opened the possibility of extending descriptive studies on human lateralization with descriptive and experimental studies on other vertebrate species. In this review, we therefore explore the evidence for the role of genes and environment on behavioural lateralization in humans and other animals. First, we discuss the predominant genetic models for human handedness, and conclude that their explanatory power alone is not sufficient, leaving, together with ambiguous results from adoption studies and selection experiments in animals, ample opportunity for a role of environmental factors. Next, we discuss the potential influence of such factors, including perinatal asymmetrical perception induced by asymmetrical head position or parental care, and social modulation, both in humans and other vertebrates, presenting some evidence from our own work on the domestic chick. We conclude that both perinatal asymmetrical perception and later social modulation are likely candidates in influencing the degree or strength of lateralization in both humans and other vertebrates. However, in most cases unequivocal evidence for this is lacking and we will point out further avenues for research.