There is remarkable diversity in brain size among vertebrates, but surprisingly little is known about how ecological species interactions impact the evolution of brain size. Using guppies, artificially selected for large and small brains, we determined how brain size affects survival under predation threat in a naturalistic environment. We cohoused mixed groups of small- and large-brained individuals in six semi-natural streams with their natural predator, the pike cichlid, and monitored survival in weekly censuses over 5 months. We found that large-brained females had 13.5% higher survival compared to small-brained females, whereas the brain size had no discernible effect on male survival. We suggest that large-brained females have a cognitive advantage that allows them to better evade predation, whereas large-brained males are more colourful, which may counteract any potential benefits of brain size. Our study provides the first experimental evidence that trophic interactions can affect the evolution of brain size.

Lay SummaryDoes a large brain make you smarter? If you are a guppy male searching for a female in a maze, it does. The association between brain size and smartness is a debated issue, largely due to the lack of experimental data. We compared guppies artificially bred for large and small brains and found that large-brained males learned the route through a spatial maze faster. These results thus support a link between brain size and smartness.

Variation in brain size and cognitive ability affects mate quality assessment and underlies variation in mate choice.

Understanding the evolution of mate choice requires dissecting the mechanisms of female preference, particularly how these differ among social contexts and preference phenotypes. Here we study the female neurogenomic response after only 10 minutes of mate exposure in both a sensory component (optic tectum) and a decision-making component (telencephalon) of the brain. By comparing the transcriptional response between females with and without preferences for colorful males, we identified unique neurogenomic elements associated with the female preference phenotype that are not present in females without preference. Network analysis revealed different properties for this response at the sensory-processing and the decision-making levels, and showed that this response is highly centralized in the telencephalon. Furthermore, we identified an additional set of genes that vary in expression across social contexts, beyond mate evaluation. We show that transcription factors among those loci are predicted to regulate the transcriptional response of the genes we found to be associated with female preference.

One key hypothesis in the study of brain size evolution is the expensive tissue hypothesis; the idea that increased investment into the brain should be compensated by decreased investment into other costly organs, for instance the gut. Although the hypothesis is supported by both comparative and experimental evidence, little is known about the potential changes in energetic requirements or digestive traits following such evolutionary shifts in brain and gut size. Organisms may meet the greater metabolic requirements of larger brains despite smaller guts via increased food intake or better digestion. But increased investment in the brain may also hamper somatic growth. To test these hypotheses we here used guppy (Poecilia reticulata) brain size selection lines with a pronounced negative association between brain and gut size and investigated feeding propensity, digestive efficiency (DE), and juvenile growth rate. We did not find any difference in feeding propensity or DE between large‐ and small‐brained individuals. Instead, we found that large‐brained females had slower growth during the first 10 weeks after birth. Our study provides experimental support that investment into larger brains at the expense of gut tissue carries costs that are not necessarily compensated by a more efficient digestive system.

The visual system is highly variable across species, and such variability is a key factor influencing animal behavior. Variation in the visual system, for instance, can influence the outcome of learning tasks when visual stimuli are used. We illustrate this issue in guppies (Poecilia reticulata) artificially selected for large and small relative brain size with pronounced behavioral differences in learning experiments and mate choice tests. We performed a study of the visual system by quantifying eye size and optomotor response of large-brained and small-brained guppies. This represents the first experimental test of the link between brain size evolution and visual acuity. We found that female guppies have larger eyes than male guppies, both in absolute terms and in relation to their body size. Likewise, individuals selected for larger brains had slightly larger eyes but not better visual acuity than small-brained guppies. However, body size was positively associated with visual acuity. We discuss our findings in relation to previous macroevolutionary studies on the evolution of brain morphology, eye morphology, visual acuity, and ecological variables, while stressing the importance of accounting for sensory abilities in behavioral studies.Significance statementPre-existing perceptual biases can be keys for the development of specific behavioral patterns. Hence, potential differences in sensory systems need to be taken into account in the study of animal behavior. We highlight this necessity concentrating on the visual domain and using experimental data on brain size-selected guppies in which we assessed eye size and visual acuity. Behavioral differences between large-brained and small-brained guppies in learning and mate choice predominantly relied on tests using visual cues. Analyses of visual capabilities in this system are therefore necessary. Furthermore, this system offers the unprecedented opportunity to experimentally test the relationship between brain size, eye morphology, and visual capabilities. Our results show similar visual acuities between large-brained and small-brained guppies. However, the differences observed in eye area between the sexes, together with the observed positive relationship between body size and visual acuity, highlight the need to incorporate perceptive differences in the study of animal behavior.Electronic supplementary materialThe online version of this article (10.1007/s00265-017-2408-z) contains supplementary material, which is available to authorized users.

Brain size is an energetically costly trait to develop and maintain. Investments into other costly aspects of an organism's biology may therefore place important constraints on brain size evolution. Sexual traits are often costly and could therefore be traded off against neural investment. However, brain size may itself be under sexual selection through mate choice on cognitive ability. Here, we use guppy (Poecilia reticulata) lines selected for large and small brain size relative to body size to investigate the relationship between brain size, a large suite of male primary and secondary sexual traits, and body condition index. We found no evidence for trade‐offs between brain size and sexual traits. Instead, larger‐brained males had higher expression of several primary and precopulatory sexual traits – they had longer genitalia, were more colourful and developed longer tails than smaller‐brained males. Larger‐brained males were also in better body condition when housed in single‐sex groups. There was no difference in post‐copulatory sexual traits between males from the large‐ and small‐brained lines. Our data do not support the hypothesis that investment into sexual traits is an important limiting factor to brain size evolution, but instead suggest that brain size and several sexual traits are positively genetically correlated.

The relationship between brain size and ageing is a paradox. The cognitive benefits of large brains should protect from extrinsic mortality and thus indirectly select for slower ageing. However, the substantial energetic cost of neural tissue may also impact the energetic budget of large-brained organisms, causing less investment in somatic maintenance and thereby faster ageing. While the positive association between brain size and survival in the wild is well established, no studies exist on the direct effects of brain size on ageing. Here we test how brain size influences intrinsic ageing in guppy ( Poecilia reticulata ) brain size selection lines with 12% difference in relative brain size. Measuring survival under benign conditions, we find that large-brained animals live 22% shorter than small-brained animals and the effect is similar in both males and females. Our results suggest a trade-off between investment into brain size and somatic maintenance. This implies that the link between brain size and ageing is contingent on the mechanism of mortality, and selection for positive correlations between brain size and ageing should occur mainly under cognition-driven survival benefits from increased brain size. We show that accelerated ageing can be a cost of evolving a larger brain.