Brain derived neurotrophic factor (BDNF) is a member of the neurotrophin family that plays a critical role in numerous neuronal activities. Recent studies report that some functions or action mechanisms of BDNF vary in a sex-dependent manner. In particular, BDNF content in some brain parts and the tendency of developing BDNF-deficient-related diseases like depression is higher in female animals. With the support of other relevant studies, it is suggested that sex hormones or steroids can modulate the activities of BDNF, which may account for its functional discrepancy in different sexes. Indeed, the cross-talk between BDNF and sex steroids has been detected for decades and some sex steroids like estrogen have a positive regulatory effect to BDNF expression and signaling. Thus, the sex of animal models used is critical when studying the functions of BDNF in vivo. In this review, we will summarize our current findings on the difference in expression, signaling, and functions of BDNF between sexes. We will also discuss the potential mechanisms in mediating these differential responses with a specific emphasis on sex steroids. By presenting and discussing these findings, we encourage taking sex influences into consideration when designing experiments, interpreting results and drawing conclusions.