Background Brain-derived neurotrophic factor (BDNF) has been a focus of psychiatric research for the past two decades. BDNF has been shown to impact neural function and development. Studies have investigated serum BDNF as a biomarker for psychiatric disorders such as depression and schizophrenia. In some studies, investigators attempt to control for variables such as smoking status, exercise, or diet. However, the relationship between these factors and BDNF is not clearly established. Furthermore, some studies have questioned whether a difference in the impact of BDNF exists between psychiatric and healthy populations. Purpose We aim to examine the association between serum BDNF levels and modifiable risk factors such as body mass index (BMI), smoking, exercise levels, and diet. Subsequently, we aim to examine whether the relationship between these risk factors and serum BDNF is different between psychiatric and control populations. Patients and Methods We use cross-sectional data from an age- and sex-matched case–control study of participants with psychiatric inpatients and community controls without psychiatric diagnoses. Participants completed comprehensive assessments at study enrolment including sociodemographic information, smoking status, exercise, diet, and BMI. Serum BDNF levels were collected from participants. Linear regression analysis was performed to determine the association between modifiable factors and serum BDNF level. Results A significant association was found between sedentary activity level and lower serum BDNF levels (Beta coefficient = –2.49, 95% confidence interval [CI] –4.70, –0.28, p = 0.028). Subgroup analysis demonstrated that this association held for psychiatric inpatients but not for community controls; it also held in females (Beta coefficient = –3.18, 95% CI –6.29, –0.07, p = 0.045) but not in males (Beta coefficient = –1.42, 95% CI –4.61, 1.78, p = 0.383). Antidepressant use had a significantly different association between male (Beta coefficient = 3.20, 95% CI 0.51, 5.88, p = 0.020) and female subgroups (Beta coefficient = –3.10, 95% CI –5.75, –0.46, p = 0.022). No significant association was found between other factors and serum BDNF. Conclusion Sedentary activity level may lead to lower serum BDNF levels in individuals with psychiatric diagnoses. Our findings support the notion that physical activity can provide a positive impact as part of treatment for psychiatric illness.