With the increased emphasis on competition in academic settings, anxiety is becoming more common, which inevitably has some impact on students’ learning processes and results. This study aimed to explore how competition-induced anxiety influences students’ subjective cognitive load (SCL), attention levels, and test scores. We also investigated the mediating role of the behavioral inhibition system/behavioral activation system (BIS/BAS) in those factors. A total of 101 college students were recruited in Study 1 to learn from five micro-lectures from massive open online courses (MOOCs) under competitive and non-competitive conditions. The results showed that participants’ state anxiety (SA) scores were higher after the experiment, participants under the competition condition had higher test scores, and the relationship between SA/ trait anxiety (TA) and SCL could be mediated by the BIS. To obtain more objective data on learning processes (attention levels), we conducted Study 2, which collected behavioral and EEG data from 42 college students during the online learning. The results showed that the competition group had higher SA, lower attention levels, and worse test scores, and the relationship between SA/TA and attention levels could be mediated through the BIS. The present study not only expands previous research by finding that BIS functioning plays an important role in the effects of anxiety on cognitive load and attention but also offers implications for using competitive strategies to motivate students according to their aptitudes.