Factory workers make up a large proportion of China’s internal migrants and may be highly susceptible to job and adaptation stress, negative affective states (e.g., depression and anxiety), and Internet gaming disorder (IGD). This cross-sectional study investigated the relationships between job stress, psychological adaptation, negative affective states and IGD among 1,805 factory workers recruited by stratified multi-stage sampling between October and December 2019. Structural equation modeling (SEM) was conducted to test the proposed mediation model. Among the participants, 67.3% were male and 71.7% were aged 35 years old or below. The prevalence of probable depression, probable anxiety, and IGD was 39.3, 28.7, and 7.5%. Being male, younger age, and shorter duration of living in Shenzhen were associated with higher IGD scores. Job stress was significantly associated with IGD (β = 0.11, p = 0.01) but not with negative affective states (β = 0.01, p = 0.77). Psychological adaptation was significantly associated with negative affective states (β = −0.37, p < 0.001) but not with IGD (β = 0.09, p > 0.05). Negative affective states were positively associated with IGD (β = 0.27, p < 0.001). The indirect effect of psychological adaptation (β = −0.10, p = 0.004) but not job stress (β = 0.003, p = 0.77) on IGD through negative affective states was statistically significant. The observed psychological correlates and mechanisms are modifiable, and can inform the design of evidence-based prevention programs for depression, anxiety, and IGD in this population.