Unsafe behavior is a critical factor leading to construction accidents. Despite numerous studies supporting this viewpoint, the process by which accidents are influenced by construction workers’ unsafe behaviors and the extent to which unsafe behaviors are involved in this process remain poorly discussed. Therefore, this paper selects cases from Chinese building construction accidents to explore the probabilistic transmission paths from unsafe behaviors to accidents using a Bayesian network. First, a list of unsafe behaviors is constructed based on safety standards and operating procedures. Second, several chains of unsafe behaviors are extracted from 287 accident cases within four types (fall, collapse, struck-by and lifting) to form a Bayesian network model. Finally, two accidents are specifically analyzed to verify the rationality of the proposed model through forward reasoning. Additionally, critical groups of unsafe behaviors leading to the four types of accidents are identified through backward reasoning. The results show the following: (i) The time sequence of unsafe behaviors in a chain does not affect the final posterior probability of an accident, but the accident attribute strength of an unsafe behavior, affects the growth rate of the posterior probability of an accident. (ii) The four critical groups of unsafe behaviors leading to fall, collapse, struck-by, and lifting are identified. This study is of theoretical and practical significance for on-site behavioral management and accident prevention.