With the continuous expansion of the Industrial Internet of Things (IIoT) and the increasing connectivity among the various intelligent devices or systems, the control of access and fusion in smart factory networks has significantly gained importance. However, the contradiction between the high Quality of Service (QoS) requirements of massive data and the limited network bandwidth and the heterogeneous network is becoming deeper and deeper. The heterogeneity of smart factory networks brings many challenges to unified access and fusion, real-time transmission, and centralized control and management. This paper provides a survey on heterogeneous networks in smart factories. We first study and discuss the heterogeneity of smart factory networks, and then discuss the existing mainstream wired and wireless network technologies, as well as promising future technologies, including 5G, OLE for Process Control Unified Architecture (OPC UA), and Time-Sensitive Networking (TSN). In addition, we also analyze current heterogeneous network fusion architecture and discuss the enabling technologies of heterogeneous network fusion in view of the shortcoming of the current solutions. Finally, we conclude with a discussion of open challenges and future research directions towards the effective realization of the smart factory.