Due to its unique band structure and topological properties, the 2D topological semimetal exhibits potential applications in photoelectric detection, polarization sensitive imaging, and Schottky barrier diodes. However, its inherent large dark current hinders the further improvement of the performance of the semimetal‐based photodetectors. In this study, a van der Waals (vdWs) field effect transistor (FET) composed of semimetal PdTe2 and transition metal dichalcogenides (TMDs) WSe2 is fabricated, which exhibits high sensitivity photoelectric detection performance in a wide band from visible light (405 nm) to mid‐infrared (5 µm). The dark current and the noise level in the device are greatly suppressed by the effective control of the gate. Benefiting from the extremely low dark current (1.2 pA), the device achieves an optical on/off ratio up to 106, a high detectivity of 9.79 × 1013 Jones and a rapid response speed (219 and 45 µs). This research demonstrates the latent capacity of the 2D topological semimetal/TMDs vdWs FET for broadband, high‐performance, and miniaturized photodetection.