Recently, all-polymer-based p−n bulk heterojunction organic photodiodes exhibited a high photocurrent density (J ph ) through the state-of-the-art synthesis in n-type conjugated polymers. However, in the organic photodetector (OPD) application, the on−off ratio (J ph /dark current density (J d )) is a performance-determining parameter; thus herein we focused on effectively increasing the on−off ratio by lowering the J d of all-polymer OPDs. We developed four n-type polymers, with nonconjugated spacers, which effectively suppressed the J d of the devices; the increased insulating properties and excellent hole-blocking properties under dark conditions aided J d suppression. Moreover, breaking the π-conjugation of the n-type polymers, by adding short bulky nonconjugated spacers, effectively increased J ph and decreased J d in relation with those upon the addition of long flexible alkyl spacers. Hence, the OPDs based on the n-type polymer with a nonconjugated spacer exhibited a high specific detectivity of >10 12 Jones and a −3 dB cutoff frequency of 219 kHz. Notably, they also displayed ultrafast rising and falling response times of 2 and 9 μs, respectively, at −2 V.