The isolation of photodetectors by deep trenches filled with insulating material is widely used to minimize crosstalk between devices fabricated on the same chip. At the same time, however, the interface of the deep trench isolation (DTI) introduces trap states that can act as recombination centers and affect the performance of Si photodetectors. In this work, the impact of carrier interaction with trap states at the DTI sidewalls on the responsivity of Si photodetectors using a 180 nm complementary metal-oxide-semiconductor technology is investigated. It is observed that DTI-terminated island photodiodes show larger responsivities compared to n-well guard (NWG) terminated island photodiodes. This superiority in terms of responsivity is more pronounced at high radiant fluxes because of reduced Shockley-Read-Hall (SRH) surface recombination at the lateral DTI to Si interface. Therefore, the responsivity of the DTI-terminated devices becomes nonlinear with respect to the incoming radiant flux. We show that this can be explained by the nonlinear minority carrier surface recombination at the DTI sidewalls. Responsivity measurements on reference samples with an NWG termination do not show this flux dependency, which confirms our conclusions. Furthermore, we investigate the influence of DTI sidewall implants to passivate the lateral oxide to silicon interfaces with respect to the nonlinear response and overall spectral response (SR). Our experimental findings are supported by TCAD simulations which are also presented in this work.