We have studied the effects of electrode fabrication and detector capacitance on the time resolution of large area electronic grade polycrystalline diamond sensors, made using chemical vapour deposition, that are suitable for time of flight measurements of heavy ions at relativistic velocities. Sensors were prepared both in house, with Al or Au metal contacts, and commercially fabricated with Au/diamond-like carbon contacts.3 He, 40 Ar and a mixture of 20 Ne and 16 O beams at 16.3, 33.5 and 21-23 MeV/u, respectively were used on these devices whilst arranged in transmission geometry. Signal processing only began over one meter away from the sensors. The present approach, where we have large-area/large capacitance multi-strip detectors with processing electronics at some distance from the target, is compatible with anticipated space limitations in particle-identification and tracking setups at existing and planned nuclear fragmentation facilities. In a systematic study under these conditions, we demonstrate that the time resolution is limited by detector capacitance and energy deposition in the sensors. An intrinsic time resolution σ t = (44±5) ps was achieved for a diamond detector of ~14 pF capacitance. We conclude that, once further refinements are made, a large area time of flight detection system using polycrystalline diamond detectors would be able to provide time resolutions better than 40 ps, approaching the requirement for particle-identification in relativistic fragmentation experiments, such as those at the facility for antiproton and ion research, FAIR.