Context. Red supergiant (RSG) stars are very massive cool evolved stars. Recently, a weak magnetic field was measured at the surface of α Ori and this is so far the only M-type supergiant for which a direct detection of a surface magnetic field has been reported. Aims. By extending the search for surface magnetic field in a sample of late-type supergiants, we want to determine whether the surface magnetic field detected on α Ori is a common feature among the M-type supergiants. Methods. With the spectropolarimeter Narval at Télescope Bernard-Lyot we undertook a search for surface magnetic fields in a sample of cool supergiant stars, and we analysed circular polarisation spectra using the least-squares deconvolution technique. Results. We detect weak Zeeman signatures of stellar origin in the targets CE Tau, α 1 Her and µ Cep. For the latter star, we also show that cross-talk from the strong linear polarisation signals detected on this star must be taken into account. For CE Tau and µ Cep, the longitudinal component of the detected surface fields is at the Gauss-level, such as in α Ori. We measured a longitudinal field almost an order of magnitude stronger for α 1 Her. We also report variability of the longitudinal magnetic field of CE Tau and α 1 Her, with changes in good agreement with the typical atmospheric dynamics time-scales. We also report a non-detection of magnetic field at the surface of the yellow supergiant star ρ Cas. Conclusions. The two RSG stars of our sample, CE Tau and µ Cep, display magnetic fields very similar to that of α Ori. The nondetection of a magnetic field on the post-RSG star ρ Cas suggests that the magnetic field disappears, or at least becomes undetectable with present methods, at later evolutionary stages. Our analysis of α 1 Her supports the proposed reclassification of the star as an M-type asymptotic giant branch star.