NICER observed several rotation-powered millisecond pulsars to search for or confirm the presence of X-ray pulsations. When broad and sine-like, these pulsations may indicate thermal emission from hot polar caps at the magnetic poles on the neutron star surface. We report confident detections (≥ 4.7σ after background filtering) of X-ray pulsations for five of the seven pulsars in our target sample: PSR J0614−3329, PSR J0636+5129, PSR J0751+1807, PSR J1012+5307, and PSR J2241−5236, while PSR J1552+5437 and PSR J1744−1134 remain undetected. Of those, only PSR J0751+1807 and PSR J1012+5307 had pulsations previously detected at the 1.7σ and almost 3σ confidence levels, respectively, in XMM-Newton data. All detected sources exhibit broad sine-like pulses, which are indicative of surface thermal radiation. As such, these MSPs are promising targets for future Xray observations aimed at constraining the neutron star mass-radius relation and the dense matter equation of state using detailed pulse profile modeling. Furthermore, we find that three of the detected millisecond pulsars exhibit a significant phase offset between their X-ray and radio pulses.