This article proposes and demonstrates an in‐line fiber Mach‐Zehnder interferometer (MZI) based on few‐mode fiber for simultaneous measurement of temperature and refractive index. The Mach‐Zehnder interferometer is fabricated by fusion splicing a section few‐mode fiber (FMF) between hollow fiber (HF) microsphere cavity and single mode fiber (SMF) sphere. Since the different sensitivity responses of the two resonance dips in the transmission spectrum, the proposed sensor can simultaneous measurement of temperature and refractive index by monitoring the wavelength shifts of the two resonance dips. The experimental results show that the temperature sensitivities are 47.7 and 43.4 pm/°C within the temperature range from 30 to 80°C, respectively (red shift). The refractive index sensitivities are −21.41 and −30.11 nm/RIU when the refractive index changes from 1.330 to 1.394 RIU, respectively (blue shift). Furthermore, this sensor has excellent advantages of high extinction ratio, cost‐effective, ease of fabrication and compact structure, which make it competitive in the field of dual‐parameter sensing applications.