Here, we present the thermal tuning capability of an alignment-free fiber-integrated Fabry-Perot cavity. The two mirrors are made of fiber Bragg gratings that can be individually temperature stabilized and tuned. We show the temperature tuning of the resonance wavelength of the cavity without any degradation of the finesse and the tuning of the individual stop bands of the fiber Bragg gratings. This not only permits for the cavity's finesse to be optimized post-fabrication but also makes this cavity applicable as a narrowband filter with a FWHM spectral width of 0.07 ± 0.02 pm and a suppression of more than -15 dB that can be wavelength tuned. Further, in the field of quantum optics, where strong light-matter interactions are desirable, quantum emitters can be coupled to such a cavity and the cavity effect can be reversibly omitted and re-established. This is particularly useful when working with solid-state quantum emitters where such a reference measurement is often not possible once an emitter has been permanently deposited inside a cavity.