Selenium-filled double-walled carbon nanotubes (Se@DWNT) have been studied by high resolution transmission electron microscopy (HRTEM) and micro-Raman spectroscopy in the temperature interval from 80to600K employing 785nm excitation wavelength. The temperature dependences of the dominant bands (G-band and G′-band) are analyzed in terms of the model developed by Klemens [Phys. Rev. 148, 845 (1966)], Hart et al. [Phys. Rev. B 1, 638 (1970)], Cowley [J. Phys. (France) 26, 659 (1965)] and extended by Balkanski et al. [Phys. Rev. B 26, 1928 (1983)] for anharmonic decay of optical phonons. The findings were compared to analogous study for empty double-walled carbon nanotubes (DWNTs). The DWNT interatomic force constant modification as a result of the presence of the Se atoms inside the tubes is revealed through larger anharmonicity constants describing the temperature dependences of the G′-band and the inner tube tangential modes (G-band).