The implantable cardioverter defibrillator (ICD) is a life-saving therapy in various patient populations. Although data on the outcomes of initial ICD implants are abundant, data on ICD replacements, especially in patients with improved left ventricular (LV) function, are scarce. Therefore, it is not known when it is safe to not replace an ICD that has reached the end of battery life. This article reviews data on patients with primary prevention ICDs who have improvement in left ventricular ejection fraction during follow-up and provides some guidance, based on the available evidence, related to circumstances when replacement of an ICD may be forgone.