Epigenetic inheritance is the transmission of nongenetic material such as gene expression levels, RNA, and other biomolecules from parents to offspring. There is a growing realization that such forms of inheritance can play an important role in evolution. Bacteria represent a prime example of epigenetic inheritance because a large array of cellular components are transmitted to offspring, in addition to genetic material. Interestingly, there is an extensive and growing empirical literature showing that many bacteria can form ‘persister’ cells that are phenotypically resistant or tolerant to antibiotics but most of these results are not interpreted within the context of epigenetic inheritance. Instead persister cells are usually viewed as a genetically encoded bet-hedging strategy that has evolved in response to a fluctuating environment. Here I show, using a relatively simple model, that many of these empirical findings can be more simply understood as arising from a combination of epigenetic inheritance and cellular noise. I therefore suggest that phenotypic drug tolerance in bacteria might represent one of the best-studied examples of evolution under epigenetic inheritance.