Adipose tissue (AT) forms depots at different anatomical locations throughout the body, being in subcutaneous and visceral regions, as well as the bone marrow. These ATs differ in the adipocyte functional profile, their insulin sensitivity, adipokines' production, lipolysis, and response to pathologic conditions. Despite the recent advances in lineage tracing, which have demonstrated that individual adipose depots are composed of adipocytes derived from distinct progenitor populations, the cellular and molecular dissection of the adipose clonogenic stem cell niche is still a great challenge. Additional complexity in AT regulation is associated with tumor-induced changes that affect adipocyte phenotype. As an integrative unit of cell differentiation, AT microenvironment regulates various phenotype outcomes of differentiating adipogenic lineages, which consequently may contribute to the neoplastic phenotype manifestations. Particularly interesting is the capacity of AT to impose and support the aberrant potency of stem cells that accompanies tumor development. In this review, we summarize the current findings on the communication between adipocytes and their progenitors with tumor cells, pointing out to the coexistence of healthy and neoplastic stem cell niches developed during tumor evolution. We also discuss tumor-induced adaptations in mature adipocytes and the involvement of alternative differentiation programs.