The basal breast cancer subtype is enriched for triple negative breast cancer (TNBC) and displays consistent large chromosomal deletions. Here, we characterize the evolution and maintenance of chromosome 4p (chr4p) loss in basal breast cancer. TCGA data analysis showed recurrent deletion of chr4p in basal breast cancer. Phylogenetic analysis of a unique panel of 23 primary tumor/patient-derived xenograft basal breast cancer revealed early evolution of chr4p deletion. Chr4p loss is associated with enhanced proliferation and immune evasion. We characterized an unknown gene, C4orf19, within chr4p, which suppressed proliferation when overexpressed and is a member of a novel CCM3-kinase module, and we refer to it as CGKA1. Genome-wide pooled overexpression screens using a barcoded library of human open reading frames, identified chromosomal regions, including chr4p, that suppress proliferation when overexpressed in a context-dependent manner. Together this sheds light on the early emergence of complex aneuploid karyotypes involving chr4p and adaptive landscapes shaping breast cancer genomes.