Structural Maintenance of Chromosomes (SMC) protein complexes are key determinants of chromosome conformation. Using Hi-C and polymer modeling, we study how cohesin and condensin, two deeply conserved SMC complexes, organize chromosomes in the budding yeast Saccharomyces cerevisiae. The canonical role of cohesin is to co-align sister chromatids whilst condensin generally compacts mitotic chromosomes. We find strikingly different roles for the two complexes in budding yeast mitosis. First, cohesin is responsible for compacting mitotic chromosome arms, independently of sister chromatid cohesion. Polymer simulations demonstrate this role can be fully accounted for through cis-looping of chromatin. Second, condensin is generally dispensable for compaction along chromosome arms. Instead it plays a targeted role compacting the rDNA proximal regions and promoting resolution of peri-centromeric regions. Our results argue that the conserved mechanism of SMC complexes is to form chromatin loops and that distinct SMC-dependent looping activities are selectively deployed to appropriately compact chromosomes.