Non-sexual social selection can underlie the evolution of sexually monomorphic phenotypes. A causal relationship between territorial competition and sexual monomorphism predicts that male and female competitors should employ similar contest behavior and that contest outcome should depend on the same traits in males and females. We test this prediction in a sexually monomorphic cichlid fish of the genus Tropheus, in which males and females defend individual feeding territories. Lineages basal to Tropheus are sexually dimorphic and have non-territorial females, suggesting that a switch to female territoriality and loss of sexual dimorphism occurred in the Tropheus lineage. We compare rates of agonistic behavior and the effects of body size asymmetries on competitive success between male-male and female-female contests in an experimental setup. Body size asymmetry had the same effect in male and female contests, being negatively correlated with contest duration and positively correlated with the probability of winning. Male and female winners employed the same rates of frontal and lateral displays as well as charges against their opponents. Contest duration was longer in females. In tied contests, females displayed more than males. Our data suggest that intraspecific contest competition for territories selects for large body size in both sexes and support a link between the evolution of female territoriality and the loss of sexual size dimorphism in Tropheus.Electronic supplementary materialThe online version of this article (doi:10.1007/s00265-014-1870-0) contains supplementary material, which is available to authorized users.