Trust mechanism has become a research focus in recent years as a novel and valid way to ensure the transaction security in peer‐to‐peer file sharing networks. Nevertheless, some fundamental challenges still exist, for example: How can malicious peers be effectively isolated? How can various threats of manipulation by strategic peers be resisted? What strategy should be used to ensure that the service providers are authentic peers? Considering these challenges in our minds, in this paper, we propose a new trust mechanism based on the maximum flow theory. We firstly add a few prestigious peers into a cluster as the original members according to their transaction behaviors in a period; then, we perform maximum flow algorithm and identify those peers that still link from (to) the peers in the cluster as new members, which is carried out repeatedly, and almost every normal peer would finally become the member of the cluster. Each request peer has the priority to select downloading sources from this cluster according to our trust mechanism. In this way, the malicious peers are isolated, and their transaction behaviors are also confined largely even though they have high reputation. Extensive experimental results confirm the efficiency of our trust mechanism against the threats of exaggeration, cheat, collusion, and disguise. Copyright © 2013 John Wiley & Sons, Ltd.