The structure of silver clusters containing up to 80 atoms is optimized by using a random tunneling algorithm (RTA). The Gupta-type many-body potential and the Sutton-Chen (SC) many-body potential are used to account for the interactions among the atoms in the cluster, respectively. It is found that many of the structural configurations optimized with the two potentials are different, especially at a small size, and the structures with the Gupta potential are highly strained and apt to be the disordered motifs, whereas the structures with the SC potential are less-strained and the ordered morphologies are more favorable. Due to the difference of long-range interaction and pair contribution between two potentials, the assignment of outermost-shell atoms in decahedron and the missing atoms on the surface of the icosahedral motifs are also different for the two potentials. Furthermore, a new global minimum of the 68-atom silver cluster with the decahedral motif is found for both two potential models.