In this paper, a no-reference perceptual quality assessment for stereoscopic image is proposed. Inspired by the binocular rivalry mechanism, the observation annoyance perception is explained as a bargain process. Game theory is exploited to model the rivalry of the left eye and right eye. The relation between annoyance perception with binocular disparity is further demonstrated and an annoyance map is calculated to simulate the observer perception. Then, with the consideration of the properties of HVS, the edge map and a saliency map are used to adjust the annoyance map. Finally, Minkowski pooling and multi-scale strategy are applied to compute the final score. We use the EPFL database to validate the proposed metric. The experimental results show that the final objective scores have a high degree of consistency with the subjective scores.