The binocular vision is an important part of machine vision measurement. The calibration accuracy is crucial for binocular vision. As for the determination of the structure parameters of the two cameras, the existing approaches usually obtain the initial values and optimize them according to the image-space errors, object-space errors or combination of them. In the optimization process, constructing the objective function only through the image-space errors or object-space errors is not enough. Moreover, the image-space errors and object-space errors can form a variety of combinations for constructing objective function. Therefore, it is hard to choose the error criterion. The inadequate error criterion may lead to over optimized or local minima (ambiguity solution). To solve the problem above, this paper proposes a simple and precise calibration method for binocular vision based on the points distance constraints and image-space errors. The process of determining the structure parameters was divided into non-iterative and iterative parts. We calculated the structure parameters of the two cameras according to distance constraints of every two feature points non-iteratively. The results obtained in this step were set as the initial value and refined through minimizing the reprojection errors by Levenberg-Marquardt method. Because the results obtained in the non-iterative step are accurate enough, only one iteration is needed. In this way, we finish the calibration avoiding to choose the error criterions. Furthermore, our method reduces the number of iterations to improve the calibration efficiency on the premise of guaranteeing the calibration accuracy. The experimental results show the superiority of this calibration method compared with other calibration methods.