Heterogeneous sensors fusion of a camera and a laser-rangefinder can greatly improve the environment perception ability, and its primary problem is the calibration of depth scan and image information. At first, the mapping relationship among world coordinate system, camera coordinate system, and image plane is discussed, and then the calibration of camera intrinsic parameters is achieved. Moreover, the intrinsic and extrinsic parameters separated calibration is presented for a camera and a laser-rangefinder, and the characteristic identification is adopted by two intersection calibration boards with a certain angel for fusion characters extraction. Furthermore, the particle swarm optimization is proposed for the extrinsic parameters estimation with different objectives, and the Gaussian elimination is utilized for the initial particle swarm. The simulation and real experimental results show that the standard deviation of calibration error in the 21-group experiments is decreased by 10.175%, and it also proves the accuracy and effectiveness of our approaches.