Pectoral fins are important actuation mechanisms in achieving maneuvering and propulsion for robotic fish. Existing designs predominantly adopt a rigid joint connecting the actuator to the pectoral fin, which requires differential actuation speeds in the power and recovery strokes in order to produce thrust and thus limits the overall actuation frequency. To address this problem, in this paper we propose a novel design of a flexible joint, which enables the pectoral fin to sweep back passively along the fish body in the recovery stroke, to minimize the drag, while maintaining the prescribed motion in the power stroke. A dynamic model is presented for a robotic fish propelled by pectoral fins incorporating such flexible joints, where the pectoral fin mechanism is modeled by two rigid segments connected with a pair of torsional spring and damper. This design results in a net thrust even with the same recovery and power stroke speed within each fin beat cycle, which simplifies the fin control. Experimental results on a robotic fish prototype are presented to validate the effectiveness of the proposed model, and to demonstrate the significant advantage of the proposed fin joint over the rigid joint. Furthermore, modeling analysis and experimental investigation are used to examine the influence of the joint length and stiffness on the locomotion performance.