Phase retrieval from a single-frame interferogram is a challenge in optical interferometry. This paper proposes an accurate physics-based deep learning method for one-shot phase retrieval. This approach involves both data-driven pre-training of a phase-shifting network and subsequent model-driven fine-tuning. The well-designed pre-training network is capable of simultaneously generating π/2, π, and 3π/2 phase-shifted versions of the input interferogram to facilitate phase extraction. Moreover, integrating the interferometric model into the testing dataset enables self-supervised fine-tuning, optimizing the use of both data and physics-based priors. Simulations and experiments demonstrate the effectiveness of the proposed method in overcoming the common generalization limitation of data-driven models and achieving accurate phase retrieval. The proposed method not only enhances the accuracy of phase retrieval but also improves the generalization capability, making it robust under experimental conditions for interferometric applications.