Objective: Smoke, uneven lighting, and color deviation are common issues in endoscopic surgery, which have increased the risk of surgery and even lead to failure. Approach: In this study, we present a new physics model driven semi-supervised learning framework for high-quality pixel-wise endoscopic image enhancement, which is generalizable for smoke removal, light adjustment, and color correction. To improve the authenticity of the generated images, and thereby improve the network performance, we integrated specific physical imaging defect models with the CycleGAN framework. No ground-truth data in pairs are required. In addition, we propose a transfer learning framework to address the data scarcity in several endoscope enhancement tasks and improve the network performance. Main results: Qualitative and quantitative studies reveal that the proposed network outperforms the state-of-the-art image enhancement methods. In particular, the proposed method performs much better than the original CycleGAN, for example, the structural similarity (SSIM) improved from 0.7925 to 0.8648, feature similarity for color images (FSIMc) from 0.8917 to 0.9283, and quaternion structural similarity (QSSIM) from 0.8097 to 0.8800 in the smoke removal task. Experimental results of the proposed transfer learning method also reveal its superior performance when trained with small datasets of target tasks. Significance: Experimental results on endoscopic images prove the effectiveness of the proposed network in smoke removal, light adjustment, and color correction, showing excellent clinical usefulness.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.