Enhancement of Visual Realism with BRDF for Patient Specific Bronchoscopy Simulation

Abstract

Abstract. This paper presents a novel method for photorealistic rendering of the bronchial lumen by directly deriving matched shading and texture parameters from video bronchoscope images. 2D/3D registration is used to match video bronchoscope images with 3D CT scan of the same patient, such that patient specific modelling and simulation with improved visual realism can be achieved. With the proposed method, shading parameters are recovered by modelling the bidirectional reflectance distribution function (BRDF) of the visible surfaces by exploiting the restricted lighting configurations imposed by the bronchoscope. The derived BRDF is then used to predict the expected shading intensity such that a texture map independent of lighting conditions can be extracted. This allows the generation of new views not captured in the original bronchoscopy video, thus allowing free navigation of the acquired 3D model with enhanced photo-realism. BackgroundWith the maturity of minimal access surgery in recent years, there has been an increasing demand of patient specific simulation devices for both training and skills assessment. This is due to the fact that the complexity of the instrument controls, restricted vision and mobility, difficult hand-eye co-ordination, and the lack of tactile perception are major obstacles in performing minimal access surgeries. They require a high degree of manual dexterity from the operator. Computer simulation provides an attractive means of performing certain aspects of this training, particularly the hand eye co-ordination and instrument control. One significant challenge to computer based simulation is the creation of patient specific models combined with photo-realistic rendering so that basic as well as advanced surgical skills can be assessed with these simulation platforms.For patient specific bronchoscope simulation, a number of techniques have been proposed for co-registering bronchoscope videos with 3D tomographic data such that camera pose in relation to the bronchial tree during video bronchoscope examination can be derived [1,2] With the use of image based modelling and rendering techniques, it is possible to extend conventional texture mapping to support the representation of 3D surface details and view motion parallax in addition to photorealism [3,4,5]. One of the major challenges of combining 2D

Patient-specific bronchoscopy visualization through BRDF estimation and disocclusion correction

“…Normally a large number of images must be captured, however there are techniques which rely on the uniformity of the materials so that the number of input images can be significantly reduced [19]. For applications such as bronchoscopy, one can make reasonable assumptions about the form of the BRDF and the variation of the texture that would allow successful recovery of shading parameters and texture maps [20].…”

“…2 where 2-D/3-D registration was applied to the 2-D video and 3-D CT data sets after radial distortion correction of the video camera [32]. Illumination parameters are extracted through BRDF and attenuation estimation, which are then used to recover the global texture map [20]. The extracted surface texture and surface illumination characteristics allow the generation of new views of the 3-D model by using the 3-D mesh derived from the 3-D CT images.…”

Photorealistic modeling of tissue reflectance properties

“…The texture map is derived directly from the video bronchoscope images. The shading parameters are recovered by modeling the bidirectional reflectance distribution function r of the visible surfaces using a cubic curve parameterized on g (the cosine of the angle between the viewing vector V and surface normal N) as follows [22]:…”

Photo-Realistic Tissue Reflectance Modelling for Minimally Invasive Surgical Simulation

“…The texture map is derived directly from the video bronchoscope images. The shading parameters are recovered by modeling the bidirectional reflectance distribution function (BRDF) ρ of the visible surfaces by using a cubic curve parameterized on γ, the cosine of the angle between the viewing vector (V) and surface normal (N) [14]:…”