Retinal fundus imaging with a plenoptic sensor

Abstract

Vitreoretinal surgery is moving towards 3D visualization of the surgical field. This require acquisition system capable of recording such 3D information. We propose a proof of concept imaging system based on a light-field camera where an array of micro-lenses is placed in front of a conventional sensor. With a single snapshot, a stack of images focused at different depth are produced on the fly, which provides enhanced depth perception for the surgeon. Difficulty in depth localization of features and frequent focus-change during surgery are making current vitreoretinal heads-up surgical imaging systems cumbersome to use. To improve the depth perception and eliminate the need to manually refocus on the instruments during the surgery, we designed and implemented a proof-of-concept ophthalmoscope equipped with a commercial light-field camera. The sensor of our camera is composed of an array of micro-lenses which are projecting an array of overlapped micro-images. We show that with a single light-field snapshot we can digitally refocus between the retina and a tool located in front of the retina or display an extended depth-of-field image where everything is in focus. The design and system performances of the plenoptic fundus camera are detailed. We will conclude by showing in vivo data recorded with our device.

3D retinal imaging and measurement using light field technology

“…Image reconstruction produced subjectively high-contrast, good-resolution images compared with results of other authors working on LF fundus photography. 11 , 21 The vascular contrast recorded with our LF system also corresponds to values in the literature measured with conventional systems. 25 , 26 Compared with images from conventional fundus cameras, a detailed image from our system requires ideal conditions such as a large pupil and little stray light.…”

“…The LF system shows a similar but slightly better MTF characteristic compared with the system using a conventional CCD imager, which is most likely due to its smaller pixels and thus higher resolution compared with Thurin et al., who measured about at ; we achieved better results with about . 21 In comparison with Thurin et al., we additionally included the optics of the model eye for MTF determination to obtain a more realistic estimate. The depth differences of the optic nerve head of the subjects with suspected glaucoma compared with those of the healthy subjects are clearly visible and measurable ( Fig.…”

“…The first applications of fundus imaging utilizing LF technology were addressed by some patents 18 – 20 and publications. 11 , 21 Palmer et al. 11 showed that image enhancement regarding suppression of glare and depth mapping of the fundus are possible.…”

“…also developed an LF fundus camera for 3D imaging. 21 Both showed the possibility of retinal LF imaging in principle, though image quality seemed to be much lower than in standard fundus photography. From their example images, no diagnostic parameters were determined.…”

Autonomy in Surgical Robotics

“…If combined with fluoroscopy imaging as in [114], plenoptic cameras allow to identify internal hidden structures such as blood vessels and nerves. Although a small amount of contributions is available in literature, interesting results have been obtained so far, such as the completion of a needle insertion [115], Level-3 suturing (as detailed in Section 5.3), and vitreoretinal surgery [116]. Alternative imaging technologies currently under evaluation to enhance feature extraction are hyperspectral imaging [117] and TeraHertz vision [118].…”