Humans perceive light in the visible spectrum (400-700 nm). Some night vision systems use infrared light that is not perceptible to humans and the images rendered are transposed to a digital display presenting a monochromatic image in the visible spectrum. We sought to develop an imaging algorithm powered by optimized deep learning architectures whereby infrared spectral illumination of a scene could be used to predict a visible spectrum rendering of the scene as if it were perceived by a human with visible spectrum light. This would make it possible to digitally render a visible spectrum scene to humans when they are otherwise in complete “darkness” and only illuminated with infrared light. To achieve this goal, we used a monochromatic camera sensitive to visible and near infrared light to acquire an image dataset of printed images of faces under multispectral illumination spanning standard visible red (604 nm), green (529 nm) and blue (447 nm) as well as infrared wavelengths (718, 777, and 807 nm). We then optimized a convolutional neural network with a U-Net-like architecture to predict visible spectrum images from only near-infrared images. This study serves as a first step towards predicting human visible spectrum scenes from imperceptible near-infrared illumination. Further work can profoundly contribute to a variety of applications including night vision and studies of biological samples sensitive to visible light.
Purpose To evaluate the potential for artificial intelligence-based video analysis to determine surgical instrument characteristics when moving in the three-dimensional vitreous space. Methods We designed and manufactured a model eye in which we recorded choreographed videos of many surgical instruments moving throughout the eye. We labeled each frame of the videos to describe the surgical tool characteristics: tool type, location, depth, and insertional laterality. We trained two different deep learning models to predict each of the tool characteristics and evaluated model performances on a subset of images. Results The accuracy of the classification model on the training set is 84% for the x – y region, 97% for depth, 100% for instrument type, and 100% for laterality of insertion. The accuracy of the classification model on the validation dataset is 83% for the x – y region, 96% for depth, 100% for instrument type, and 100% for laterality of insertion. The close-up detection model performs at 67 frames per second, with precision for most instruments higher than 75%, achieving a mean average precision of 79.3%. Conclusions We demonstrated that trained models can track surgical instrument movement in three-dimensional space and determine instrument depth, tip location, instrument insertional laterality, and instrument type. Model performance is nearly instantaneous and justifies further investigation into application to real-world surgical videos. Translational Relevance Deep learning offers the potential for software-based safety feedback mechanisms during surgery or the ability to extract metrics of surgical technique that can direct research to optimize surgical outcomes.
IntroductionClinical tools are neither standardized nor ubiquitous to monitor volumetric or morphological changes in the periorbital region and ocular adnexa due to pathology such as oculofacial trauma, thyroid eye disease, and the natural aging process. We have developed a low-cost, three dimensionally printedPHotogrammetry forAutomatedCarE(PHACE) system to evaluate three-dimensional (3D) measurements of periocular and adnexal tissue.MethodsThe PHACE system uses two Google Pixel 3 smartphones attached to automatic rotating platforms to image a subject’s face through a cutout board patterned with registration marks. Photographs of faces were taken from many perspectives by the cameras placed on the rotating platform. Faces were imaged with and without 3D printed hemispheric phantom lesions (black domes) affixed on the forehead above the brow. Images were rendered into 3D models in Metashape (Agisoft, St. Petersburg, Russia) and then processed and analyzed in CloudCompare (CC) and Autodesk’s Meshmixer. The 3D printed hemispheres affixed to the face were then quantified within Meshmixer and compared to their known volumes. Finally, we compared digital exophthalmometry measurements with results from a standard Hertel exophthalmometer in a subject with and without an orbital prosthesis.ResultsQuantification of 3D printed phantom volumes using optimized stereophotogrammetry demonstrated a 2.5% error for a 244μL phantom, and 7.6% error for a 27.5μL phantom. Digital exophthalmometry measurements differed by 0.72mm from a standard exophthalmometer.ConclusionWe demonstrated an optimized workflow using our custom apparatus to analyze and quantify oculofacial volumetric and dimensions changes with a resolution of 244μL. This apparatus is a low-cost tool that can be used in clinical settings to objectively monitor volumetric and morphological changes in periorbital anatomy.
Purpose: To compare a custom Photogrammetry for Anatomical CarE (PHACE) system with other cost-effective 3-dimensional (3D) facial scanning systems to objectively characterize morphology and volume of periorbital and adnexal anatomy. Methods: The imaging systems evaluated include the low-cost custom PHACE system and commercial software product for the iPhone called Scandy Pro (iScandy) application (Scandy, USA), and the mid-priced Einscan Pro 2X (Shining3D Technologies, China) device and Array of Reconstructed Cameras 7 (ARC7) facial scanner (Bellus3D, USA). Imaging was performed on a manikin facemask and humans with various Fitzpatrick scores. Scanner attributes were assessed using mesh density, reproducibility, surface deviation, and emulation of 3D printed phantom lesions affixed above the superciliary arch (brow line). Results: The Einscan served as a reference for lower cost imaging systems because it qualitatively and quantitatively renders facial morphology with the highest mesh density, reproducibility (0.13 ± 0.10 mm), and volume recapitulation (approximately 2% of 33.5 μL). Compared to the Einscan, the PHACE system (0.35 ± 0.03 mm, 0.33 ± 0.16 mm) demonstrated non-inferior mean accuracy and reproducibility root mean square (RMS) compared to the iScandy (0.42 ± 0.13 mm, 0.58 ± 0.09 mm), and significantly more expensive ARC7 (0.42 ± 0.03 mm, 0.26 ± 0.09 mm). Similarly, the PHACE system showed non-inferior volumetric modeling when rendering a 124 μL phantom lesion compared to the iScandy and more costly ARC7 (mean percent difference from the Einscan: 4.68 ± 3.73%, 9.09 ± 0.94%, and 21.99 ± 17.91% respectively). Conclusions: The affordable PHACE system accurately measures periorbital soft tissue as well as other established mid-cost facial scanning systems. Additionally, the portability, affordability, and adaptability of PHACE can facilitate widespread adoption of 3D facial anthropometric technology as an objective measurement tool in ophthalmology.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.