Miniaturized 3D Depth Sensing-Based Smartphone Light Field Camera

Kim, Hyun Myung; Kim, Min Seok; Lee, Gil Ju; Jang, Hyuk Jae; Song, Young Min

doi:10.3390/s20072129

Cited by 28 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 28 Light-field imaging is an emerging alternative to stereoscopic imaging but has been evaluated in a relatively small number of studies and has not yet to be integrated in commercial systems. 29 – 31 Most recently, 3D time-of-flight cameras are an emerging capability which can potentially provide depth information at sufficiently quantitative spatial and temporal resolutions for dynamically measuring distances and volumes. 32 , 33 Ulrich et al.…”

Section: System Interface Designmentioning

confidence: 99%

Smartphone-based imaging systems for medical applications: a critical review

2021

View full text Add to dashboard Cite

. Significance: Smartphones come with an enormous array of functionality and are being more widely utilized with specialized attachments in a range of healthcare applications. A review of key developments and uses, with an assessment of strengths/limitations in various clinical workflows, was completed. Aim: Our review studies how smartphone-based imaging (SBI) systems are designed and tested for specialized applications in medicine and healthcare. An evaluation of current research studies is used to provide guidelines for improving the impact of these research advances. Approach: First, the established and emerging smartphone capabilities that can be leveraged for biomedical imaging are detailed. Then, methods and materials for fabrication of optical, mechanical, and electrical interface components are summarized. Recent systems were categorized into four groups based on their intended application and clinical workflow: ex vivo diagnostic, in vivo diagnostic, monitoring, and treatment guidance. Lastly, strengths and limitations of current SBI systems within these various applications are discussed. Results: The native smartphone capabilities for biomedical imaging applications include cameras, touchscreens, networking, computation, 3D sensing, audio, and motion, in addition to commercial wearable peripheral devices. Through user-centered design of custom hardware and software interfaces, these capabilities have the potential to enable portable, easy-to-use, point-of-care biomedical imaging systems. However, due to barriers in programming of custom software and on-board image analysis pipelines, many research prototypes fail to achieve a prospective clinical evaluation as intended. Effective clinical use cases appear to be those in which handheld, noninvasive image guidance is needed and accommodated by the clinical workflow. Handheld systems for in vivo , multispectral, and quantitative fluorescence imaging are a promising development for diagnostic and treatment guidance applications. Conclusions: A holistic assessment of SBI systems must include interpretation of their value for intended clinical settings and how their implementations enable better workflow. A set of six guidelines are proposed to evaluate appropriateness of smartphone utilization in terms of clinical context, completeness, compactness, connectivity, cost, and claims. Ongoing work should prioritize realistic clinical assessments with quantitative and qualitative comparison to non-smartphone systems to clearly demonstrate the value of smartphone-based systems. Improved hardware design to accommodate the rapidly changing smartphone ecosystem, creation of open-source image acquisition and analysis pipelines, and adoption of robust calibration techniques to address phone-to-phone variability are three high priority areas to move SBI research forwar...

show abstract

Section: System Interface Designmentioning

confidence: 99%

Smartphone-based imaging systems for medical applications: a critical review

2021

View full text Add to dashboard Cite

show abstract

“…Apart from these systems, SUGO exploits only the depth video information, which is available on most recently released smartphones. Such new data modality collected from a smartphone's depth camera has been recently applied in designing novel application systems such as Hand gesture recognition [26], authentication [59], and mobile augmented reality (AR) [12]. Exploiting depth videos allow for the ubiquitous use of sign language translation independent of the environment, and without additional hand-carried sensors.…”

Section: Related Workmentioning

confidence: 99%

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Park

Lee

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

In this work we present SUGO, a depth video-based system for translating sign language to text using a smartphone's front camera. While exploiting depth-only videos offer benefits such as being less privacy-invasive compared to using RGB videos, it introduces new challenges which include dealing with low video resolutions and the sensors' sensitiveness towards user motion. We overcome these challenges by diversifying our sign language video dataset to be robust to various usage scenarios via data augmentation and design a set of schemes to emphasize human gestures from the input images for effective sign detection. The inference engine of SUGO is based on a 3-dimensional convolutional neural network (3DCNN) to classify a sequence of video frames as a pre-trained word. Furthermore, the overall operations are designed to be light-weight so that sign language translation takes place in real-time using only the resources available on a smartphone, with no help from cloud servers nor external sensing components. Specifically, to train and test SUGO, we collect sign language data from 20 individuals for 50 Korean Sign Language words, summing up to a dataset of ~5,000 sign gestures and collect additional in-the-wild data to evaluate the performance of SUGO in real-world usage scenarios with different lighting conditions and daily activities. Comprehensively, our extensive evaluations show that SUGO can properly classify sign words with an accuracy of up to 91% and also suggest that the system is suitable (in terms of resource usage, latency, and environmental robustness) to enable a fully mobile solution for sign language translation.

show abstract

“…The fourth revolution of science and technology provides a new driving force for the development of high-temperature metal component 3D size and morphology measurements. The accuracy and speed of measurement systems will continue to be improved, and miniaturized measurement systems, such as the low-depth camera systems of mobile phones [ 105 , 106 , 107 , 108 , 109 , 110 ], will continue to be produced. The following is an analysis of and outlook on the future development trends of the measurement of high-temperature metal components with stereo vision:…”

Section: Future Development Trendsmentioning

confidence: 99%

Three-Dimensional Morphology and Size Measurement of High-Temperature Metal Components Based on Machine Vision Technology: A Review

Wen

Wang

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

The three-dimensional (3D) size and morphology of high-temperature metal components need to be measured in real time during manufacturing processes, such as forging and rolling. Since the surface temperature of a metal component is very high during the forming and manufacturing process, manually measuring the size of a metal component at a close distance is difficult; hence, a non-contact measurement technology is required to complete the measurement. Recently, machine vision technology has been developed, which is a non-contact measurement technology that only needs to capture multiple images of a measured object to obtain the 3D size and morphology information, and this technology can be used in some extreme conditions. Machine vision technology has been widely used in industrial, agricultural, military and other fields, especially fields involving various high-temperature metal components. This paper provides a comprehensive review of the application of machine vision technology in measuring the 3D size and morphology of high-temperature metal components. Furthermore, according to the principle and method of measuring equipment structures, this review highlights two aspects in detail: laser scanning measurement and multi-view stereo vision technology. Special attention is paid to each method through comparisons and analyses to provide essential technical references for subsequent researchers.

show abstract

Miniaturized 3D Depth Sensing-Based Smartphone Light Field Camera

Cited by 28 publications

References 31 publications

Smartphone-based imaging systems for medical applications: a critical review

Smartphone-based imaging systems for medical applications: a critical review

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Three-Dimensional Morphology and Size Measurement of High-Temperature Metal Components Based on Machine Vision Technology: A Review

Contact Info

Product

Resources

About