Deep learning integral imaging for three-dimensional visualization, object detection, and segmentation

Yi, Faliu; Jeong, Ongee; Moon, Inkyu; Javidi, Bahram

doi:10.1016/j.optlaseng.2021.106695

Cited by 10 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a recent study [29], a deep learning integral imaging system was proposed that can reconstruct a 3D object without dealing with the out-of-focus (blurred) areas that occur in the Integral-Imaging computationally reconstructed depth planes. Targets in the scene are first detected and segmented in the 2D elemental images using a pre-trained Mask R-CNN.…”

Section: Previous Related Workmentioning

confidence: 99%

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

Kadosh

Yitzhaky

2023

Sensors

View full text Add to dashboard Cite

This study aims to achieve accurate three-dimensional (3D) localization of multiple objects in a complicated scene using passive imaging. It is challenging, as it requires accurate localization of the objects in all three dimensions given recorded 2D images. An integral imaging system captures the scene from multiple angles and is able to computationally produce blur-based depth information about the objects in the scene. We propose a method to detect and segment objects in a 3D space using integral-imaging data obtained by a video camera array. Using objects’ two-dimensional regions detected via deep learning, we employ local computational integral imaging in detected objects’ depth tubes to estimate the depth positions of the objects along the viewing axis. This method analyzes object-based blurring characteristics in the 3D environment efficiently. Our camera array produces an array of multiple-view videos of the scene, called elemental videos. Thus, the proposed 3D object detection applied to the video frames allows for 3D tracking of the objects with knowledge of their depth positions along the video. Results show successful 3D object detection with depth localization in a real-life scene based on passive integral imaging. Such outcomes have not been obtained in previous studies using integral imaging; mainly, the proposed method outperforms them in its ability to detect the depth locations of objects that are in close proximity to each other, regardless of the object size. This study may contribute when robust 3D object localization is desired with passive imaging, but it requires a camera or lens array imaging apparatus.

show abstract

Section: Previous Related Workmentioning

confidence: 99%

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

Kadosh

Yitzhaky

2023

Sensors

View full text Add to dashboard Cite

show abstract

Section: Designmentioning

confidence: 99%

“…A software platform based on machine-learning architecture and driven by AI principles that mimic human intelligence to learn not only from data fed into the system, but also real-time data is the ultimate goal of the AAF. These systems could improve target recognition and tracking capabilities to analyse data from sensors and other sources (Yi et al, 2021), and make decisions about when and how to engage targets. The system should be able to accomplish the following basic functions: authenticate whether the person holding the gun is authorized to use it, and if not, lock the weapon while sending out an unauthorized use warning to the relevant authorities; track the position of the gun using GPS, gyroscopic data and Bluetooth; record the basic characteristics, body position, actions and non-verbal cues of the target; measure the height of the target, the distance from the shooter and the trajectory of projectiles (whether smart ammunition or traditional); record, detect, identify and label every object in its field of view; stitch together 360-degree images and video inside the gun to prevent tampering; use advanced threat-detection algorithms, such as real-time emergency detection models built on machine-learning infrastructures; efficiently process data and determine outcomes with the ability to display information swiftly in a readable and understandable format; smartly integrate battery-saving features, such as sleep mode (when idle; holstered) and quick wake (drawn); monitor ammunition and alert when the firearm needs reloading. A conceptual drawing of an AAF is presented in Figure 1.…”

Section: Conceptualizing the Future Police Firearmmentioning

confidence: 99%

Going cyberpunk: Conceptualizing the smart(er) artificially intelligent firearm for policing's Utopian future

Chowdhury

2023

International Journal of Police Science & Management

View full text Add to dashboard Cite

As policing develops into a more professional, evidence-driven and technical endeavour, heightened public concern regarding organizational competence and police culture-related fallacies have become augmented, especially in the case of officer-involved shootings. Introduction of body-worn cameras, increased CCTV coverage, vehicle dashcams and the advent of social media, have provided avenues for investigation into misconduct, but institutional and individual failings such as racism, sexism and other forms of discrimination remain a concern. Technical innovations like smart guns, smart targeting and programmable projectiles have instigated conversations about traditional firearms and whether alternatives using cutting-edge technology could address some of these shortcomings. This article examines existing policing technologies, providing an overview of advanced computational and sensor systems, the risks and dangers of these mechanisms, as well as their potential benefits and drawbacks. It conceptualizes whether existing technologies can be transformed into a smarter, more efficient firearm, powered by artificial intelligence (AI). The premise of the AI-assisted firearm being the promise of a future in which unwanted outcomes in officer/citizen encounters can be counteracted through AI assisting in better decision-making. The article considers hardware and software, policy issues, associated risks and potential advantages of the firearms system, providing a wider perspective on the increasing use of computational technologies in policing practice, and highlighting areas for further research and discussion.

show abstract

“…Target detection based on deep learning has been a research hot spot over the past few years and has aroused wide attention, 1 and relevant technologies have been applied to face recognition, 2 car driverless, 3,4 pedestrian tracking, 5 intelligent transportation, 6 and other fields. Target detection 7 technology has been found as a key technology of Internet of Things in the livestock industry, which is a vital means to monitor the location, number, health status, as well as whether the livestock are in estrus in real time; this technology also lays a basis for achieving intelligent pasture.…”

Section: Introductionmentioning

confidence: 99%

Enhanced lightweight deep network for efficient livestock detection in grazing areas

Du,

Qi,

Zhu

et al. 2024

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

There are problems in the special pastoral environment, including large changes in target size and serious interference from light and environmental factors. To solve the above problems, an enhanced YOLOv4-tiny target detection network is proposed in this study. This network first solves the problem of livestock size fluctuation in pastoral areas, uses a pyramid network with multiscale feature fusion, and considers shallow local detail features and deep semantic information. Subsequently, a novel compound multichannel attention mechanism is proposed to increase the accuracy of the target detection network for the pastoral environment. The problem of poor accuracy of target detection network is solved. The algorithm is ported to Jetson AGX embedded platform for validation to examine the real-time performance of the algorithm. As revealed by the experimental results, enhanced YOLOv4-tiny achieves 89.77% detection accuracy and 30 frames/second detection speed, which increases the average detection accuracy by 11.67% compared with the conventional YOLOv4-tiny while maintaining almost the same detection rate.

show abstract

Deep learning integral imaging for three-dimensional visualization, object detection, and segmentation

Cited by 10 publications

References 33 publications

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video

Going cyberpunk: Conceptualizing the smart(er) artificially intelligent firearm for policing's Utopian future

Enhanced lightweight deep network for efficient livestock detection in grazing areas

Contact Info

Product

Resources

About