Artificial Intelligence-Based Algorithms in Medical Image Scan Segmentation and Intelligent Visual Content Generation—A Concise Overview

Reconstructing large-scale scenes using Neural Radiance Fields (NeRFs) is a research hotspot in 3D computer vision. Existing MLP (multi-layer perception)-based methods often suffer from issues of underfitting and a lack of fine details in rendering large-scale scenes. Popular solutions are to divide the scene into small areas for separate modeling or to increase the layer scale of the MLP network. However, the subsequent problem is that the training cost increases. Moreover, reconstructing large scenes, unlike object-scale reconstruction, involves a geometrically considerable increase in the quantity of view data if the prior information of the scene is not effectively utilized. In this paper, we propose an innovative method named MM-NeRF, which integrates efficient hybrid features into the NeRF framework to enhance the reconstruction of large-scale scenes. We propose employing a dual-branch feature capture structure, comprising a multi-resolution 3D hash grid feature branch and a multi-view 2D prior feature branch. The 3D hash grid feature models geometric details, while the 2D prior feature supplements local texture information. Our experimental results show that such integration is sufficient to render realistic novel views with fine details, forming a more accurate geometric representation. Compared with representative methods in the field, our method significantly improves the PSNR (Peak Signal-to-Noise Ratio) by approximately 5%. This remarkable progress underscores the outstanding contribution of our method in the field of large-scene radiance field reconstruction.

Section: Discussionmentioning

confidence: 99%

“…NeRF [5] 21 1 The upward arrow indicates that the higher the metric, the better. The downward arrow indicates that the lower the metric, the better.…”

Section: Aim (Avg) Yrt (Avg) Mac (Avg) Gxm (Avg) Psnr↑ 1 Lpips↓ Ssim↑...mentioning

confidence: 99%

See 1 more Smart Citation

MM-NeRF: Large-Scale Scene Representation with Multi-Resolution Hash Grid and Multi-View Priors Features

Dong,

Chen,

Wang

et al. 2024

“…MRI often excels in contrast resolution compared to other diagnostics methods. Therefore, for medical image segmentation (mostly semantic segmentation), different types of neural networks are applied [84], see also Table 2. The basic concept of AI application in cardiology is presented in Figure 2.…”

Section: Artificial Intelligence-based Support In Cardiologymentioning

confidence: 99%

Cardiac Healthcare Digital Twins Supported by Artificial Intelligence-Based Algorithms and Extended Reality—A Systematic Review

Rudnicka,

Proniewska,

Perkins

et al. 2024

Self Cite

Recently, significant efforts have been made to create Health Digital Twins (HDTs), Digital Twins for clinical applications. Heart modeling is one of the fastest-growing fields, which favors the effective application of HDTs. The clinical application of HDTs will be increasingly widespread in the future of healthcare services and has huge potential to form part of mainstream medicine. However, it requires the development of both models and algorithms for the analysis of medical data, and advances in Artificial Intelligence (AI)-based algorithms have already revolutionized image segmentation processes. Precise segmentation of lesions may contribute to an efficient diagnostics process and a more effective selection of targeted therapy. In this systematic review, a brief overview of recent achievements in HDT technologies in the field of cardiology, including interventional cardiology, was conducted. HDTs were studied taking into account the application of Extended Reality (XR) and AI, as well as data security, technical risks, and ethics-related issues. Special emphasis was put on automatic segmentation issues. In this study, 253 literature sources were taken into account. It appears that improvements in data processing will focus on automatic segmentation of medical imaging in addition to three-dimensional (3D) pictures to reconstruct the anatomy of the heart and torso that can be displayed in XR-based devices. This will contribute to the development of effective heart diagnostics. The combination of AI, XR, and an HDT-based solution will help to avoid technical errors and serve as a universal methodology in the development of personalized cardiology. Additionally, we describe potential applications, limitations, and further research directions.

“…Deep convolutional neural networks (DCNNs) have realized great success in many computer vision tasks, such as object identification [1][2][3], object detection [4][5][6] and image segmentation [7][8][9]. The design of DCNNs is becoming increasingly intricate, which is accompanied by a simultaneous enhancement in their performance.…”

Section: Introductionmentioning

confidence: 99%

Towards Super Compressed Neural Networks for Object Identification: Quantized Low-Rank Tensor Decomposition with Self-Attention

Liu,

Wang,

et al. 2024

Deep convolutional neural networks have a large number of parameters and require a significant number of floating-point operations during computation, which limits their deployment in situations where the storage space is limited and computational resources are insufficient, such as in mobile phones and small robots. Many network compression methods have been proposed to address the aforementioned issues, including pruning, low-rank decomposition, quantization, etc. However, these methods typically fail to achieve a significant compression ratio in terms of the parameter count. Even when high compression rates are achieved, the network’s performance is often significantly deteriorated, making it difficult to perform tasks effectively. In this study, we propose a more compact representation for neural networks, named Quantized Low-Rank Tensor Decomposition (QLTD), to super compress deep convolutional neural networks. Firstly, we employed low-rank Tucker decomposition to compress the pre-trained weights. Subsequently, to further exploit redundancies within the core tensor and factor matrices obtained through Tucker decomposition, we employed vector quantization to partition and cluster the weights. Simultaneously, we introduced a self-attention module for each core tensor and factor matrix to enhance the training responsiveness in critical regions. The object identification results in the CIFAR10 experiment showed that QLTD achieved a compression ratio of 35.43×, with less than 1% loss in accuracy and a compression ratio of 90.61×, with less than a 2% loss in accuracy. QLTD was able to achieve a significant compression ratio in terms of the parameter count and realize a good balance between compressing parameters and maintaining identification accuracy.