Multimodal registration across 3D point clouds and CT-volumes

Saiti, Evdokia; Theoharis, T.

doi:10.1016/j.cag.2022.06.012

Cited by 9 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, they employed MeVisLab software [68] to visualize the inner and outer models along with the DICOM images simultaneously. In recent developments, automatic registration procedures utilizing deep networks have been proposed for aligning 3D point clouds and micro-CT 3D volumes of the same object [69].…”

Section: Virtual Animation Of the Body: Simulation Of The Antemortem ...mentioning

confidence: 99%

A Virtual, 3D Multimodal Approach to Victim and Crime Scene Reconstruction

Villa,

Lynnerup,

Jacobsen

2023

Diagnostics

View full text Add to dashboard Cite

In the last two decades, forensic pathology and crime scene investigations have seen a rapid increase in examination tools due to the implementation of several imaging techniques, e.g., CT and MR scanning, surface scanning and photogrammetry. These tools encompass relatively simple visualization tools to powerful instruments for performing virtual 3D crime scene reconstructions. A multi-modality and multiscale approach to a crime scene, where 3D models of victims and the crime scene are combined, offers several advantages. A permanent documentation of all evidence in a single 3D environment can be used during the investigation phases (e.g., for testing hypotheses) or during the court procedures (e.g., to visualize the scene and the victim in a more intuitive manner). Advanced computational approaches to understand what might have happened during a crime can also be applied by, e.g., performing a virtual animation of the victim in the actual context, which can provide important information about possible dynamics during the event. Here, we present an overview of the different techniques and modalities used in forensic pathology in conjunction with crime scene investigations. Based on our experiences, the advantages and challenges of an image-based multi-modality approach will be discussed, including how their use may introduce new visualization modalities in court, e.g., virtual reality (VR) and 3D printing. Finally, considerations about future directions in research will be mentioned.

show abstract

Section: Virtual Animation Of the Body: Simulation Of The Antemortem ...mentioning

confidence: 99%

A Virtual, 3D Multimodal Approach to Victim and Crime Scene Reconstruction

Villa,

Lynnerup,

Jacobsen

2023

Diagnostics

View full text Add to dashboard Cite

show abstract

“…Uni 1-to-1 Coarse-f. Classical Bones (Saiti et al, 2022) Multimodal CT Uni 1-to-1 1 Supervised (Santarossa et al, 2022) Multimodal IR-FAF/OCT Uni 1-to-1 1 Classical Eye (Schmidt et al, 2022) Uni 1-to-1 Coarse-f. Unsupervised Veins (Su et al, 2021) Unimodal CT/MR Uni 1-to-1 Classical (Terpstra et al, Unimodal greyscale Uni 1-to-1 Coarse-f. Classical Brain (Yang et al, 2022) Multimodal MR Uni 1-to-1 1 Unsupervised Prostate (Ye et al, 2021) Unimodal MR Bi 1-to-1 1 Unsupervised Heart (Ying et al, 2022) Unimodal MR Uni 1-to-1 1 Classical Breast (Zhang, G. et al, 2021) Unimodal Uni 1-to-1 Pyramid Unsupervised Brain Unimodal MR Uni 1-to-1 Pyramid Unsupervised Brain (Zhu et al, 2021) Unimodal MR Uni 1-to-1 Pyramid Unsupervised Head…”

Section: Medical Applicationsmentioning

confidence: 99%

Deep Learning in Medical Image Registration: Introduction and Survey

Hammoudeh,

Dupont

2023

Preprint

View full text Add to dashboard Cite

Image registration (IR) is a process that deforms images to align them with respect to a reference space, making it easier for medical practitioners to examine various medical images in a standardized reference frame, such as having the same rotation and scale. This document introduces image registration using a simple numeric example. It provides a definition of image registration along with a space-oriented symbolic representation. This review covers various aspects of image transformations, including affine, deformable, invertible, and bidirectional transformations, as well as medical image registration algorithms such as Voxelmorph, Demons, SyN, Iterative Closest Point, and SynthMorph. It also explores atlas-based registration and multistage image registration techniques, including coarse-fine and pyramid approaches. Furthermore, this survey paper discusses medical image registration taxonomies, datasets, evaluation measures, such as correlation-based metrics, segmentation-based metrics, processing time, and model size. It also explores applications in image-guided surgery, motion tracking, and tumor diagnosis. Finally, the document addresses future research directions, including the further development of transformers.

show abstract

“…Point cloud registration has been actively studied in computer vision and graphics [1][2][3][4][5][6], and most studies mainly focus on pairwise registration [7]. The primary objective of pairwise registration is to estimate the transformation parameters that align a source point cloud to a target point cloud.…”

Section: Introductionmentioning

confidence: 99%

“…To enhance the efficiency, stability, and robustness of multiple pairwise registrations, we introduce SGRTmreg, a new computational framework. Given a collection of point clouds, a source point cloud, and a target point cloud, the process for SGRTmreg to achieve registration unfolds in three steps: (1) Selecting a point cloud similar to the source from the collection based on graph structure, coordinates, node importance, and normal vectors via a searching scheme. (2) Learning regressors from the source using the Graph-based Reweighted Discriminative Optimization (GRDO) method by registering the source to the target.…”

Section: Introductionmentioning

confidence: 99%

SGRTmreg: A Learning-Based Optimization Framework for Multiple Pairwise Registrations

Zhao,

Deng,

Gao

et al. 2024

Sensors

View full text Add to dashboard Cite

Point cloud registration is a fundamental task in computer vision and graphics, which is widely used in 3D reconstruction, object tracking, and atlas reconstruction. Learning-based optimization and deep learning methods have been widely developed in pairwise registration due to their own distinctive advantages. Deep learning methods offer greater flexibility and enable registering unseen point clouds that are not trained. Learning-based optimization methods exhibit enhanced robustness and stability when handling registration under various perturbations, such as noise, outliers, and occlusions. To leverage the strengths of both approaches to achieve a less time-consuming, robust, and stable registration for multiple instances, we propose a novel computational framework called SGRTmreg for multiple pairwise registrations in this paper. The SGRTmreg framework utilizes three components—a Searching scheme, a learning-based optimization method called Graph-based Reweighted discriminative optimization (GRDO), and a Transfer module to achieve multi-instance point cloud registration.Given a collection of instances to be matched, a template as a target point cloud, and an instance as a source point cloud, the searching scheme selects one point cloud from the collection that closely resembles the source. GRDO then learns a sequence of regressors by aligning the source to the target, while the transfer module stores and applies the learned regressors to align the selected point cloud to the target and estimate the transformation of the selected point cloud. In short, SGRTmreg harnesses a shared sequence of regressors to register multiple point clouds to a target point cloud. We conduct extensive registration experiments on various datasets to evaluate the proposed framework. The experimental results demonstrate that SGRTmreg achieves multiple pairwise registrations with higher accuracy, robustness, and stability than the state-of-the-art deep learning and traditional registration methods.

show abstract

Multimodal registration across 3D point clouds and CT-volumes

Cited by 9 publications

References 32 publications

A Virtual, 3D Multimodal Approach to Victim and Crime Scene Reconstruction

A Virtual, 3D Multimodal Approach to Victim and Crime Scene Reconstruction

Deep Learning in Medical Image Registration: Introduction and Survey

SGRTmreg: A Learning-Based Optimization Framework for Multiple Pairwise Registrations

Contact Info

Product

Resources

About