Patch-Based Nonlinear Image Registration for Gigapixel Whole Slide Images

Abstract

Introduction/ BackgroundImage Registration of whole slide histology images allows the fusion of fine-grained information like different immunohistochemical stains from neighboring tissue slides. Traditionally, pathologists fuse this information by looking subsequently at one slide at a time. If the slides are digitized and accurately aligned at cell-level, automatic analysis can be used to ease the pathologist's work. However, the size of those images exceeds the memory capacity of regular computers, preventing the application of established image registration methods at a high magnification. AimsAn accurate and automatic alignment helps the pathologist to analyze the combination of different antibodies without memorizing multiple slides. For some applications, cell-level accuracy is needed. This also enables automatic image analysis to take advantage of multislide information. MethodsWe extend available registration methods by using image data at fine spatial resolution. However, this data is not simultaneously globally available due to the computer's memory restrictions. Typical approaches either reduce the amount of data to be processed by downsampling or partition the data into smaller chunks to be processed separately. We combine the patch based approach with an elastic deformation model to obtain a global registration result. Our method first registers the complete images on a low resolution with a nonlinear deformation model and later refines this result on patches by using a second nonlinear registration on each patch. The deformation information on the individual patches can be contradictory and needs to be combined into one global model. As an extension to our previous work, the global solution is computed by minimizing an energy function that preserves the patch-wise deformation and establishes global smoothness. The NGF distance measure is used to handle multi-stain images. ResultsThe method will be applied to whole slide image pairs. The alignment of corresponding structures will be measured by comparing manual segmentations from neighboring slides. First results show an improvement of the registration accuracy compared to the low-resolution nonlinear registration.

PathFlow-MixMatch for Whole Slide Image Registration: An Investigation of a Segment-Based Scalable Image Registration Method

“…Additionally, there are many registration loss functions that can be utilized to improve registration quality, such as cross-correlational, structural similarity, and mutual information losses, which may warrant further study. Future works will focus on improvements to shape matching 37,38 , microarchitectural alignment on restained slides, finetuning registrations through patched based alignment 39,40 , apply distributed computing techniques for high throughput processing 41 and investigate the impact of this histology registration on 3D macroarchitecture reconstruction of tissue and on stain translation techniques through Pix2Pix 10,42 43 44 .…”

Integration of 3D multimodal imaging data of a head and neck cancer and advanced feature recognition

“…Using a medium magnification level we controled the computational load when registering the whole slide images. However, this is an active research field and as published in [33] we are able to base the registration on the full resolution scanned images by registering them patch-wise and achieve a more accurate alignment. Performance is an issue that needs to be addressed further.…”

“…Among these are simple manual inspection and manual registration or so-called moment based methods such as principle component analysis (PCA) [20,32], which is used in this study. See [33] for a detailed description.…”

Robust Hierarchical Density Estimation and Regression for Re-stained Histological Whole Slide Image Co-registration

“…From a computational perspective, this means the re-staining scenario is more suitable for rigid image registration, while serial sectioning is more suitable for non-rigid image registration. Most of the previous work in the WSI registration field has been focused on aligning serial sections of tissues for cross-sectional observations (e.g., 3-dimensional tissue visualization) [1][2][3][4][5]. Methods such as those proposed by Mueller et al [6] use a deformable multi-modal WSI registration technique are well suited for serial section registration.…”