ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate

Shao, Wei; Banh, Linda; Kunder, Christian A.; Fan, Richard E.; Soerensen, Simon John Christoph; Wang, Jeffrey B.; Teslovich, Nikola C.; Madhuripan, Nikhil; Jawahar, A.; Ghanouni, Pejman; Brooks, James D.; Sonn, Geoffrey A.; Rusu, Mirabela

doi:10.1016/j.media.2020.101919

Cited by 51 publications

(46 citation statements)

References 37 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most studies have primarily investigated DL methods regarding diagnosis support, e.g., regarding the detection of cancer or analysis of lung tissue [8,18,22]. It was shown in recent studies on prostate MRI that DL can be applied for cancer detection and classification and also for registration with histopathological images [10,11,23,24]. Wang et al demonstrated that DL could also be applied for the omission of endorectal coils in mpMRI without compromising the image quality regarding noise [25].…”

Section: Discussionmentioning

confidence: 99%

Accelerated T2-Weighted TSE Imaging of the Prostate Using Deep Learning Image Reconstruction: A Prospective Comparison with Standard T2-Weighted TSE Imaging

Gassenmaier

Afat

Nickel

et al. 2021

Cancers

View full text Add to dashboard Cite

Multiparametric MRI (mpMRI) of the prostate has become the standard of care in prostate cancer evaluation. Recently, deep learning image reconstruction (DLR) methods have been introduced with promising results regarding scan acceleration. Therefore, the aim of this study was to investigate the impact of deep learning image reconstruction (DLR) in a shortened acquisition process of T2-weighted TSE imaging, regarding the image quality and diagnostic confidence, as well as PI-RADS and T2 scoring, as compared to standard T2 TSE imaging. Sixty patients undergoing 3T mpMRI for the evaluation of prostate cancer were prospectively enrolled in this institutional review board-approved study between October 2020 and March 2021. After the acquisition of standard T2 TSE imaging (T2S), the novel T2 TSE sequence with DLR (T2DLR) was applied in three planes. Overall, the acquisition time for T2S resulted in 10:21 min versus 3:50 min for T2DLR. The image evaluation was performed by two radiologists independently using a Likert scale ranging from 1–4 (4 best) applying the following criteria: noise levels, artifacts, overall image quality, diagnostic confidence, and lesion conspicuity. Additionally, T2 and PI-RADS scoring were performed. The mean patient age was 69 ± 9 years (range, 49–85 years). The noise levels and the extent of the artifacts were evaluated to be significantly improved in T2DLR versus T2S by both readers (p < 0.05). Overall image quality was also evaluated to be superior in T2DLR versus T2S in all three acquisition planes (p = 0.005–<0.001). Both readers evaluated the item lesion conspicuity to be superior in T2DLR with a median of 4 versus a median of 3 in T2S (p = 0.001 and <0.001, respectively). T2-weighted TSE imaging of the prostate in three planes with an acquisition time reduction of more than 60% including DLR is feasible with a significant improvement of image quality.

show abstract

Section: Discussionmentioning

confidence: 99%

Accelerated T2-Weighted TSE Imaging of the Prostate Using Deep Learning Image Reconstruction: A Prospective Comparison with Standard T2-Weighted TSE Imaging

Gassenmaier

Afat

Nickel

et al. 2021

Cancers

View full text Add to dashboard Cite

show abstract

“…Other studies utilized a similar coarse-to-fine strategy for lung images and achieved high registration accuracies (38,115,116). Shen et al and Shao et al also integrated affine registration into their DL models and successfully addressed large deformations in prostate and knee images (118,119).…”

Section: Similarity Metric-based Registrationmentioning

confidence: 99%

A review of deep learning-based three-dimensional medical image registration methods

Xiao¹,

Teng²,

Liu³

et al. 2021

Quant Imaging Med Surg

View full text Add to dashboard Cite

Medical image registration is a vital component of many medical procedures, such as imageguided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research.The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.

show abstract

“…An alternative approach to labeling cancer location on MRI is to perform automatic registration of preoperative MRI and digital histopathology images from patients undergoing radical prostatectomy. 22 , 23 , 24 , 25 , 26 , 27 Labels obtained from automatic registration are more accurate than radiologist labels since they do not depend on human interpretation of MRI and allow for the full extent of lesions found on histopathology to be mapped on MRI, including cancers that are invisible or hardly visible. Figure 1 illustrates how cancer labels mapped from histopathology images onto MRI typically extend beyond the radiologist annotation and often include cancers that were not detected by the radiologist.…”

Section: Introductionmentioning

confidence: 99%

Automated detection of aggressive and indolent prostate cancer on magnetic resonance imaging

et al. 2021

Self Cite

View full text Add to dashboard Cite

Purpose While multi‐parametric magnetic resonance imaging (MRI) shows great promise in assisting with prostate cancer diagnosis and localization, subtle differences in appearance between cancer and normal tissue lead to many false positive and false negative interpretations by radiologists. We sought to automatically detect aggressive cancer (Gleason pattern ≥ 4) and indolent cancer (Gleason pattern 3) on a per‐pixel basis on MRI to facilitate the targeting of aggressive cancer during biopsy. Methods We created the Stanford Prostate Cancer Network (SPCNet), a convolutional neural network model, trained to distinguish between aggressive cancer, indolent cancer, and normal tissue on MRI. Ground truth cancer labels were obtained by registering MRI with whole‐mount digital histopathology images from patients who underwent radical prostatectomy. Before registration, these histopathology images were automatically annotated to show Gleason patterns on a per‐pixel basis. The model was trained on data from 78 patients who underwent radical prostatectomy and 24 patients without prostate cancer. The model was evaluated on a pixel and lesion level in 322 patients, including six patients with normal MRI and no cancer, 23 patients who underwent radical prostatectomy, and 293 patients who underwent biopsy. Moreover, we assessed the ability of our model to detect clinically significant cancer (lesions with an aggressive component) and compared it to the performance of radiologists. Results Our model detected clinically significant lesions with an area under the receiver operator characteristics curve of 0.75 for radical prostatectomy patients and 0.80 for biopsy patients. Moreover, the model detected up to 18% of lesions missed by radiologists, and overall had a sensitivity and specificity that approached that of radiologists in detecting clinically significant cancer. Conclusions Our SPCNet model accurately detected aggressive prostate cancer. Its performance approached that of radiologists, and it helped identify lesions otherwise missed by radiologists. Our model has the potential to assist physicians in specifically targeting the aggressive component of prostate cancers during biopsy or focal treatment.

show abstract

ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate

Cited by 51 publications

References 37 publications

Accelerated T2-Weighted TSE Imaging of the Prostate Using Deep Learning Image Reconstruction: A Prospective Comparison with Standard T2-Weighted TSE Imaging

Accelerated T2-Weighted TSE Imaging of the Prostate Using Deep Learning Image Reconstruction: A Prospective Comparison with Standard T2-Weighted TSE Imaging

A review of deep learning-based three-dimensional medical image registration methods

Automated detection of aggressive and indolent prostate cancer on magnetic resonance imaging

Contact Info

Product

Resources

About