Spatial Clockwork Recurrent Neural Network for Muscle Perimysium Segmentation

Abstract

Accurate segmentation of perimysium plays an important role in early diagnosis of many muscle diseases because many diseases contain different perimysium inflammation. However, it remains as a challenging task due to the complex appearance of the perymisum morphology and its ambiguity to the background area. The muscle perimysium also exhibits strong structure spanned in the entire tissue, which makes it difficult for current local patch-based methods to capture this long-range context information. In this paper, we propose a novel spatial clockwork recurrent neural network (spatial CW-RNN) to address those issues. Specifically, we split the entire image into a set of non-overlapping image patches, and the semantic dependencies among them are modeled by the proposed spatial CW-RNN. Our method directly takes the 2D structure of the image into consideration and is capable of encoding the context information of the entire image into the local representation of each patch. Meanwhile, we leverage on the structured regression to assign one prediction mask rather than a single class label to each local patch, which enables both efficient training and testing. We extensively test our method for perimysium segmentation using digitized muscle microscopy images. Experimental results demonstrate the superiority of the novel spatial CW-RNN over other existing state of the arts.

The Rise of Deep Learning in Radiology: An Overview of Recent Research

“…RNNs, known for their success in temporal data related applications and tasks that involve sequences, and hence massively successful in text-related and other sequence-related applications, have also been used sometimes, e.g., in organ segmentation, though not as widely as CNNs. 1) Some Applications of RNNs: Xie et al [44] used spatial "clockwork" RNN for medical segmentation in histopathology images. Zheng and Yi [45] tweaked the traditional "vanilla RNN" architecture by adding a "competitive layer model" to produce accurate results in brain MRI segmentation.…”

AIIMDs: An Integrated Framework of Automatic Idiopathic Inflammatory Myopathy Diagnosis for Muscle

“…Our recent work [21] has used recurrent neural networks ( RNNs ) to model the semantic information by considering the context information as chain structured data and successfully applied it to annotate perimysium in 2D muscle images. Specifically, this work presents a 2D spatial clockwork RNN ( SCW-RNN ), which is an extension to the chain structured clockwork RNN ( CW-RNN ) [22].…”

“…In this work, we present an SCW-RNN by extending the idea of CW-RNN to image domain for region annotation. Since the static images do not exhibit sequential information, we modify the CW-RNN such that at any time state it can receive information from its predecessors in both the dimensions of the image (see [21] for details).…”

Exploiting Interslice Correlation for MRI Prostate Image Segmentation, from Recursive Neural Networks Aspect

“…Bao and Chung [13] introduced a multiscale structured FCN model for brain MRI segmentation by capturing discriminative features from input patch. Other examples on introducing deep learning into biomedical image segmentation can be found in [14][15][16][17].…”

“…Motivated by the fact that the acquired MRI images typically have a high intraslice resolution and there exists a high spatial dependence between slices from the same patient, we utilize interslice as shape prior to guide the process of feature extraction and explore necessary information from interslice to alleviate information loss as shown in Figure 2. Besides the architecture of RNNs has superiority performances in modeling sequential data [17,20,21]. To improve the performance of prostate segmentation, in this paper, we propose a network, called UR-Net, which treats prostate slices as a data sequence, utilizing the intraslice contexts and features to assist segmentation.…”