Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling

Tao, Yunzhe; Sun, Qi; Du, Qiang; Liu, Wei

doi:10.48550/arxiv.1806.00681

Cited by 5 publications

(5 citation statements)

References 17 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Nonlocal block [31] can capture remote dependencies more robustly and flexibly to help the deep network better integrate Nonlocal information. Some Nonlocal network modules currently proposed are NL [31], A2 [32], NS [33], CC [34], CGNL [35], SNL [36], etc. Chen et al [32] propose the Double Attention block, which first collects the features in the entire space and then assigns them back to each location.…”

Section: A Related Workmentioning

confidence: 99%

“…Huang et al [34] proposed a lightweight Nonlocal block called an interleaved attention block, which decomposes the positional attention of NL into conterminously column-wise and row-wise attention. In order to improve the stability of the NL block, Tao et al [33] proposed using the Laplacian of the incidence matrix as an attention map, and the Nonlocal stage (NS) module can follow the diffusion characteristics. Zhu et al [36] proposed The SNL (Spectral Nonlocal Block) module, which symmetrically processes the attention feature block.…”

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Segmentation and Measurement of Superalloy Microstructure Based on Improved Nonlocal Block

Zhang

Yao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The microstructure of superalloy materials has a decisive impact on its service performance. When preparing the material and photographing the microstructure, different depths of metallography perpendicular to the cut plane appear in the microstructure image. These metallographic features are major factors causing inaccurate segmentation. Aiming at the problems of traditional image processing methods, such as large noise and poor robustness of the edge extraction, the deep learning method is introduced. The receptive field of the traditional convolutional neural network method is too local to obtain remote dependencies, and dense feature extraction also brings problems such as excessive noise.To address the problem of non-tangential metallographic information in microstructure images that is not easily distinguishable. We consider how to combine and utilize the feature maps of the intermediate process as effectively as possible, and find that reusing affinity matrices from different angles and stacking them can improve the overall effect, and can solve the problem of inaccurate segmentation of metallographic images at different depths. We propose to improve and optimize the non-local attention module and further combine the module with the UNet network to form a new improved SNL-Unet image segmentation structure, which significantly raises the accuracy and efficiency of image segmentation, Additionally, we measure the characteristic parameters such as volume fraction, average thickness and the degree of rafting. The code for this paper will be available at github.com/ustbjdl1021/improved-snl-unet.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Segmentation and Measurement of Superalloy Microstructure Based on Improved Nonlocal Block

Zhang

Yao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…If f q(l−1) j incorporates complementary information or more significant cues compared to the current probe feature f q(l−1) i , then our redundency-aware attention scheme will eliminate the information from the inferior f q(l−1) i and emphasis the more discriminative f q(l−1) j . Compared to the original non-local network which uses only f q(l−1) j [19], our formulation can be more similar to the diffusion maps [58], graph Laplacian [59] and non-local image processing [60]. All of them are non-local analogues [61] of local diffusions, which are expected to be more stable than its original non-local counterpart [19] due to the nature of its inherit Hilbert-Schmidt operator [61].…”

Section: Redundancy-aware Self-attentionmentioning

confidence: 99%

Interpreting Depression From Question-Wise Long-Term Video Recording of SDS Evaluation

Xie

Liang

et al. 2022

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Self-Rating Depression Scale (SDS) questionnaire has frequently been used for efficient depression preliminary screening. However, the uncontrollable self-administered measure can be easily affected by insouciantly or deceptively answering, and producing the different results with the clinicianadministered Hamilton Depression Rating Scale (HDRS) and the final diagnosis. Clinically, facial expression (FE) and actions play a vital role in clinician-administered evaluation, while FE and action are underexplored for self-administered evaluations. In this work, we collect a novel dataset of 200 subjects to evidence the validity of self-rating questionnaires with their corresponding question-wise video recording. To automatically interpret depression from the SDS evaluation and the paired video, we propose an end-to-end hierarchical framework for the long-term variablelength video, which is also conditioned on the questionnaire results and the answering time. Specifically, we resort to a hierarchical model which utilizes a 3D CNN for local temporal pattern exploration and a redundancy-aware self-attention (RAS) scheme for question-wise global feature aggregation. Targeting for the redundant long-term FE video processing, our RAS is able to effectively exploit the correlations of each video clip within a question set to emphasize the discriminative information and eliminate the redundancy based on feature pair-wise affinity. Then, the question-wise video feature is concatenated with the questionnaire scores for final depression detection. Our thorough evaluations also show the validity of fusing SDS evaluation and its video recording, and the superiority of our framework to the conventional state-of-the-art temporal modeling methods.

show abstract

“…If x l−1 n j incorporates complementary information and has better imaging/content quality compared to x l−1 n i , then RSA will erase some information of the inferior x l−1 n i and replaces it by the more discriminative feature representation x l−1 n j . Compared to the method of using only x l−1 n j [75], our setting shares more common features with diffusion maps [70], graph Laplacian [9] and non-local image processing [17]. All of them are non-local analogues [12] of local diffusions, which are expected to be more stable than its original non-local counterpart [75] due to the nature of its inherit Hilbert-Schmidt operator [12].…”

Section: Global 3d Encoding With Arbitrary Number Of Appearance Descr...mentioning

confidence: 99%

AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation

Liu,

Che,

et al. 2020

Preprint

View full text Add to dashboard Cite

This paper targets on learning-based novel view synthesis from a single or limited 2D images without the pose supervision. In the viewer-centered coordinates, we construct an end-to-end trainable conditional variational framework to disentangle the unsupervisely learned relative-pose/rotation and implicit global 3D representation (shape, texture and the origin of viewer-centered coordinates, etc.). The global appearance of the 3D object is given by several appearance-describing images taken from any number of viewpoints. Our spatial correlation module extracts a global 3D representation from the appearance-describing images in a permutation invariant manner. Our system can achieve implicitly 3D understanding without explicitly 3D reconstruction. With an unsupervisely learned viewer-centered relative-pose/rotation code, the decoder can hallucinate the novel view continuously by sampling the relative-pose in a prior distribution. In various applications, we demonstrate that our model can achieve comparable or even better results than pose/3D model-supervised learning-based novel view synthesis (NVS) methods with any number of input views.

show abstract

Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling

Cited by 5 publications

References 17 publications

Segmentation and Measurement of Superalloy Microstructure Based on Improved Nonlocal Block

Segmentation and Measurement of Superalloy Microstructure Based on Improved Nonlocal Block

Interpreting Depression From Question-Wise Long-Term Video Recording of SDS Evaluation

AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation

Contact Info

Product

Resources

About