Learnable graph convolutional network and feature fusion for multi-view learning

Chen, Zhaoliang; Fu, Lele; Yao, Jie; Guo, Wenzhong; Plant, Claudia; Wang, Shiping

doi:10.1016/j.inffus.2023.02.013

Cited by 66 publications

(20 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is worth noting that optimizing (7) jointly generates the view graph matrix S v and the soft label matrix F v . In our work, the latter is used as a flexible linear projection of the features X v .…”

Section: View-based Graph Learning S Vmentioning

confidence: 99%

See 1 more Smart Citation

Multi-view Semi-supervised Learning with a Unified Graph Convolutional Network

Dornaika,

2024

Preprint

View full text Add to dashboard Cite

In recent years, the proliferation of data-driven applications across diverse fields has sparked a surge in interest in applying semi-supervised learning to graphs. This surge is driven by the widespread use of graph data structures in real-world scenarios, such as interpersonal relationships in social networks, user behavior graphs in recommender systems, and molecular interaction networks in bioinformatics. However, certain data types, like images, pose challenges due to the lack of explicit graph structures and the presence of multiple view description methods. These complexities hinder the direct application of traditional semi-supervised learning techniques to graphs. As a result, researchers are exploring the integration of semi-supervised learning with deep learning to capitalize on its rich information and improve model performance. Effectively combining graph structures with multi-view data remains a daunting task that requires further investigation. This paper presents the Linear projection Fused Graph-based Semi-supervised Classification (LFGSC) method designed specifically for multi-view data, building upon the Graph Convolutional Network (GCN) architecture. Initially, for each view, we employ a semi-supervised approach to simultaneously estimate the corresponding graph and flexible linear data representations in a low-dimensional feature space. Subsequently, an adaptive and unified graph is constructed, followed by the use of a fully connected network to fuse the projected features and reduce dimensionality. Finally, the fused features and graph are fed into a GCN for semi-supervised classification. During training, the model incorporates cross-entropy loss, manifold regularization loss, graph auto-encoder loss, and supervised contrastive loss. Leveraging linear transformation significantly reduces the input feature dimensions for GCN, resulting in high accuracy while minimizing computational overhead. Additionally, experimental results on various benchmarked multiview image datasets demonstrate the superiority of LFGSC over existing semi-supervised learning methods for multi-view scenarios. Source code: https://github.com/BiJingjun/LFGSC.

show abstract

Section: View-based Graph Learning S Vmentioning

confidence: 99%

“…As mentioned before, using the graph construction technique (7), we also derive V soft label predictions F v , v = 1, ..., V . Then, we can compute an additional view X V +1 that includes all V views by a convex combination of the projected features F v :…”

Section: Additional View X V +1mentioning

confidence: 99%

Multi-view Semi-supervised Learning with a Unified Graph Convolutional Network

Dornaika,

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Besides, GCN has also been used for hyperspectral image classification [49] and instance segmentation [50]. Although several works have applied GCN to deal with multi-feature fusion, such as multiview learning [51] and hyperspectral image classification [52], the former mainly fused the features by a fully-connected neural network in the shared latent space of the original multi-view representations, and the latter combined CNN with GCN to extract the small-scale and large-scale spectral-spatial features in pixel-or super-pixel nodes. However, they all failed to consider the NLss of intra-and inter-modals of multi-modal features.…”

Section: Graph Convolutional Network For Vision Tasksmentioning

confidence: 99%

TFIV: Multigrained Token Fusion for Infrared and Visible Image via Transformer

Li,

Yang,

Bai

et al. 2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Infrared and visible image fusion aims to extract complementary features to synthesize a single fused image. Many methods employ convolutional neural networks (CNNs) to extract local features due to its translation invariance and locality. However, CNNs fail to consider the image's non-local self-similarity (NLss), though it can expand the receptive field by pooling operations, it still inevitably leads to information loss. In addition, the transformer structure extracts long-range dependence by considering the correlativity among all image patches, leading to information redundancy of such transformer-based methods. However, graph representation is more flexible than grid (CNN) or sequence (transformer structure) representation to address irregular objects, and graph can also construct the relationships among the spatially repeatable details or texture with far-space distance. Therefore, to address the above issues, it is significant to convert images into the graph space and thus adopt graph convolutional networks (GCNs) to extract NLss. This is because the graph can provide a fine structure to aggregate features and propagate information across the nearest vertices without introducing redundant information. Concretely, we implement a cascaded NLss extraction pattern to extract NLss of intra-and inter-modal by exploring interactions of different image pixels in intra-and inter-image positional distance. We commence by preforming GCNs on each intra-modal to aggregate features and propagate information to extract independent intra-modal NLss. Then, GCNs are performed on the concatenate intramodal NLss features of infrared and visible images, which can explore the cross-domain NLss of inter-modal to reconstruct the fused image. We progressively expand the kennel sizes and dilations to increase the receptive field of GCNs, when we extract intra-and inter-modal NLss. Ablation studies and extensive experiments illustrates the effectiveness and superiority of the proposed method on three datasets.

show abstract

“…For example, ξ consists of 102 400 samples, and w is set to 1024, which generates 100 samples. The dimension of each sample is [1,1024]. In order to mitigate the effect of random noise and ensure similarity between samples, samples of the same dimension need to be normalized.…”

Section: Data Preprocessingmentioning

confidence: 99%

“…Rolling bearings play a pivotal role in the operation of rotating machinery, exerting a profound influence on the dependability and security of mechanical equipment [1]. However, due to frequent exposure to adverse working conditions, bearings are susceptible to damage, which increases the risk of failures that could lead to accidents and significantly reduce equipment efficiency [2].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised rolling bearing fault diagnosis method across working conditions based on multiscale convolutional neural network

Fu,

Yu,

Zhan

et al. 2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

In practical engineering, the features of rolling bearing vibration signals often vary in distribution under different working conditions, and obtaining sample labels for target working conditions can be challenging. Therefore, a multi-scale depth subdomain adaptive convolutional neural network (MS-DSACNN) fault diagnosis method is proposed. The MS-DSACNN method is based on a multi-scale feature extraction method combined with an improved 1D-ConvNeXt architecture, which fuses low- and high-level features into multi-scale fault features. A channel attention mechanism is also introduced to adaptively assign weights to the fused multi-scale features. The local maximum mean discrepancy is constructed by extracting features and their corresponding source and pseudo-label inputs for the source and target working conditions, respectively, to construct the loss function for backpropagation. The MS-DSACNN model was evaluated in experiments with two different loads and one variable speed dataset. The experimental results indicate that MS-DSACNN has higher feature extraction capability and diagnostic accuracy than other unsupervised cross-condition fault diagnosis techniques. Additionally, the scenario where the signal captured by the sensor contains a considerable amount of noise in actual working conditions is considered.

show abstract

Learnable graph convolutional network and feature fusion for multi-view learning

Cited by 66 publications

References 33 publications

Multi-view Semi-supervised Learning with a Unified Graph Convolutional Network

Multi-view Semi-supervised Learning with a Unified Graph Convolutional Network

TFIV: Multigrained Token Fusion for Infrared and Visible Image via Transformer

Unsupervised rolling bearing fault diagnosis method across working conditions based on multiscale convolutional neural network

Contact Info

Product

Resources

About