Attention-based fusion network for human eye-fixation prediction in 3D images

Lv, Yiliang; Zhou, Wujie; Lei, Jingsheng; Lv, Ye; Luo, Ting

doi:10.1364/oe.27.034056

Cited by 15 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e CNN is used to extract representational image features, and the RNN is naturally suitable for the recognition problem of sequence data, so this network architecture is well suited for image-based text line recognition. Convolutional RNN [15] is a representative approach for this type of network architecture, which uses CNN networks to extract high-level semantic features of images, transforms the extracted features into feature sequences, and then, uses a bidirectional long-and short-term (BiLSTM) memory network [2,16]. is model uses a CNN network to extract high-level semantic features, converts the extracted features into feature sequences, and then, uses a BiLSTM network [17][18][19] to capture the contextual information in both directions before and after the sequences and uses a CTC (Connectionist Temporal Classifier) [20] to decode the sequence features to obtain the final text recognition results.…”

Section: Introductionmentioning

confidence: 99%

A Study of Spatial Attention and Squeeze Excitation Block Fusion Improved ResNet for Identifying Bank Notes

Huo

2021

Security and Communication Networks

View full text Add to dashboard Cite

Based on deep learning and digital image processing algorithms, we design and implement an accurate automatic recognition system for bank note text and propose an improved recognition method based on ResNet for the problems of difficult image text extraction and insufficient recognition accuracy. Firstly, a deep hyperparameterized convolution (DO-Conv) is used instead of the traditional convolution in the network to improve the recognition rate while reducing the model parameters. Then, the spatial attention model (SAM) and the squeezed excitation block (SE-Block) are fused and applied to a modified ResNet to extract detailed features of bank note images in the channel and spatial domains. Finally, the label-smoothed cross-entropy (LSCE) loss function is used to train the model to automatically calibrate the network to prevent classification errors. The experimental results demonstrate that the improved model is not easily affected by the image quality, and the model in this paper has good performance in text detection and recognition in specific business ticket scenarios.

show abstract

Section: Introductionmentioning

confidence: 99%

A Study of Spatial Attention and Squeeze Excitation Block Fusion Improved ResNet for Identifying Bank Notes

Huo

2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Cross-modal attention [14,34] is an extension of the previous combination of different modal feature vectors that enhances the representation of multimodal features by assigning different attention weights for modeling information interactions between modalities. lv et al [35] input multilevel features from a three-stream network model into the channel attention mechanism to model the common feature space and focus the network on salient targets. In addition, Attn-Hybrid Net [36] was used to alleviate the redundancy between hybrid features.…”

Section: Attention-based Fusionmentioning

confidence: 99%

Multimodal Finger Recognition Based on Asymmetric Networks With Fused Similarity

Huang

Wang

2023

IEEE Access

View full text Add to dashboard Cite

Multimodal biometric system has received increasing interest as it offers a more secure and accurate authentication solution than unimodal systems. However, existing biometric fusion methods are still inadequate in dealing with correlations and redundancy of multimodal features simultaneously, causing bottlenecks in performance improvement. To overcome the above problem, this paper proposes an end-toend multimodal finger recognition model that incorporates attention mechanisms into a similarity-aware encoder to achieve accurate recognition results for accurate identification results. Firstly, due to the different distribution of fingerprint and finger vein images, we propose a finger asymmetric backbone network (FAB-Net) for extracting discriminative intra-modal features, which reduces the network width by efficient utilization of feature maps. Then, a novel attention-based encoder fusion network (AEF-Net) with fused similarity performs dimensionality reduction-based fusion on multimodal multilevel features to alleviate performance degradation due to information redundancy. We also introduce channel attention in AEF-Net, which differs from the traditional attention mechanism by considering interdependencies between modalities to further improve performance. Extensive recognition experiments are conducted on three multimodal finger databases to verify the effectiveness of our method compared to state-of-the-art methods. Detailed ablation studies have also been carried out, which demonstrated that encoder-based reconstruction of redundant information can improve recognition performance.INDEX TERMS Multimodal biometric recognition, feature fusion, autoencoder, deep learning.

show abstract

“…Lv et al. [34] proposed a three‐stream architecture to accurately predict human visual attention on 3D images. Xi et al.…”

Section: Related Workmentioning

confidence: 99%

A fine‐to‐coarse‐to‐fine weakly supervised framework for volumetric SD‐OCT image segmentation

Niu

Xing

Gao

et al. 2022

IET Computer Vision

View full text Add to dashboard Cite

Obtaining accurate segmentation of central serous chorioretinopathy in spectral-domain optical coherence tomography (SD-OCT) is critical for the determination of the disease severity. Although existing methods achieve considerable segmentation results, they heavily depend on large-scale data with high-quality annotations. Also, the lesions bear a large shape variation across different patients, which are often difficult to encode. To address the above problems, we propose a fine-to-coarse-to-fine weakly supervised framework. Specifically, global alternate max-avg pooling (GTP) network can be employed to locate the lesion regions accurately by using only image-level annotations. A network module based on the GTP network and a semantic transfer module are proposed to iteratively guide the network to continuously discover and expand the target lesion regions. Then, we employ 3D grey distribution histogram to generate pseudo-volumetric labels. Finally, a novel 3D level set loss function is proposed to perform coarse-to-fine volumetric segmentation. Experiments on a challenging dataset demonstrate that the performance of our proposed method is closer to those of models trained with pixel-level supervision. Sijie Niu and Ruiwen Xing contributed to the work equallyThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

show abstract

Attention-based fusion network for human eye-fixation prediction in 3D images

Cited by 15 publications

References 14 publications

A Study of Spatial Attention and Squeeze Excitation Block Fusion Improved ResNet for Identifying Bank Notes

A Study of Spatial Attention and Squeeze Excitation Block Fusion Improved ResNet for Identifying Bank Notes

Multimodal Finger Recognition Based on Asymmetric Networks With Fused Similarity

A fine‐to‐coarse‐to‐fine weakly supervised framework for volumetric SD‐OCT image segmentation

Contact Info

Product

Resources

About