Cross-directional consistency network with adaptive layer normalization for multi-spectral vehicle re-identification and a high-quality benchmark

Zheng, Aihua; Zhu, Xianpeng; Ma, Zhiqi; Li, Chenglong; Tang, Jin; Ma, Jixin

doi:10.1016/j.inffus.2023.101901

Cited by 10 publications

(9 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thanks to the strong complementarity between visible light and infrared, these two modalities are increasingly being used together in a variety of scenarios, making cross-modal Vehicle Re-Identification a prominent research area. Several studies [21][22][23][24][25] have been conducted to explore this field. Pan et al 21 proposes a hybrid vision transformer (H-ViT) to learn inter and intra-modal information to reduce feature deviations caused by modal variations based on the modal-specific controller (MC) and the modal information embedding (MIE) structure.…”

Section: Cross-modal Vehicle Re-identificationmentioning

confidence: 99%

“…Adversarial learning is also employed to bridge the modality gap at the image level. Zheng et al 23 proposes to deal with this task in complex lighting environments and diverse scenes by utilizing multi-spectral sources, such as visible and infrared information. Li et al 24 proposes a multi-spectral vehicle Re-ID benchmark named RGBN300, including RGB and NIR vehicle images of 300 identities from 8 camera views.…”

Section: Cross-modal Vehicle Re-identificationmentioning

confidence: 99%

“…To this end, cross-modality vehicle re-identification gets more attention. Towards this task, some pioneer works [21][22][23][24][25] have explored and achieved initial success based on these two modalities. However, they mainly focus on extracting the high-level semantic information for identification, while ignoring the underlying information such as texture and color, with critically local information loss.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cross-modal vehicle re-identification based on multi-scale features and attention mechanism

Jin,

Zou,

Zhong

2024

MIPPR 2023: Pattern Recognition and Computer Vision

View full text Add to dashboard Cite

Our work focuses on exploring the emerging field of cross-modal vehicle re-identification. Achieving accurate cross-modal vehicle re-identification requires a network that can capture local details from two different modality images while effectively fusing their valid information. However, existing methods only consider extracting high-level semantics, leading to a loss of fine-grained details and imprecise identification. Additionally, insufficient attention has been paid to effective information in different modalities, as cross-modality interaction has not been thoroughly explored. To address these issues, we propose a new cross-modal vehicle re-identification network consisting of a multi-scale feature fusion module and a cross-modal attention module. Specifically, the multiscale feature fusion module captures both global high-level semantics and local details by integrating multi-scale information in the feature extraction process, reducing the loss of local details. The cross-modal attention module explores valid information from different modalities and achieves feature-level fusion. We conducted experiments on the RGBNT100 cross-modal vehicle re-identification dataset to verify the proposed method’s effectiveness.

show abstract

Section: Cross-modal Vehicle Re-identificationmentioning

confidence: 99%

Section: Cross-modal Vehicle Re-identificationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cross-modal vehicle re-identification based on multi-scale features and attention mechanism

Jin,

Zou,

Zhong

2024

MIPPR 2023: Pattern Recognition and Computer Vision

View full text Add to dashboard Cite

show abstract

“…As a result, although non-visible images have great potential to boost vehicle ReID performance in low illumination environments, there is an open question in multi-modal ReID in practice: how to effectively fuse the complementary information from multi-modal data? Existing multi-modal vehicle Re-ID [17][18][19][20] most focus on learning modal robust features. For example, Wang et al [20] designed a cross-modal interacting module and a relation-based embedding module to exchange useful information from multi-modal features so as to enhance features' richness.…”

Section: Introductionmentioning

confidence: 99%

“…Both cross-modal interacting and relation-based embedding modules are convolutional neural network (CNN) branches. Zheng et al [19] proposed a cross-directional consistency network to mitigate cross-modal discrepancies and adjust individual feature distributions for learning modal robust features. Li et al [17] proposed a heterogeneity collaboration aware multi-stream convolutional neural network to constrain scores of different instances of the same identity to be coherent.…”

Section: Introductionmentioning

confidence: 99%

Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification

Pan

Huang

Liang

et al. 2023

Sensors

View full text Add to dashboard Cite

Multi-modal (i.e., visible, near-infrared, and thermal-infrared) vehicle re-identification has good potential to search vehicles of interest in low illumination. However, due to the fact that different modalities have varying imaging characteristics, a proper multi-modal complementary information fusion is crucial to multi-modal vehicle re-identification. For that, this paper proposes a progressively hybrid transformer (PHT). The PHT method consists of two aspects: random hybrid augmentation (RHA) and a feature hybrid mechanism (FHM). Regarding RHA, an image random cropper and a local region hybrider are designed. The image random cropper simultaneously crops multi-modal images of random positions, random numbers, random sizes, and random aspect ratios to generate local regions. The local region hybrider fuses the cropped regions to let regions of each modal bring local structural characteristics of all modalities, mitigating modal differences at the beginning of feature learning. Regarding the FHM, a modal-specific controller and a modal information embedding are designed to effectively fuse multi-modal information at the feature level. Experimental results show the proposed method wins the state-of-the-art method by a larger 2.7% mAP on RGBNT100 and a larger 6.6% mAP on RGBN300, demonstrating that the proposed method can learn multi-modal complementary information effectively.

show abstract