Hierarchical decoding with latent context for image captioning

Zhang, Jing; Xie, Yingshuai; Li, Kangkang; Wang, Zhe; Du, Wen

doi:10.1007/s00521-022-07726-z

Cited by 6 publications

(2 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we suggest some potential solutions such as hierarchical LSTM architectures Zhang J. et al (2023), or transformers Zhang H. et al (2023) for the future. Furthermore, our qualitative analysis shown in Figure 5 highlights the model's capability to create different and contextually fitting responses, showcasing its adaptability in various situations.…”

Section: Resultsmentioning

confidence: 99%

ChestBioX-Gen: contextual biomedical report generation from chest X-ray images using BioGPT and co-attention mechanism

Ouis,

Akhloufi

2024

Front. Imaging

View full text Add to dashboard Cite

Efficient and accurate radiology reporting is critical in modern healthcare for timely diagnosis and patient care. In this paper, we present a novel deep learning approach that leverages BioGPT and co-attention mechanisms for automatic chest X-ray report generation. Our model, termed “ChestBioX-Gen” is designed to bridge the gap between medical images and textual reports. BioGPT, a biological language model, contributes its contextual understanding to the task, while the co-attention mechanism efficiently aligns relevant regions of the image with textual descriptions. This collaborative combination enables ChestBioX-Gen to generate coherent and contextually accurate reports that embed complex medical findings. Our model not only reduces the burden on radiologists but also enhances the consistency and quality of reports. By automating the report generation process, ChestBioX-Gen contributes to faster diagnoses and improved patient care. Quantitative evaluations, measured through BLEU-N and Rouge-L metrics, demonstrate the model's proficiency in producing clinically relevant reports with scores of 0.6685, 0.6247, 0.5689, 0.4806, and 0.7742 on BLUE 1, 2, 3, 4, and Rouge-L, respectively. In conclusion, the integration of BioGPT and co-attention mechanisms in ChestBioX-Gen represents an advancement in AI-driven medical image analysis. As radiology reporting plays a critical role in healthcare, our model holds the potential to revolutionize how medical insights are extracted and communicated, ultimately benefiting both radiologists and patients.

show abstract

Section: Resultsmentioning

confidence: 99%

ChestBioX-Gen: contextual biomedical report generation from chest X-ray images using BioGPT and co-attention mechanism

Ouis,

Akhloufi

2024

Front. Imaging

View full text Add to dashboard Cite

show abstract

“…Figure 1: A causal structure with two observed variables X and Y affected by one latent variable L, where λ 1 , λ 2 and η represent the causal strength between L and X, L and Y , X and Y , respectively. et Kummerfeld and Ramsey 2016;Huang et al 2022), and Generalized Independence Noise (GIN) condition (Xie et al 2020(Xie et al , 2022 to identify the relationships between latent variables and observed variables. By utilizing the non-Gaussianity, PraceLiNGAM (Tashiro et al 2014), MLCLiNGAM (Chen et al 2021) and RCD (Maeda and Shimizu 2020) can only identify some causal structure among observed variables that are not directly affected by the latent variables.…”

Section: Introductionmentioning

confidence: 99%

Identification of Causal Structure with Latent Variables Based on Higher Order Cumulants

Chen,

Huang,

Cai

et al. 2024

AAAI

View full text Add to dashboard Cite

Causal discovery with latent variables is a crucial but challenging task. Despite the emergence of numerous methods aimed at addressing this challenge, they are not fully identified to the structure that two observed variables are influenced by one latent variable and there might be a directed edge in between. Interestingly, we notice that this structure can be identified through the utilization of higher-order cumulants. By leveraging the higher-order cumulants of non-Gaussian data, we provide an analytical solution for estimating the causal coefficients or their ratios. With the estimated (ratios of) causal coefficients, we propose a novel approach to identify the existence of a causal edge between two observed variables subject to latent variable influence. In case when such a causal edge exits, we introduce an asymmetry criterion to determine the causal direction. The experimental results demonstrate the effectiveness of our proposed method.

show abstract