Image captioning based on scene graphs: A survey

Jia, Je-Lun; Ding, Xiaoxi; Pang, Shunpeng; Gao, Xiaoyan; Xin, Xiaowei; Hu, Ruotong; Nie, Jie

doi:10.1016/j.eswa.2023.120698

Cited by 7 publications

(1 citation statement)

References 150 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These solutions, whether they rely on transformers and attention mechanisms [6,7,8], or scene graphs as presented in [9], in which learning is supervised, or relying on beam search analysis or gated recurrent units (GRU) units, in which learning is unsupervised [10,11], generate one single sentence for each input image. Such models are trained on RGB image datasets [12,13].…”

Section: Sentence Captioningmentioning

confidence: 99%

Egocentric Scene Description for the Blind and Visually Impaired

Delloul

Larabi

2022

2022 5th International Symposium on Informatics and Its Applications (ISIA)

View full text Add to dashboard Cite

In recent years, image captioning and segmentation have emerged as crucial tasks in computer vision, with applications ranging from autonomous driving to content analysis. Although multiple solutions have emerged to help blind and visually impaired people move around their environment, few are applications that help them understand and rebuild a scene in their minds through text. Most built models focus on helping users move and avoid obstacles, restricting the number of environments blind and visually impaired people can be in.In this paper, we will propose an approach that helps them understand their surroundings using image captioning. The particularity of our research is that we offer them descriptions with positions of regions and objects regarding them (left, right, front), as well as positional relationships between regions, while we aim to give them access to theatre plays by applying the solution to our TS-RGBD dataset.

show abstract