“…Visually-aided Language Learning Previous research attempt to introduce visual information to improve language learning on various Natural Language Processing (NLP) scenarios, including but not limited to machine translation (Grubinger et al, 2006;Elliott et al, 2016), information retrieval (Funaki and Nakayama, 2015;Gu et al, 2018), semantic parsing (Christie et al, 2016;Shi et al, 2019), natural language inference (Xie et al, 2019), bilingual lexicon learning (Kiela et al, 2015;Vulic et al, 2016), natural language generation evaluation (Zhu et al, 2021), spatial commonsense reasoning (Liu et al, 2022) and language representation learning (Lazaridou et al, 2015;Collell et al, 2017;Kiela et al, 2018;Bordes et al, 2019;Lu et al, 2019;Li et al, 2019;Luo et al, 2020;Li et al, 2020;Tan and Bansal, 2020;Radford et al, 2021). While most of these studies acquire visual information through retrieval from the web or large-scale image sets, a recent line of studies attempt to generate visual supervision from scratch.…”