Learning to Discover, Ground and Use Words with Segmental Neural Language Models

Abstract

We propose a segmental neural language model that combines the generalization power of neural networks with the ability to discover word-like units that are latent in unsegmented character sequences. In contrast to previous segmentation models that treat word segmentation as an isolated task, our model unifies word discovery, learning how words fit together to form sentences, and, by conditioning the model on visual context, how words' meanings ground in representations of nonlinguistic modalities. Experiments show that the unconditional model learns predictive distributions better than character LSTM models, discovers words competitively with nonparametric Bayesian word segmentation models, and that modeling language conditional on visual context improves performance on both.

Unsupervised Recurrent Neural Network Grammars

“…In terms of grammar induction, they are competitive with recently-proposed neural architectures that discover tree-like structures through gated attention (Shen et al, 2018). Our results, along with other recent work on joint language modeling/structure learning with deep networks (Shen et al, 2018(Shen et al, , 2019Wiseman et al, 2018;Kawakami et al, 2018), suggest that it is possible learn generative models of language that model the underlying data well (i.e. assign high likelihood to held-out data) and at the same time induce meaningful linguistic structure.…”

Grounding ‘Grounding’ in NLP

“…There is some work that presents a bayesian probabilistic formulation to learn referential grounding in dialog (Liu et al, 2014), user preferences (Cadilhac et al, 2013), color descriptions (McMahan and Stone, 2015Andreas and Klein, 2014). A huge chunk of work also focus on leveraging attention mechanism for grounding multimodal phenomenon in images (Srinivasan et al, 2020;Chu et al, 2018;Fan et al, 2019;Vu et al, 2018;Kawakami et al, 2019;Dong et al, 2019), videos (Lei et al, 2020; and navigation of embodied agents (Yang et al, 2020), etc., Some approach this using data structures such as graphs in the domains of grounding images (Chang et al, 2015;Liu et al, 2014), videos ), text (Laws et al, 2010;Chen, 2012;Massé et al, 2008), entities (Zhou et al, 2018a), knowledge graphs and ontologies (Jauhar et al, 2015;Zhang et al, 2020) and interactive settings Jauhar et al (2015); Xu et al (2020).…”

Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence

“…Unlike word-to-word alignment, we focus on learning the alignment between data records and text segments. Some works also integrate neural language models to jointly learn the segmentation and correspondence, e.g., phrase-based machine translation (Huang et al, 2018), speech recognition (Wang et al, 2017) and vision-grounded word segmentation (Kawakami et al, 2019). Data-to-text naturally fits into this scenario since each data record is normally verbalized in one continuous text segment.…”