Semantic spaces for improving language modeling

Brychcín, Tomáš; Konopík, Miloslav

doi:10.1016/j.csl.2013.05.001

Cited by 21 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The first line regards the exploration of other types of information, instead of word-level linguistic knowledge. In this respect, recent work in language modelling follows an unsupervised approach by using semantic spaces for clustering words (Brychcin and Konopík, 2014). The benefit is obvious, especially for under-resourced languages, as this approach avoids the use of tools such as PoS taggers, lemmatisers and NE recognisers.…”

Section: Discussionmentioning

confidence: 99%

Linguistically-augmented perplexity-based data selection for language models

Toral

Pecina

Wang

et al. 2015

Computer Speech & Language

View full text Add to dashboard Cite

This paper explores the use of linguistic information for the selection of data to train language models. We depart from the stateof-the-art method in perplexity-based data selection and extend it in order to use word-level linguistic units (i.e. lemmas, named entity categories and part-of-speech tags) instead of surface forms. We then present two methods that combine the different types of linguistic knowledge as well as the surface forms (1, naïve selection of the top ranked sentences selected by each method; 2, linear interpolation of the datasets selected by the different methods). The paper presents detailed results and analysis for four languages with different levels of morphologic complexity (English, Spanish, Czech and Chinese). The interpolation-based combination outperforms the purely statistical baseline in all the scenarios, resulting in language models with lower perplexity. In relative terms the improvements are similar regardless of the language, with perplexity reductions achieved in the range 7.72-13.02%. In absolute terms the reduction is higher for languages with high type-token ratio (Chinese, 202.16) or rich morphology (Czech, 81.53) and lower for the remaining languages, Spanish (55.2) and English (34.43 on the English side of the same parallel dataset as for Czech and 61.90 on the same parallel dataset as for Spanish).

show abstract

Section: Discussionmentioning

confidence: 99%

Linguistically-augmented perplexity-based data selection for language models

Toral

Pecina

Wang

et al. 2015

Computer Speech & Language

View full text Add to dashboard Cite

show abstract

“…There are a lot of methods for deriving word meaning from the local context. We have already experimented with five of them in Brychcín and Konopík (2014). In this paper we continue our research and use only the three best performing methods in language modelling (HAL, COALS, and RI).…”

Section: Local Contextmentioning

confidence: 95%

“…In this paper we extend our work on the application of semantic spaces in language modelling (Brychcín and Konopík, 2014), where we have achieved significant improvements in perplexity and in machine translation task especially with HAL, COALS and RI models. Thus, these models are investigated more deeply in this paper.…”

Section: Introductionmentioning

confidence: 94%

Latent semantics in language models

Brychcín

Konopík

2015

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

“…One connected cluster is formed including all numbers. Reconstruction is based on the following formula: (1) Results are represented by the distributions (Figure 2A …”

Section: Validation Of the Methodsmentioning

confidence: 99%

“…Many modern applications in cognitive sciences rely on semantic maps, also called (semantic) cognitive maps or semantic spaces [1][2][3][4]. These are continuous manifolds with semantics attributed to their elements and with semantic relations among those elements captured by geometry.…”

Section: Introductionmentioning

confidence: 99%

Universal Dimensions of Meaning Derived from Semantic Relations among Words and Senses: Mereological Completeness vs. Ontological Generality

Samsonovich

Ascoli

2014

Computation

View full text Add to dashboard Cite

Abstract:A key to semantic analysis is a precise and practically useful definition of meaning that is general for all domains of knowledge. We previously introduced the notion of weak semantic map: a metric space allocating concepts along their most general (universal) semantic characteristics while at the same time ignoring other, domain-specific aspects of their meanings. Here we address questions of the number, quality, and mutual independence of the weak semantic dimensions. Specifically, we employ semantic relationships not previously used for weak semantic mapping, such as holonymy/meronymy ("is-part/member-of"), and we compare maps constructed from word senses to those constructed from words. We show that the "completeness" dimension derived from the holonym/meronym relation is independent of, and practically orthogonal to, the "abstractness" dimension derived from the hypernym-hyponym ("is-a") relation, while both dimensions are orthogonal to the maps derived from synonymy and antonymy. Interestingly, the choice of using relations among words vs. senses implies a non-trivial trade-off between rich and unambiguous information due to homonymy and polysemy. The practical utility of the new and prior dimensions is illustrated by the automated evaluation of different kinds of documents. Residual analysis of available linguistic resources, such as WordNet, suggests that the number of universal semantic dimensions representable in natural language may be finite. Their complete characterization, as well as the extension of results to non-linguistic materials, remains an open challenge. OPEN ACCESSComputation 2014, 2 62

show abstract

Semantic spaces for improving language modeling

Cited by 21 publications

References 15 publications

Linguistically-augmented perplexity-based data selection for language models

Linguistically-augmented perplexity-based data selection for language models

Latent semantics in language models

Universal Dimensions of Meaning Derived from Semantic Relations among Words and Senses: Mereological Completeness vs. Ontological Generality

Contact Info

Product

Resources

About