Noun distribution in natural languages

Liang, Junying; Liu, Haitao

doi:10.1515/psicl-2013-0019

Cited by 7 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It is evident that despite the existence of a large number of POS across various languages, and some disagreement about the definitions, there is strong evidence that a set of coarse POS lexical categories exists across all languages in one form or another [ 66 ]. This indicates that while fine-grained relative cross-lingual POS probabilities may vary, when linked to the same observed linguistic instantiations, the cross-lingual probabilities of coarse POS categories are likely to be similar [ 67 , 68 , 69 ]. Therefore, while we have used probabilistic POS rankings from an English language corpora in this example, since we are using only coarse-grained categories, this means that the rankings might be expected to be stable across languages.…”

Section: Example Resultsmentioning

confidence: 99%

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

Back

Wiles

2022

Entropy

View full text Add to dashboard Cite

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf–Mandelbrot–Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.

show abstract

Section: Example Resultsmentioning

confidence: 99%

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

Back

Wiles

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…Specifically, the average HR@1 values of nouns, adjectives, and adverbs are 35.1%, 27.4%, and 11.8%, respectively. Interestingly, PLMs have a high error rate when dealing with nouns even though they are trained with a large written English corpus, where nouns form the greatest portion (at least 37%) of all POS tags (Hudson, 1994;Liang and Liu, 2013).…”

Section: Plms Lack Knowledge Of Antonymsmentioning

confidence: 99%

Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence

Jang¹,

Mtumbuka²,

Lukasiewicz³

2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

The logical negation property (LNP), which implies generating different predictions for semantically opposite inputs (p is true iff ¬p is false), is an important property that a trustworthy language model must satisfy. However, much recent evidence shows that large-size pre-trained language models (PLMs) do not satisfy this property. In this paper, we perform experiments using probing tasks to assess PLMs' LNP understanding. Unlike previous studies that only examined negation expressions, we expand the boundary of the investigation to lexical semantics. Through experiments, we observe that PLMs violate the LNP frequently. To alleviate the issue, we propose a novel intermediate training task, named meaning-matching, designed to directly learn a meaning-text correspondence, instead of relying on the distributional hypothesis. Through multiple experiments, we find that the task enables PLMs to learn lexical semantic information. Also, through fine-tuning experiments on 7 GLUE tasks, we confirm that it is a safe intermediate task that guarantees a similar or better performance of downstream tasks. Finally, we observe that our proposed approach 1 outperforms our previous counterparts despite its time and resource efficiency.

show abstract

“…Véanse, a modo de ejemplo, el trabajo deNikolaeva (2008) sobre el tungús; así comoLiang & Liu (2013) para la distribución de los nombres en las lenguas naturales.…”

unclassified

Lenguas Naturales, Sistemas Cognitivos Humanos, Y Los Límites Del Cerebro Lingüístico

Ivanova

2024

Borealis

View full text Add to dashboard Cite

show abstract

Noun distribution in natural languages

Cited by 7 publications

References 20 publications

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence

Lenguas Naturales, Sistemas Cognitivos Humanos, Y Los Límites Del Cerebro Lingüístico

Contact Info

Product

Resources

About