Word class frequencies in Brazilian‐Portuguese press texts

Ziegler, Arne

doi:10.1080/09296179808590136

Cited by 7 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is evident that despite the existence of a large number of POS across various languages, and some disagreement about the definitions, there is strong evidence that a set of coarse POS lexical categories exists across all languages in one form or another [ 66 ]. This indicates that while fine-grained relative cross-lingual POS probabilities may vary, when linked to the same observed linguistic instantiations, the cross-lingual probabilities of coarse POS categories are likely to be similar [ 67 , 68 , 69 ]. Therefore, while we have used probabilistic POS rankings from an English language corpora in this example, since we are using only coarse-grained categories, this means that the rankings might be expected to be stable across languages.…”

Section: Example Resultsmentioning

confidence: 99%

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

Back

Wiles

2022

Entropy

View full text Add to dashboard Cite

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf–Mandelbrot–Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.

show abstract

Section: Example Resultsmentioning

confidence: 99%

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

Back

Wiles

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…Hammerl (1990), Best (1994), Judt (1995), Ziegler (2001), Liu (2009), andTuzzi, Popescu, Altmann (2009)). Though the term part-of-speech (or word-class) distribution is used, actually the rank-frequency distribution of the number of words of a given part of speech is studied.…”

Section: Introductionmentioning

confidence: 96%

Part-of-speech studies in Chinese

Wang

2016

Journal of Quantitative Linguistics

View full text Add to dashboard Cite

This paper studies parts of speech in Chinese on data taken from the Modern Chinese Dictionary (5th edition). First, the part-of-speech polyfunctionality (ambiguity) of words is determined; then the corresponding distribution and rank-frequency sequence are analysed. The Waring and right truncated modified Zipf-Alekseev distributions are successfully fitted to the data. Second, the 121 patterns that the total 3742 polyfunctional words yield are presented. The polyfunctionality of patterns distributes according to the positive Cohen-binomial distribution, while the rank-frequency sequence abides by the negative hypergeometric distribution. Third, we discuss the mechanism behind the polyfunctionality phenomenon: Chinese words diversify in the dimension of their function but not in their form as can be expected from an analytic language. The Popescu-Altmann function captures the distribution of the variants of each part of speech. Fourth, we analyse the polyfunctionality distributions of individual parts of speech. Out of the 12 parts of speech which the dictionary distinguishes, six can be modelled by the Poisson distribution, four by the mixed Poisson, and two by the Singh-Poisson distribution. In order to obtain a general form, we apply the mixed Poisson distribution for all the parts of speech by controlling one parameter. We make a first attempt to plot the polyfunctionality distributions of individual parts of speech in Ord's system, which surprisingly shows approximately a hyperbola.

show abstract

“…Except Chinese (with verbs taking 21.84%), the noun proportion in other three languages takes up the largest share of all word classes. Ziegler (1998Ziegler ( , 2001 also investigated the Brazilian-Portuguese press texts with respect to their word class frequency distributions. It is shown that the noun proportion and the pronoun proportion clearly follow a regularity, with the former taking up around 20%, whereas the latter taking 10%.…”

Section: Introductionmentioning

confidence: 99%

Noun distribution in natural languages

Liang¹,

Liu²

2013

Poznań Studies in Contemporary Linguistics

View full text Add to dashboard Cite

Previous research on word class distribution claimed that 37% of word tokens are nouns, suggesting that there might exist a certain regularity of noun proportion among human languages. To explore this possibility, we examined the proportion of noun and four other word classes within British and American English, and across seven languages in terms of different word frequency band. Results indicated that the noun proportion is evidently about or larger than 37%, and meanwhile increases with word rarity. Among frequent words, nouns increase as minor word classes decrease, whereas among rare words, the noun proportion remains a stable level.

show abstract

Word class frequencies in Brazilian‐Portuguese press texts

Cited by 7 publications

References 4 publications

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

Part-of-speech studies in Chinese

Noun distribution in natural languages

Contact Info

Product

Resources

About