Multifractal correlations in natural language written texts: Effects of language family and long word statistics

Chatzigeorgiou, Manolis; Constantoudis, Vassilios; Diakonos, F. K.; Karamanos, Konstantinos; Papadimitriou, Constantinos; Papageorgiou, Harris

doi:10.1016/j.physa.2016.11.028

Cited by 18 publications

(14 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…14a, we can infer occurrences of two node duplications in the node sequence that occur at about 8th and 16th nodes later from the first appearance of the node including the considered word. 3 Combining with the discussion about Fig. 13a in Sect.…”

Section: Difference In Functional Form Of Acfmentioning

confidence: 54%

“…During the last decades, it has been clarified that dynamic correlations in written texts are universally observed at any scale of language hierarchy (letters, syllables, words, sentences, etc.) when we regard written texts as time series data ( [1][2][3] and references cited therein). Since our thoughts or ideas are multidimensional complex objects, the reduction in the multidimensional space is inevitable to express the thoughts/ideas as one-dimensional streams of words in written texts.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Simulation of pseudo-text synthesis for generating words with long-range dynamic correlations

2020

View full text Add to dashboard Cite

In a previous study, we have regarded real written texts as time series data and have tried to investigate dynamic correlations of word occurrences by utilizing an autocorrelation function (ACF). The results showed that the obtained ACFs can be classified into two groups: One was a group of ACFs showing dynamic correlations and the other was a group of ACFs showing no correlations. A word having the former ACF was called Type-I word, and a word with the latter ACF was called Type-II word. In this study, we investigate the mechanism of generating dynamic correlations of Type-I words by conducting simulations of pseudo-text synthesis. The results of the simulation clarified that inheritance of words between subsequent paragraphs is key to generate dynamic correlations of Type-I words, indicating a important role of long-range memory with duration time from few to several tens of sentences.

show abstract

Section: Difference In Functional Form Of Acfmentioning

confidence: 54%

Section: Introductionmentioning

confidence: 99%

Simulation of pseudo-text synthesis for generating words with long-range dynamic correlations

2020

View full text Add to dashboard Cite

show abstract

“…We constructed this recursive model by incorporating common hierarchical document structures (chapters, sections, subsections, paragraphs, and sentences), so it can be applied to a broad range of applications for analyzing written texts. Although long-range correlations in written texts have been modeled from various point of views, the methodologies used are completely different from ours [3] [4].…”

Section: Introductionmentioning

confidence: 99%

Origin of Dynamic Correlations of Words in Written Texts

Ogura¹,

Amano²,

Kondo³

2019

JDAIP

View full text Add to dashboard Cite

In a previous study, we introduced dynamical aspects of written texts by regarding serial sentence number from the first to last sentence of a given text as discretized time. Using this definition of a textual timeline, we defined an autocorrelation function (ACF) for word occurrences and demonstrated its utility both for representing dynamic word correlations and for measuring word importance within the text. In this study, we seek a stochastic process governing occurrences of a given word having strong dynamic correlations. This is valuable because words exhibiting strong dynamic correlations play a central role in developing or organizing textual contexts. While seeking this stochastic process, we find that additive binary Markov chain theory is useful for describing strong dynamic word correlations, in the sense that it can reproduce characteristics of autocovariance functions (an unnormalized version of ACFs) observed in actual written texts. Using this theory, we propose a model for time-varying probability that describes the probability of word occurrence in each sentence in a text. The proposed model considers hierarchical document structures such as chapters, sections, subsections, paragraphs, and sentences. Because such a hierarchical structure is common to most documents, our model for occurrence probability of words has a wide range of universality for interpreting dynamic word correlations in actual written texts. The main contributions of this study are, therefore, finding usability of the additive binary Markov chain theory to analyze dynamic correlations in written texts and offering a new model of word occurrence probability in which common hierarchical structure of documents is taken into account.

show abstract

“…The aim of our proposal is to automatically analyse these reports in order to check the priority given by local authorities to the criteria related to SD assessment. Natural language processing remains a complex challenge [9], [10], [11]. Because of the imprecision inherent to language, it is difficult to automatically identify and extract topics from a text in natural language (NL).…”

Section: Introductionmentioning

confidence: 99%

A text-mining and possibility theory based model using public reports to highlight the sustainable development strategy of a city

Duthil

Imoussaten

Montmain

2017

2017 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applicati

View full text Add to dashboard Cite

Nowadays, ecology and sustainable development are priority government's actions. In Europe, and more specifically in France, sustainable development (SD) is generally broken down into several distinct evaluation criteria. Each criterion is a requirement imposed by the government and corresponds to strategic stakes. When SD improvement actions are financed in an economic region or a city of the French territory by the government, a set of measures is usually set up to assess and control the impact of these actions. More precisely, these measures are used to check whether the region or the city has efficiently invested its budget in respect to the SD strategy of the government. This assessment process is a complex task for the government. Indeed, evaluations are only based on reports provided by the financed regions. These very numerous reports are written in natural language and thus, it is a thorny and time-consuming task for the government to efficiently identify the meaningful information in a plethora of reports and then objectively assess all the expected priorities. This project aims at automating the assessment process from the huge corpus of documents. Text-mining and segmentation techniques are introduced to automatically quantify the attention the region or the city pays to a given criterion. Obviously, this quantification can only be imprecisely determined. Then, the possibility theory is used to merge the information related to each criterion prioritization from all the documents. Finally, an application on the 265 largest cities in France shows the potential of the approach.

show abstract

Multifractal correlations in natural language written texts: Effects of language family and long word statistics

Cited by 18 publications

References 25 publications

Simulation of pseudo-text synthesis for generating words with long-range dynamic correlations

Simulation of pseudo-text synthesis for generating words with long-range dynamic correlations

Origin of Dynamic Correlations of Words in Written Texts

A text-mining and possibility theory based model using public reports to highlight the sustainable development strategy of a city

Contact Info

Product

Resources

About