A study on Chinese register characteristics based on regression analysis and text clustering

Abstract

This paper reports an innovative Chinese register study based on regression analysis for sentence length distribution and text clustering. Although end of sentence is not conventionally marked in Chinese, we resolve this issue by assuming that segments between periods, question marks, and exclamation marks are sentences, which can be further divided into simple sentences and compound sentences. We also assume that segments between punctuation marks that express pauses in utterances form sentences (i.e., clauses). Using regression analysis, we find that the frequency distribution of sentence and clause lengths in Chinese can be fitted by the formula F = aLbcL, where L is sentence/clause length. Texts from different registers give rise to different fitted values of the parameters, and hence can serve to differentiate these registers. Finally, we use these parameters to represent and cluster texts from different registers. The successful text clustering results further prove that the parameters of the fitted results are reliable linguistic characteristics for different registers. In terms of linguistic theories, our study shows that it is just as effective to model sentence length in Chinese using sociological words (i.e., characters) as it is using linguistic words.

Classification of regional and genre varieties of Chinese: A correspondence analysis approach based on comparable balanced corpora

“…This result suggests that robust identification technology of similar and related languages must also take into consideration other dimensions of textual variations such as gender, genre, or register. In fact, in a series of study we did take register into consideration (Hou et al 2017, Hou, Huang andLiu 2019;. In addition to show that fitted power function model between linguistic units and their constituents is an effective tool for classification of similar languages, we also showed that single feature works better with binary classification.…”

“…Altmann (2005, 2007) demonstrated an effective way to model discrete phenomenon using continuous models and vice versa. Hou, Huang, and Liu (2019) showed that the sentence/clause length in Chinese texts can also be fitted by Formula (1) in variations of the Chinese language based on data from Mainland China.…”

“…Hou et al (2014) examined sentence length distribution in terms of words and word length distribution in terms of Chinese characters in order to distinguish formal written style and daily informal style based on text clustering. Hou, Huang, and Liu (2019) modeled the sentence length using Formula (1) and proved that the fitted parameters of the sentence length distribution can differentiate different register texts in Mainland variety of Chinese.…”

“…The clause length distribution in Chinese Mandarin can be fitted by Formula (1) from Hou, Huang and Liu (2019). Then, the texts can be represented by the fitted parameters, a, b, and c, of the clause length distribution.…”

Robust stylometric analysis and author attribution based on tones and rimes

“…According to the approach of many Chinese treebanks (e.g., Chen et al 1996 for Sinica TreeBank, Huang and Chen 2017) and the analysis of sentence length distribution in quantitative linguistics (Hou, Huang, and Liu 2017), all segments between commas, semicolons, colons, periods, exclamation marks, and question marks that express pauses in utterances are marked as sentences. Actually, the sentences that are identified by this definition are clauses (Hou et al 2017) and conform to the definitions that rely on pauses and intonation changes in the utterances.…”