Research on Text Value and Linguistic Characteristics in Ancient Literature Based on Text Mining Technology

Abstract

By combining mining algorithms, this paper provides a new method for quantifying the characteristics of literary language, thereby reducing the subjectivity of the value and language discussion of ancient literature in Chinese. Firstly, the TF-IDF algorithm is improved, the methods of inter-class dispersion and intra-class dispersion are adopted, and the optimal number of topics K of the LDA topic model is determined by using the confusion degree, and the quantitative model of the value and language characteristics of ancient literary texts is constructed. Furthermore, the concept of maximum word-to-frequency ratio is introduced. It is integrated into the traditional information gain method, and an old academic text recognition algorithm based on XGBoost model is constructed. The model’s results were applied to the network corpus mining and analysis, and the results showed that the word “cherishing spring” ranked first with a frequency of 7085 occurrences, followed by “hurt autumn” with 4598 occurrences. Among the eight themes, “natural imagery” (Topic 3) accounted for the highest proportion, reaching 23.68%, followed by “landscape and pastoral” (Topic 7) and “euphemistic words” (Topic2), accounting for 16.29% and 14.54%, respectively. The method of this paper not only provides a new perspective and tool for the quantitative analysis of the linguistic characteristics of literary works, but also points out a new research direction for the in-depth discussion of textual value and linguistic characteristics in the future.