Evaluation of the Coherence of Polish Texts Using Neural Network Models

Abstract

Coherence evaluation of texts falls into a category of natural language processing tasks. The evaluation of texts’ coherence implies the estimation of their semantic and logical integrity; such a feature of a text can be utilized during the solving of multidisciplinary tasks (SEO analysis, medicine area, detection of fake texts, etc.). In this paper, different state-of-the-art coherence evaluation methods based on machine learning models have been analyzed. The investigation of the effectiveness of different methods for the coherence estimation of Polish texts has been performed. The impact of text’s features on the output coherence value has been analyzed using different approaches of a semantic similarity graph. Two neural networks based on LSTM layers and a pre-trained BERT model correspondingly have been designed and trained for the coherence estimation of input texts. The results obtained may indicate that both lexical and semantic components should be taken into account during the coherence evaluation of Polish documents; moreover, it is advisable to analyze corresponding documents in a sentence-by-sentence manner taking into account word order. According to the retrieved accuracy of the proposed neural networks, it can be concluded that suggested models may be used in order to solve typical coherence estimation tasks for a Polish corpus.

LSTM and GRU Neural Networks as Models of Dynamical Processes Used in Predictive Control: A Comparison of Models Developed for Two Chemical Reactors

“…The unique long-term memory properties of LSTM and GRU neural networks made them widely popular in a large variety of machine learning tasks. Example applications of the LSTM architecture are: data classification [ 22 ], speech recognition [ 23 , 24 ], handwriting recognition [ 25 ], speech synthesis [ 26 ], text coherence tests [ 27 ], biometric authentication and anomaly detection [ 28 ], detecting deception from gaze and speech [ 29 ] and anomaly detection [ 30 ]. Similarly, example applications of the GRU structure are: facial expression recognition [ 31 ], human activity recognition [ 32 ], cyberbullying detection [ 33 ], defect detection [ 34 ], human activity surveillance [ 35 ], automated classification of cognitive workload tasks [ 36 ] and speaker identification [ 37 ].…”

Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT

“…The first one is based on Semantic Similarity Graph (SSG); the second one is based on Long Short Term Memory (LSTM); and the third one is based on BERT. The results show that the neural network related methods offer better accuracy than the SSG related methods; and within the neural networks, although the LSTM based method shows better accuracy compared to the BERT based method, it is emphasized that the latter can increase the value of said metric with an additional Fine-Tuned [4].…”

“…Coherence also implies the type of informational and semantic connectivity that a text possesses [3]. A text is considered coherent if it is semantically consistent and provides cognitive integrity [4], therefore, a coherent document is easier to understand than an incoherent document. Coherence is more important when analyzing scientific papers, as it must communicate information effectively to reviewers and researchers.…”

“…The evaluation of coherence requires a thorough analysis of the parts of the text at the structural and semantic level, since being a natural language it does not follow a set of rules like formal languages [4]. It is too abstract a concept [6]; however, it is a problem that has been given attention in different studies and/or solutions since the twentieth century, as mentioned in the following paragraphs.…”

A MLP-Hedge-Algebras Admittance Controller for Physical Human–Robot Interaction

“…Currently, artificial intelligence is an attractive research topic. Classes of networks such as multilayer perceptron networks (MLPs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs) provide much flexibility and have proven themselves over decades to be useful and reliable in various areas [34][35][36][37][38][39][40][41][42]. RNNs were designed to work with sequence prediction problems, which are traditionally difficult to train and not appropriate for tabular datasets.…”