Sentence generation is a key task in many natural language processing systems. Models based on a variational autoencoder (VAE) can generate plausible sentences from a continuous latent space. However, the VAE forces the latent distribution of each input sentence to match the same prior, which results in a large overlap among the latent subspaces of different sentences and a limited informative latent space. Therefore, the sentences generated by sampling from a subspace may have little correlation with the corresponding input, and the latent space cannot capture rich useful information from the input sentences, which leads to the failure of the model to generate diverse sentences from the latent space. Additionally, the Kullback-Leibler (KL) divergence collapse problem makes the VAE notoriously difficult to train. In this paper, a latent space expanded VAE (LSE-VAE) model is presented for sentence generation. The model maps each sentence to a continuous latent subspace under the constraint of its own prior distribution, and constrains nearby sentences to map to nearby subspaces. Sentences are dispersed to a large continuous latent space according to sentence similarity, where the latent subspaces of different sentences may be relatively far away from each other and arranged in an orderly manner. The experimental results show that the LSE-VAE improves the reconstruction ability of the VAE, generates plausible and more diverse sentences, and learns a larger informative latent space than the VAE with the properties of continuity and smoothness. The LSE-VAE does not suffer from the KL collapse problem, and it is robust to hyperparameters and much easier to train.
Automated essay scoring aims to evaluate the quality of an essay automatically. It is one of the main educational application in the field of natural language processing. Recently, Pre-training techniques have been used to improve performance on downstream tasks, and many studies have attempted to use pre-training and then fine-tuning mechanisms in an essay scoring system. However, obtaining better features such as prompts by the pre-trained encoder is critical but not fully studied. In this paper, we create a prompt feature fusion method that is better suited for fine-tuning. Besides, we use multi-task learning by designing two auxiliary tasks, prompt prediction and prompt matching, to obtain better features. The experimental results show that both auxiliary tasks can improve model performance, and the combination of the two auxiliary tasks with the NEZHA pre-trained encoder produces the best results, with Quadratic Weighted Kappa improving 2.5% and Pearson’s Correlation Coefficient improving 2% on average across all results on the HSK dataset.
Vocabulary grading is of great importance in Chinese vocabulary teaching. This paper starts with an analysis of the lexical attributes that affect lexical complexity, followed by an explanation of the extraction of lexical attribute information combined with the constructed word-formation knowledge base, the construction of mapping functions corresponding to lexical attributes, and the quantitative representation of the attributes that form the basis for vocabulary grading. Based on this, a machine learning classification algorithm is creatively applied to the Chinese vocabulary grading problem. Using the comparative analysis of vocabulary grading models based on common machine learning classification algorithms, the importance measurement analysis of Chinese vocabulary attributes based on different feature selection methods is performed, and a vocabulary grading model is constructed based on the machine learning classification algorithm and feature importance selection of different feature selection algorithms. A comparison of the experimental results demonstrated that the classification model based on the support vector machine (SVM) algorithm and top six attribute groups by the importance of feature selection received the best effect. To improve vocabulary grading, a variety of feature selection algorithms were used to fuse the importance of lexical attributes on average. Then an experiment was conducted for vocabulary grading combined with the Bagging + SVM integration algorithm and top six attribute groups by the importance of feature selection. The experimental results demonstrated that the combination scheme achieved a better effect.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.