Letter2vec+Freq2vec: A Comprehensive Approach to Text Vectorization and Predicting Word Difficulty

Abstract

This paper introduces the self-developed word embedded algorithm Letter2vec+Freq2vec. It combines with biKmeans and Stacking algorithm to judge word difficulty in Wordle. The algorithm combines Letter2vec, which models letter interactions and context, and Freq2vec, which calculates letter frequencies at different positions of words to produce a more comprehensive text vectorization method. The results show that this algorithm can get better results than BERT algorithm in small sample size when vectorizing a single word. Overall, this paper provides insights into the development and application of new algorithms for text vectorization and the prediction of word difficulty.