Research on Words Segmentation Technology in Chinese Full Text Retrieval System

Abstract

In order to improve the speed of Chinese full-text retrieval in the premise of ensuring Chinese ambiguity inclusion and length limitation, this paper introduces the application methods of Chinese full-text retrieval system and the current application situation of Chinese word segmentation technology. Based on the existed word segmentation algorithms, this paper proposed an improved Chinese word segmentation algorithm. In the proposed method, the procedure of indexing is to construct the map between the relative words in the context and the dictionary. This paper improves the diction to realize better mapping with relative words, so as to realize Chinese words segmentation. The experiments demonstrate that the proposed Chinese full-text words segmentation algorithm is more effective than the existing methods.