Enhanced cross-domain document clustering with a semantically enhanced text stemmer (SETS)

Abstract

The aim of document clustering is to produce coherent clusters of similar documents. Clustering algorithms rely on text normalisation techniques to represent and cluster documents. Although most document clustering algorithms perform well in specific knowledge domains, processing cross-domain document repositories is still a challenge. This paper attempts to address this challenge. It investigates the performance of the sk-means clustering algorithm across domains, by comparing the cluster coherence produced with semantic-based and traditional (TF-IDF-based) document representations. The evaluation is conducted on 20 different generic sub-domains of a thousand documents, each randomly selected from the Reuters21578 corpus. The experimental results obtained from the evaluation demonstrate improved coherence of clusters produced by using a semantically enhanced text stemmer (SETS), when compared to the text normalisation obtained with the Porter stemmer. In addition, semantic-based text normalisation is shown to be resistant to noise, which is often introduced in the index aggregation stage, a stage that acquires features to represent documents.

Exploring User Experience with Image Schemas, Sentiments, and Semantics

“…The lexical resource used for this purpose is OntoRo. This is an electronic resource based on Roget's Thesaurus [60] and has been employed in several other studies related to design [61] and information retrieval [62], [63]. Moreover, the use of OntoRo is also justified by linguistic studies [36], which have found that many of the names of image schemas correspond to Roget's subcategories.…”

“…However, only one of these concepts, #224 interiority, is related to interior as a space enclosed by a boundary, which is the description of container. Even though concept disambiguation using algorithms developed by the authors in previous design studies [61] and information retrieval research [62], [63] is possible, this was considered unnecessary because this procedure is only performed as part of the conceptualization phase of the ontology.…”

“…In addition, the words in plural forms (e.g., buttons), gerund forms (e.g., looking), and past tense (e.g., asked) are stemmed. However, rather than using the standard Porter stemmer [64], which removes all suffixes, this work uses the semantic stemmer SETS [63], which has shown good results with OntoRo. SETS keeps the words in the form they are used (e.g., connecting is not stemmed to connect) as the inflected words may have different meaning and different semantic representation (i.e., can be linked to different OntoRo concepts).The Porter stemmer creates more polysemic stems, which makes concept disambiguation-especially when working with short sentencesextremely difficult and inaccurate.…”

“…If the word is found in OntoRo, it is considered stemmed, and the SETS algorithm proceeds to the next word. If the word does not occur in it, the algorithm proceeds with the first, second, third, etc., step of the Porter stemmer, repeating the search in OntoRo after each step [63]. Finally, if the algorithm does not find a stem, the word is considered a named entity and is not linked to any concepts.…”

“…The next step is identifying all possible senses for each task-related word. The step uses a semantic algorithm [61], [63], which identifies all possible senses and their corresponding concept numbers in OntoRo. Table 5 shows this process of semantic expansion using the example discussed above.…”