Improving the Performance of Multivariate Bernoulli Model based Documents Clustering Algorithms using Transformation Techniques

Abstract

<b>Problem statement:</b> Document clustering is the most important areas of data mining since&#13; they are very much and currently the subject of significant global research since such areas strengthen&#13; the enterprises of web intelligence, web mining, web search engine design and so forth. Generative&#13; models based on the multivariate Bernoulli and multinomial distributions have been widely used for&#13; text classification. <b>Approach: </b>This study explores the suitability of multivariate Bernoulli model&#13; based probabilistic algorithm for text clustering application. In a multivariate Bernoulli model, a&#13; document is represented as a binary vector over the space of words with 0 and 1, indicating that&#13; whether word occurs or not in the document. The number of occurrences is not considered. So the&#13; word frequency information is lost due to this nature of implementation. In this work, we propose a&#13; FFT based transformation technique for improving clustering performance of multivariate Bernoulli&#13; model based probabilistic algorithm. We are using the transformation technique to transform the actual&#13; term frequency count data in to a time domain signal. So, the weight of frequency of each word will be&#13; distributed throughout each row of records. Now if we apply multivariate Bernoulli model on values&#13; less than zero and greater than zero, the performance will get increased since there is no information&#13; loss in this kind of data representation. <b>Results:</b> In this work, Bernoulli model-based clustering and an&#13; improved version of the same will be implemented and evaluated using suitable metrics and the results&#13; are shown. <b>Conclusion:</b> The transformation technique in multivariate Bernoulli model improves the&#13; performance of document clustering significantly

SocialRec: A Context-Aware Recommendation Framework With Explicit Sentiment Analysis

“…We attempt to address some of the inherent issues of recommendation systems, such as cold start and data sparseness. To address the cold start, our framework utilizes the Hub-Average (HA) inference model [23] that maintains a precomputed list of most popular venues in a user's current vicinity. To address data sparseness caused by zero similarity values, we enhanced the CF algorithm by utilizing textual review as an additional source of user preferences.…”

“…A reviewer is assigned a higher rank (and called as expert in later the text), if the reviewer has given feedback about many higher ranked venues. Similarly, a venue gets a higher score (called as popular in the later text) if the venue is given feedback by many higher ranked reviewers [23] , [46]. Moreover, the recommendation module computes a similarity graph of the expert reviewers.…”

“…The classifier models the distribution of the reviews in each class using a probabilistic model. We assume that the reviews are modelled according to a Bernoulli document model [23] that computes the posterior probability of each class based on the distribution of words in a review. In Bernoulli document model, presence and absence of words in a review is considered as a binary vector that represents a point in a space of words.…”