An Introduction to Text Mining

Aggarwal, Charų C.; Zhai, Cheng Xiang

doi:10.1007/978-1-4614-3223-4_1

Cited by 153 publications

(109 citation statements)

References 7 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…Stop_words are common and high frequency words such as "a", "the", "of", "and", "an" "in" etc. Finally, the stemming process converts all the inflected words present in the text into a root form called a stem 7 . For example, 'automatic,' 'automate,'…”

Section: Tweets Preprocessingmentioning

confidence: 99%

Target Oriented Tweets Monitoring System during Natural Disasters

Win

Aung

2017

IJNDC

View full text Add to dashboard Cite

Twitter, Social Networking Site, becomes most popular microblogging service and people have started publishing data on the use of it in natural disasters. Twitter has also created the opportunities for first responders to know the critical information and work effective reactions for impacted communities. This paper introduces the tweet monitoring system to identify the messages that people updated during natural disasters into a set of information categories and provide user desired target information type automatically. In this system, classification is done at tweet level with three labels by using LibLinear classifier. This system is intended to extract the small number of informational and actionable tweets from large amounts of raw tweets on Twitter using machine learning and natural language processing (NLP). Feature extraction of this work exploited only linguistic features, sentiment lexicon based features and especially disaster lexicon based features. The monitoring system also creates disaster related corpus with new tweets collected from Twitter API and annotation is done on real time manner. The performance of this system is evaluated based on four publicly available annotated datasets. The experiments showed the classification accuracy on the proposed features set is higher than the classifier based on neural word embeddings and standard bag-of-words (BOW) models. This system automatically annotated the Myanmar_Earthquake_2016 dataset at 75% accuracy on average.

show abstract

Section: Tweets Preprocessingmentioning

confidence: 99%

Target Oriented Tweets Monitoring System during Natural Disasters

Win

Aung

2017

IJNDC

View full text Add to dashboard Cite

show abstract

“…It di↵ers from information retrieval in the traditional sense that uses keyword search. According to Aggarwal and Zhai (2012a), "[T]ext mining can be regarded as going beyond information access to further help users analyze and digest information and facilitate decision making (Ibid, p. 2)". While information retrieval involves providing the user with documents that match the keywords in his query as it happens with Google search, information extraction goes a step further by extracting and compressing semantic information from text data.…”

Section: Text Mining and Sentiment Analysismentioning

confidence: 99%

Information aggregation and computational intelligence

Chen

Venkatachalam

2016

Evolut Inst Econ Rev

View full text Add to dashboard Cite

This paper examines the possibility that the computational intelligence (CI) inspired tools can e↵ectively aggregate the rich information generated from the Web 2.0 economy, and thereby enhance the quality of decision-making. Despite many advancements and commendable applications of CI in recent years, this issue has not been well addressed. We argue that this question is intimately related to the central issue of the socialist calculation debate since the time of Friedrich Hayek. In terms of information aggregation, we examine whether there is a better engineering than the market mechanism. More precisely, we focus on whether the CI-driven sentiment analysis can generate signals like prices and whether CI can process the unstructured text data better than the market. We argue that Web 2.0 economy may not be able to set us free from information overload problems that have long co-existed with the presence of markets. We attribute this to the tacitness and subjectivity of knowledge and the recursive (feedback) characteristic of the sentiments. In this sense, Hayek's fundamental assertion that the e↵ectiveness of the market mechanism may not be so much conditioned on the information and communication technology still applies.The first author and the second author are grateful for the research support in the form of Ministry of Science and Technology (MOST) grants, MOST 103-2410-H-004-009-MY3 and MOST 104-2811-H-004-003, respectively. We thank the two anonymous referees for their valuable suggestions that helped to improve the paper. All remaining errors are our own.

show abstract

“…In order to save the manpower and find interesting knowledge effectively, text mining becomes more and more important. Text mining also refers to as text data mining [1]- [4], roughly equivalent to text analytics, refers to the process of deriving high-quality information from text. People usually use methods in data mining to find interesting information from text, such as the well-known data mining methods association rule mining [4]- [6], classification [7]- [8], clustering [9]- [10], etc.…”

Section: Introductionmentioning

confidence: 99%

A Hierarchical Document Clustering Approach with Frequent Itemsets

Lee¹,

Hsu²,

Chen³

2017

IJET

View full text Add to dashboard Cite

Abstract-In order to effectively retrieve required information from the large amount of information collected from the Internet, document clustering in text mining becomes a popular research topic. Clustering is the unsupervised classification of data items into groups without the need of training data. Many conventional document clustering methods perform inefficiently for large document of collected information and require special handling for high dimensionality and high volume. We propose the OCFI (Ontology and Closed Frequent Itemset-based Hierarchical Clustering) method, which is a hierarchical clustering method developed for document clustering. OCFI uses common words to cluster documents and builds hierarchical topic tree. In addition, OCFI utilizes ontology to solve the semantic problem and mine the meaning behind the words in documents. Furthermore, we use the closed frequent itemsets instead of only use frequent itemsets, which increases efficiency and scalability. The experimental results reveal that our method is more effective than the well-known document clustering algorithms. The clustering results can be used in the personalized search service to assist users to obtain the information they need.Index Terms-OCFI, documents clustering, ontology, closed frequent itemsets. I. INTRODUCTIONDue to the popularity of the Internet, a large number of documents, reports, e-mails, and web pages cause the information overload problem. Many enterprises spend lots of manpower on organizing these unstructured data into a logical structure for later use. In order to save the manpower and find interesting knowledge effectively, text mining becomes more and more important. information from text, text mining usually involves the process of parsing the input text, mining valuable information through the analysis models, and finally evaluating and interpreting the output. However, most of the text mining algorithms do not have enough capability to solve the problems from text mining, because many methods are modifications of traditional data mining algorithms that were originally designed for relational database. Therefore, traditional clustering algorithms become impractical in real-world document clustering which requires special handling for high dimensionality, high volume, and ease of browsing.Fung proposed Frequent Itemset-based Hierarchical Clustering (FIHC) [11] method to solve the problems from traditional algorithms. FIHC is a hierarchical clustering method developed for document clustering. Clustering or cluster analysis is the task of assigning a set of objects into groups, called clusters, so that the objects in the same cluster are more similar to each other than to those in other clusters. Clustering is a main topic of data mining algorithm, and a common technique for statistical data analysis used in many fields, including machine learning, image analysis, information retrieval, and bioinformatics.A major breakthrough of FIHC is that the clustering algorithm utilizes an important notion, frequent items...

show abstract

An Introduction to Text Mining

Cited by 153 publications

References 7 publications

Target Oriented Tweets Monitoring System during Natural Disasters

Target Oriented Tweets Monitoring System during Natural Disasters

Information aggregation and computational intelligence

A Hierarchical Document Clustering Approach with Frequent Itemsets

Contact Info

Product

Resources

About