Pre-Processing Of Medical Documents And Reducing Dimensionality

Abstract

The exponential growth of online repositories in medical science has led to the development of various text mining tool. Theses tools assist the users in analyzing text data stored in the online repositories like Pubmed and Medline. The pubmed repositories are growing at the rate of 500000 articles per year. Classification of Medline documents becomes very complex due to high dimensionality of feature space. In this study we discussed how dimensionality is reduced. We study and compared various dimensionality reduction techniques at the pre-processing stage. We introduce a novel feature weighting scheme 'GRW ' and proved that this schema improves classification accuracy. Our experimental results indicate that existing feature weighting methods has less accuracy rate when compared to GRW schema and tested on medical data set

Snake species identification by using natural language processing

“…This work involved the collection of the text-based description of snakes based on the presented snake images by using survey methods (questionnaire). Then, important features were extracted by using term frequencyinverse document frequency (TF-IDF), and these features were provided to machine learning algorithms to learn and predict the snake species using Weka tool [12].…”