Towards Large Scale Semantic Annotation Built on MapReduce Architecture

Abstract

Abstract. Automated annotation of the web documents is a key challenge of the Semantic Web effort. Web documents are structured but their structure is understandable only for a human that is the major problem of the Semantic Web. Semantic Web can be exploited only if metadata understood by a computer reach critical mass. Semantic metadata can be created manually, using automated annotation or tagging tools. Automated semantic annotation tools with the best results are built on different machine learning algorithms requiring training sets. Another approach is to use pattern based semantic annotation solutions built on NLP, information retrieval or information extraction methods. Most of developed methods are tested and evaluated on hundreds of documents which cannot prove its real usage on large scale data such as web or email communication in enterprise or community environment. In this paper we present how a pattern based annotation tool can benefit from Google's MapReduce architecture to process large amount of text data.

CRFs based parallel biomedical named entity recognition algorithm employing MapReduce framework

“…Now it is popular in text mining of various applications [18], especially Natural Language Processing (NLP) and Machine Learning (ML), as the MapReduce paradigm has emerged as a highly successful programing model for large-scale data-intensive computing applications [19]. Laclavik et al presented a pattern of annotation tool based on MapReduce architecture to process large amount of text data [20]. Lin and Dyer discussed the processing method of data intensive text based on MapReduce, such as parallelization of EM algorithm and HMM model [4].…”

GATECloud.net: a platform for large-scale, open-source text processing on the cloud

“…They commented that UIMA and GATE would benefit from adopting MapReduce. Laclavik et al [16] demonstrated using Ontea [17] with Hadoop. This study presents GATECloud.net-the adaptation of the GATE infrastructure to the cloud, following the PaaS paradigm. It enables researchers to run their NLP applications without the significant overheads of re-implementing their algorithms for MapReduce and understanding Amazon's IaaS APIs.…”

Cognitive traveling in digital space: from keyword search through exploratory information seeking

“…Interested fellows leave traces in the digital space, sometimes even without being aware of it. For example: evaluations, recommendations, annotations, inscriptions on a virtual wall [45,46]. Interested fellows communicate with others, forming communities of those sharing interests.…”