Semantic HMC: A Predictive Model Using Multi-label Classification for Big Data

Abstract

International audienceOne of the biggest challenges in Big Data is the exploitation of Value from large volume of data. To exploit value one must focus on extracting knowledge from Big Data sources. In this paper we present a new simple but highly scalable process to automatically learn the label hierarchy from huge sets of unstructured text. We aim to extract knowledge from these sources using a Hierarchical Multi-Label Classification process called Semantic HMC. Five steps compose the Semantic HMC: Indexation, Vectorization, Hierarchization, Resolution and Realization. The first three steps construct the label hierarchy from data sources. The last two steps classify new items according to the hierarchy labels. To perform the classification without heavily relying on the user, the process is unsupervised, where no thesaurus or label examples are required. The process is implemented in a scalable and distributed platform to process Big Data

Semantic HMC: Ontology-Described Hierarchy Maintenance in Big Data Context

“…In previous work [2] the hierarchy relations (i.e. the hierarchy created using a traditional subsumption method) are induced from a collection of documents using the co-occurrence of terms in documents.…”

“…The new documents from are included in the collection originating a new collection impacting the term co-occurrence and consequently the hierarchy. In order to persist the co-occurrence values, a term cooccurrence frequency matrix is used to represent the co-occurrence of any pair of terms in a collection of documents [2].…”

“…The term of Big Data is mainly used to describe datasets that cannot be processed using traditional tools. To extract knowledge from Big Data sources we propose to use a Semantic HMC process [1,2] that is capable of Hierarchically Multi-Classify a large Variety and Volume of unstructured data items. Hierarchical Multi-Label Classification (HMC) is the combination of Multi-Label classification and Hierarchical classification [13].…”