How deep data becomes big data

Abstract

We present some problems and solutions for situations when compound and semantically rich nature of data records, such as scientific articles, creates challenges typical for big data processing. Using a case study of named entity matching in SONCA system we show how big data problems emerge and how they are solved by bringing together methods from database management and computational intelligence.

Networks of compound object comparators

“…The above case study is an example of utilization of a well-known strategy of solving complex computational tasks, where one first heuristically partitions a problem onto more feasible pieces, then conducts detailed computations over such pieces, and finally aggregates local results [18], [19]. In our case, partitioning is performed by the first layer of comparators over the Cartesian product A × B of input and reference objects, while detailed calculations refer to comparators in the second layer.…”

“…Our system is called SONCA (Search based on ON tologies and C ompound Analytics; see e.g. [17], [18]). In SONCA, each acquired document is stored in a parsed form in a data warehouse called SoncaDB, in order to make the best possible use of the available structural and semantic information while searching, filtering, grouping, etc.…”

“…Let us now go back to the task of matching (deduplicating) the instances of scientists who occur in the SoncaDB [17], [18] database as the authors of loaded scientific publications. Let us note that, in some cases, it is difficult to extract from the input texts describing publications their components corresponding to their authors, titles, dates and other relevant properties.…”

Deep Data Example: Zbiva, Early Medieval Data Set for the Eastern Alps

“…The term Deep Data describes data that is not very big, but semantically very rich, i.e., it provides contextually rich information intended to provide a rich user experience. The term Deep Data was proposed by analogy with the term Deep Web, which is used to describe the vast amount of information that resides beneath the surface of web pages (Szczuka & Ślȩzak, 2013). The term Deep Data has also been used to describe an approach where all the information available in the data is fully exploited to gain knowledge (Belianinov et al, 2015).…”

Window-Based Feature Extraction Framework for Multi-Sensor Data: A Posture Recognition Case Study

“…• 2199 -one sliding window per short time series • 6315 -two sliding windows per short time series • 18663 -five sliding windows per short time series Making a total of 27177 attributes [27] from the conditionaland inter-sliding windows constructed for both raw and virtual sensors. Elements of the automatic feature selection and reducing the number of attributes are in the study phase and still have not been introduced to the data processing mechanisms.…”