Towards Specification of a Software Architecture for Cross-Sectoral Big Data Applications

Abstract

The proliferation of Big Data applications puts pressure on improving and optimizing the handling of diverse datasets across different domains. Among several challenges, major difficulties arise in data-sensitive domains like banking, telecommunications, etc., where strict regulations make very difficult to upload and experiment with real data on external cloud resources. In addition, most Big Data research and development efforts aim to address the needs of IT experts, while Big Data analytics tools remain unavailable to non-expert users to a large extent. In this paper, we report on the work-in-progress carried out in the context of the H2020 project I-BiDaaS (Industrial-Driven Big Data as a Self-service Solution) which aims to address the above challenges. The project will design and develop a novel architecture stack that can be easily configured and adjusted to address crosssectoral needs, helping to resolve data privacy barriers in sensitive domains, and at the same time being usable by non-experts. This paper discusses and motivates the need for Big Data as a self-service, reviews the relevant literature, and identifies gaps with respect to the challenges described above. We then present the I-BiDaaS paradigm for Big Data as a self-service, position it in the context of existing references, and report on initial work towards the conceptual specification of the I-BiDaaS software architecture.

Towards a Multi-Perspective Methodology for Big Data Requirements

“…In the context of the H2020 project I-BiDaaS [7], [8], this initial classification was further elaborated in a top-down manner, based on the review of related research and was revised in a bottom-up manner through the generalization of over 300 specific requirements collected in the context of 9 real use case scenarios within three industrial sectors (telecommunications, banking and manufacturing). This resulted in 24 generic big data requirement categories in the above perspectives.…”

The OTree: Multidimensional Indexing with efficient data Sampling for HPC

“…Many authors [1] [2] [3] [4] pointed out how multidimensional and spatial big data will be an essential part of future scientific and business applications. In particular, Eldawy et al [1] described how we are entering the "Era of Big Spatial Data", with a single space telescope generating up to 150 GB of spatial data per week [5], medical devices producing spatial images at a rate of 50 PB per year and social networks managing billions of geo-tagged events per day.…”