The State of Big Data Reference Architectures: A Systematic Literature Review

Abstract

Big Data (BD) is a nascent term emerged to describe large amount of data that comes in different forms from various channels. In the modern world, users are the ceaseless generators of structured, semi-structured, and unstructured data that if gleaned and crunched precisely, will reveal game-changing patterns. While opportunities exist with BD, the unprecedented amount of data has brought traditional approaches to a bottleneck. The growth of data is outpacing technological and scientific advances in data analytics. According to multiple sources, it is estimated that 75% of the BD projects have failed within the last decade. Among the challenges, system development and data architecture are prominent. This paper aims to facilitate BD system development and architecture by conducting a systematic literature review on BD reference architectures (RAs). The primary goal is to highlight the state of BD RAs and how they can be helpful for BD system development. The secondary goal is to find all BD RAs, describe the challenges of creating these RAs, discuss the common architectural components of these RA and the limitations of these RA. As a result of this work, firstly major concepts about RA are discussed and their applicability to BD system development is depicted. Secondly, 22 BD reference architecture is assessed from academia and practice and their commonalities, challenges, and limitations are identified. The findings gained emerges the understanding that RAs can be an effective artefact to tackle complex BD system development.

NeoMycelia: A software reference architecturefor big data systems

“…For instance, the standard has bolded the identification of system stakeholders (clause 5), however RAs are highly abstract and do not have a clear group of stakeholders [30]. Another focal point in conveyance of architectural descriptions is the concept of views.…”

Exploiting Big Data for Experiment Reporting: The Hi-Drive Collaborative Research Project Case

“…Khalajzadeh et al [ 18 ] presented a survey and analysis of several current research and practice approaches to support data analytics for end-users. Ataei and Litchfield [ 19 ] conducted a systematic literature review on big data reference architectures. They highlighted major challenges for research such as micro-services architectures, event-driven paradigms, security and privacy issues, and metadata management.…”

Massadata-arkkitehtuuri sosiaali- ja terveydenhuollossa: Tiedonhallinnan yleislainsäädännöstä seuraavat vaatimukset henkilötietojen käsittelylle

“…Massadatasta on laadittu useita referenssiarkkitehtuureja, jotka voidaan jaotella akateemisiin ja käytännöllisiin kuvauksin [24]. Tähän tutkimukseen valittiin analyysin tueksi yhdysvaltalaisen viranomaisen National Institute of Standards and Technologyn laatima ja ylläpitämä NIST Big Data Reference Architecture [25].…”

“…Kuvaus yhdistää sekä käytäntöä, että akateemisia lähtökohtia. Useimmat referenssiarkkitehtuurit ovat laadittu vain toisesta lähtökohdasta [24]. Alkuperästään huolimatta kuvauksen hyödyntämiselle Euroopassa ei ole esteitä.…”