Facing Big Data System Architecture Deployments: Towards an Automated Approach Using Container Technologies for Rapid Prototyping

Abstract

Within the last decade, big data became a promising trend for many application areas, offering immense potential and a competitive edge for various organizations. As the technical foundation for most of today´s data-intensive projects, not only corresponding infrastructures and facilities but also the appropriate knowledge is required. Currently, several projects and services exist that not only allow enterprises to utilize but also to deploy related technologies and systems. However, at the same time, the use of these is accompanied by various challenges that may result in huge monetary expenditures, a lack of modifiability, or the risk of vendor lock-ins. To overcome these shortcomings, in the contribution at hand, modern container and task automation technologies are used to wrap complex big data technologies into re-usable and portable resources.