From a Monolithic Big Data System to a Microservices Event-Driven Architecture

Laigner, Rodrigo; Kalinowski, Marcos; Diniz, Pedro C.; Barros, Leonardo; Cassino, Carlos; Lemos, Marcelo J. S. de; Arruda, Darlan; Lifschitz, Sérgio; Zhou, Yongluan

doi:10.1109/seaa51224.2020.00045

Cited by 27 publications

(15 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We opted to start with the collection of information from the literature, following the guidelines of [39]. A total of more than 300 papers retrieved from different digital libraries, such as Scopus and Scholar, were analyzed and a set of 10 papers [4,15,17,31,42,43,49,50,66,78] were able to meet our selection criteria. Even though very few papers provided detailed enough information about data management in microservices, we believe these studies are strongly representative of the peer-reviewed literature.…”

Section: Methodsmentioning

confidence: 99%

“…We highlight the following important observations. Functional partitioning: To support scalability (i.e., spreading functional groups across databases) and high data availability (i.e., achieving functional isolation of errors), functional decomposition of the application is a major driver for adopting microservices according to both survey respondents (57%) and the literature [15,17,31,43,49,50,66]. These results suggest that the design of data management technologies for microservices should focus not only on scalability, but also on stronger mechanisms for fault-tolerance and error isolation.…”

Section: Motivationsmentioning

confidence: 99%

“…A set of 23 participants consider event-driven data management a primary motivation for microservices, which may indicate a trend in industry adoption. Most papers [15,31,42,43,49,50,66,78] in the literature mention the use of asynchronous primitives for message-oriented communication to achieve loose coupling among microservices and decentralized data management. However, while open-source repositories [3,38,53,69,70] present use cases that rely on the generation of event streams for further processing, using microservices for stream processing is not significantly covered in literature.…”

Section: Motivationsmentioning

confidence: 99%

“…Hohpe [34] argues that orchestration and choreography are the two types of conversations that take place in the context of distributed web applications. Most papers [4,31,42,43,49,50,66] and open-source projects [3,19,38,52,69,70,72,81] report the use of the choreography conversation pattern [34] through both synchronous workflows and asynchronous event-based workflows.…”

Section: Business Transactions Across Microservicesmentioning

confidence: 99%

“…While one may argue that queries spanning multiple microservices are antagonistic to the principles of state encapsulation and independent data silos of microservices, we found abundant evidence of such cases in the literature [4,15,43,78] and open-source repositories [3,53,69,70,72]. Therefore, to further understand this trend, we asked the survey participants to identify if they have implemented queries aggregating data from multiple microservices and to describe one of their use cases.…”

Section: Online Queriesmentioning

confidence: 99%

See 4 more Smart Citations

Data Management in Microservices: State of the Practice, Challenges, and Research Directions

Laigner,

Zhou,

Salles

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We are recently witnessing an increased adoption of microservice architectures by the industry for achieving scalability by functional decomposition, fault-tolerance by deployment of small and independent services, and polyglot persistence by the adoption of different database technologies specific to the needs of each service. Despite the accelerating industrial adoption and the extensive research on microservices, there is a lack of thorough investigation on the state of the practice and the major challenges faced by practitioners with regard to data management.To bridge this gap, this paper presents a detailed investigation of data management in microservices. Our exploratory study is based on the following methodology: (i) we conducted a systematic literature review of articles reporting the adoption of microservices in industry settings, where more than 300 articles were filtered down to 11 representative studies; (ii) we analyzed a set of 9 popular open-source microservice-based applications, selected out of more than 20 open-source projects; furthermore, (iii) to strengthen our evidence, we conducted an online survey that we then used to crossvalidate the findings of the previous steps with the perceptions and experiences of over 120 practitioners and researchers.Through this process, we were able to categorize the state of practice and reveal several principled challenges that cannot be solved by software engineering practices, but rather need systemlevel support to alleviate the burden of practitioners. Based on the observations we also identified a series of research directions to achieve this goal. Fundamentally, novel database systems and data management tools that support isolation for microservices, which include fault isolation, performance isolation, data ownership, and independent schema evolution across microservices must be built to address the needs of this growing architectural style.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Motivationsmentioning

confidence: 99%

Section: Motivationsmentioning

confidence: 99%

Section: Business Transactions Across Microservicesmentioning

confidence: 99%

Section: Online Queriesmentioning

confidence: 99%

See 3 more Smart Citations

Data Management in Microservices: State of the Practice, Challenges, and Research Directions

Laigner,

Zhou,

Salles

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Advanced discovery mechanisms in model repositories

Indamutsa,

Di Rocco,

Almonte

et al. 2024

Softw Pract Exp

View full text Add to dashboard Cite

SummaryAs model‐driven engineering gains traction and poses as the new paradigm for software engineering, it raises a need for efficient approaches and tools to manage, discover, and retrieve relevant modeling artifacts. Hence, industry and academia are conceiving effective ways to store, search, and retrieve heterogeneous model artifacts that employ advanced discovery mechanisms. This paper presents MDEForge‐Search, a novel approach to discovering heterogeneous model artifacts over MDEForge, a distributed cloud‐based model repository. We designed advanced discovery mechanisms that retrieve heterogeneous artifacts within their context (megamodel) and reuse them across model management services. In addition, a domain‐specific approach has been proposed to formulate queries in terms of keywords, search tags, conditional operators, quality model assessment services and a transformation chain discoverer. Finally, the applicability of our approach was assessed in a recommender system modeling framework, which, thanks to the operated integration, can rely on the availability of more than 5000 model artifacts currently persisted in our cloud‐based model repository.

show abstract

The Adoption of Microservices Architecture as a Natural Consequence of Legacy System Migration at Police Intelligence Department

Almeida

Canedo

2022

Computational Science and Its Applications – ICCSA 2022

View full text Add to dashboard Cite

From a Monolithic Big Data System to a Microservices Event-Driven Architecture

Cited by 27 publications

References 14 publications

Data Management in Microservices: State of the Practice, Challenges, and Research Directions

Data Management in Microservices: State of the Practice, Challenges, and Research Directions

Advanced discovery mechanisms in model repositories

The Adoption of Microservices Architecture as a Natural Consequence of Legacy System Migration at Police Intelligence Department

Contact Info

Product

Resources

About