Developing software systems to Big Data platform based on MapReduce model: An approach based on Model Driven Engineering

Osvaldo, S. Sousa; Lopes, Denivaldo; Silva, Aristófanes Corrêa; Abdelouahab, Zair

doi:10.1016/j.infsof.2017.07.006

Cited by 11 publications

(11 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the states sta:loadErr and sta:saveErr widgets wdg:ShowError generate an error message when loading and saving data [10]. These messages are inserted into the same point in the HTML of the parent widget, into which the result is inserted in the absence of errors.…”

Section: Json To Xml Transformation Examplementioning

confidence: 99%

“…The development of modern web applications is strongly influenced by new approaches such as Service-Oriented that estimated in work [6] in aspect of characteristics of Model-Driven Architectures which use domain-specific languages [7]. Recently, the experience of creating situation-oriented web applications has become relevant [8], applicable for NoSQL in model-driven [9] database development such as SODB together with Big Data technologies [10].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

JSON Documents Processing Using Situation-Oriented Databases

Mironov¹,

Гусаренко²,

Yusupova³

et al. 2020

ACTA POLYTECH HUNG

View full text Add to dashboard Cite

Situation-oriented databases provide processing of documents from heterogeneous data sources under the control of a hierarchical situational model. This article discusses the problem of processing database documents in JSON format, along with XML. Two implementation approaches are discussed: (1) on the fly JSON to XML document conversion and using Document Object Model for processing XML, and (2) loading the JSON document into an associative/indexed array followed by applying the template engine. The database interpreter works with external heterogeneous data extracted from files, databases, archives, web services, data is processed using virtual documents. Examples of processing JSON documents received from a web service are analyzed. Data from the San Francisco Open Data web server is used as the JSON test source. Query in Socrata Query Language used for JSON data extraction is presented. The implementation of approaches in the research situation-oriented database prototype based on Hypertext Preprocessor is considered.

show abstract

Section: Json To Xml Transformation Examplementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

JSON Documents Processing Using Situation-Oriented Databases

Mironov¹,

Гусаренко²,

Yusupova³

et al. 2020

ACTA POLYTECH HUNG

View full text Add to dashboard Cite

show abstract

“…In addition, it is assumed that these portions of data can be cached in RAM. That is, "big data" [12][13][14][15] is considered, which is a large set of similar elements. Moreover, each element has a foreseeable size that allows cached processing.…”

Section: The Data Stream Modelmentioning

confidence: 99%

Stream Documents Processing Invariance in Situation-Oriented Databases

Mironov¹,

Гусаренко²,

Yusupova³

2019

Proceedings of the 7th Scientific Conference on Information Technologies for Intelligent Decision Making Support (ITIDS 2019)

View full text Add to dashboard Cite

Data stream processing in situational-oriented databases (SODB) is discussed. SODB is a heterogeneous data integrator operating under the control of the built-in hierarchical situational model (HSM). The processed data is defined in the HSM as virtual documents (VD) mapped to real data stores. HSM specifies the loading of a VD data into Data Processing Objects (DPO), the data transformation in DPO, and unloading of processing results to a VD. Stream processing of large documents that do not fit entirely into RAM is discussed. A VD model involving multiple pieces of data that can be processed separately is considered. Portions of data are extracted from the VD input stream, processed in the DPO, and then sent to the VD output stream. Invariance should ensure the independence of the DPO model when the VD model changes. Invariance to different schemes streaming data is discussed. Algorithms for loading and unloading DPO are considered, ensuring invariance due to the hidden cyclical processing of data portions. The proposed solutions are illustrated with examples of processing XML and JSON documents.

show abstract

“…Among them, workflow orchestration frameworks are placed on top of distributed data processing applications [16], modeldriven approaches are adopted to configure run time observability frameworks [17], to configure machine learning [5], or to design visual analytics [18]. An example of the application of these principles to Big Data technologies is proposed in [19], [6], [20].…”

Section: Related Workmentioning

confidence: 99%

A Fast and Incremental Development Life Cycle for Data Analytics as a Service

Ardagna

Bellandi

Ceravolo

et al. 2018

2018 IEEE International Congress on Big Data (BigData Congress)

View full text Add to dashboard Cite

Big Data does not only refer to a huge amount of diverse and heterogeneous data. It also points to the management of procedures, technologies, and competencies associated with the analysis of such data, with the aim of supporting high-quality decision making. There are, however, several obstacles to the effective management of a Big Data computation, such as data velocity, variety, and veracity, and technological complexity, which represent the main barriers towards the full adoption of the Big Data paradigm. The goal of this work is to define a new software Development Life Cycle for the design and implementation of a Big Data computation.Our proposal integrates two model-driven methods: a first method based on pre-configured services that reduces the cost of deployment and a second method based on custom component development that provides an incremental process of refinement and customization. The proposal is experimentally evaluated by clustering a data set of the distribution of the population in the United States based on contextual criteria.

show abstract

Developing software systems to Big Data platform based on MapReduce model: An approach based on Model Driven Engineering

Cited by 11 publications

References 26 publications

JSON Documents Processing Using Situation-Oriented Databases

JSON Documents Processing Using Situation-Oriented Databases

Stream Documents Processing Invariance in Situation-Oriented Databases

A Fast and Incremental Development Life Cycle for Data Analytics as a Service

Contact Info

Product

Resources

About