Investigation of technologies of processing of big data

Abstract

An overview of the technologies and methods is presented, on the basis of which the authors of this article model the processing of a large amount of data, developing a web application. In particular, it is proposed to combine models to improve the efficiency of processing large amounts of data. Large data set before traditional storage systems and processing a new challenge. This article analyzes possible methods their decisions, limitations that do not allow to do it effectively, and also provides an overview of three modern approaches to working with large data: NoSQL and real-time event flow processing. Analysis of large data requires the use of technology and the means to implement highly productive computing. The main factors of the problem are, first of all, the complexity and the second physical volume of the information collection. It should be noted that the actual processing of data includes the construction of the algorithm and the time for its description and debugging. Unique data collections require the development of unique algorithms, which increases the total processing time by an order of magnitude.

Development of an Application for the Thermal Processing of Oil Slime in the Industrial Oil and Gas Sector

“…A mathematical model and computer simulation of the heat conduction process and the implementation of the program code that graphically displays the results of the model, that is, the construction of a digital model of the process, for subsequent use as a module, is described here including processing of the graphical representation of data [7]. To achieve this goal, the following tasks are solved:  Analysis of the subject area using various information sources of scientific content (books, articles, research papers, etc.).…”

Development of an Application for the Thermal Treatment of Oil Slime

“…Математикалық модель [2] және жылуөткізгіштік процесін компьютерлік модельдеу және модель нәтижелерін графикалық түрде бейнелейтін программалық кодты жүзеге асыру, яғни процесстің цифрлық моделін құру, кейіннен модуль ретінде пайдалану, мұнда деректердің графикалық көрінісін өңдеуді қоса сипатталған. Осы мақсатқа жету үшін келесі міндеттер шешіледі:…”

Development of Employee Rating Analysis Model

“…User`s rating data was stored in "ratings" collections. Rating collection object properties have the following properities [6]: { _id: String, userId: String, createdAt: Date, deletedAt: Date, data: Array of objects } Here, the property is an array of individual rating data objects calculated for each month, starting from the month "createdAt" (Fig. 5).…”