AROM: Processing big data with Data Flow Graphs and functional programming

Tran, Nam-Luc; Skhiri, Sabri; Lesuisse, Arthur; Zimányi, Esteban

doi:10.1109/cloudcom.2012.6427487

Cited by 6 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some authors have already pointed out the necessity of parallel and distributed programming for handling the big data sets in the general context or even in the geospatial context (Lee et al 2014, Shekhar et al 2012. Others have pointed out the usefulness of functional programming concepts or languages such as Haskell Domain-Specific Language (Mintchev 2014), Map-reduce (Maitrey & Jha 2015, Mohammed et al 2014, Data Flow Graphs (Tran et al 2012), or self-adjusting computation (Acar & Chen 2013). However, there is a gap between the research works that advocate functional programming techniques but do not handle specifically geospatial data, and research works that focus on geospatial big data, but do not guarantee the absence of data races (which are the races of different threads to gain access to the same data item in some shared memory [Milewski 2009]).…”

Section: Data Modelling and Structuringmentioning

confidence: 99%

Geospatial big data handling theory and methods: A review and research challenges

Dragićević

Castro

et al. 2016

ISPRS Journal of Photogrammetry and Remote Sensing

407

218

View full text Add to dashboard Cite

Big data has now become a strong focus of global interest that is increasingly attracting the attention of academia, industry, government and other organizations. Big data can be situated in the disciplinary area of traditional geospatial data handling theory and methods. The increasing volume and varying format of collected geospatial big data presents challenges in storing, managing, processing, analyzing, visualizing and verifying the quality of data. This has implications for the quality of decisions made with big data. Consequently, this position paper of the International Society for Photogrammetry and Remote Sensing (ISPRS) Technical Commission II (TC II) revisits the existing geospatial data handling methods and theories to determine if they are still capable of handling emerging geospatial big data. Further, the paper synthesises problems, major issues and challenges with current developments as well as recommending what needs to be developed further in the near future.

show abstract

Section: Data Modelling and Structuringmentioning

confidence: 99%

Geospatial big data handling theory and methods: A review and research challenges

Dragićević

Castro

et al. 2016

ISPRS Journal of Photogrammetry and Remote Sensing

407

218

View full text Add to dashboard Cite

show abstract

“…The design of our tool relies on recent advances in graph database technology and graph computing: once a Java project has been analyzed by Arcan, a new graph database is created containing the structural dependencies of the system. Thanks to graph computing and connected big data processing technology [5], it is then possible to run detection algorithms on this graph to extract information about the analyzed project (package/class metrics, architectural issues).…”

Section: Introductionmentioning

confidence: 99%

Arcan: A Tool for Architectural Smells Detection

Fontana

Pigazzini

Roveda

et al. 2017

2017 IEEE International Conference on Software Architecture Workshops (ICSAW)

View full text Add to dashboard Cite

Abstract-Code smells are sub-optimal coding circumstances such as blob classes or spaghetti code -they have received much attention and tooling in recent software engineering research. Higher-up in the abstraction level, architectural smells are problems or sub-optimal architectural patterns or other design-level characteristics. These have received significantly less attention even though they are usually considered more critical than code smells, and harder to detect, remove, and refactor. This paper describes an open-source tool called Arcan developed for the detection of architectural smells through an evaluation of several architecture dependency issues. The detection techniques inside Arcan exploit graph database technology, allowing for high scalability in smells detection and better management of large amounts of dependencies of multiple kinds. In the scope of this paper, we focus on the evaluation of Arcan results carried out with real-life software developers to check if the architectural smells detected by Arcan are really perceived as problems and to get an overall usefulness evaluation of the tool.

show abstract

“…pagerank(graph)). Therefore, it seems that functional [50], data flow [68] and domain specific [29] graph analysis languages are being created to ease our graph processing tasks.…”

Section: Other Aspectsmentioning

confidence: 99%

Systems for Near Real-Time Analysis of Large-Scale Dynamic Graphs

Vaquero¹,

Cuadrado²,

Ripeanu³

2014

Preprint

View full text Add to dashboard Cite

Graphs are widespread data structures used to model a wide variety of problems. The sheer amount of data to be processed has prompted the creation of a myriad of systems that help us cope with massive scale graphs. The pressure to deliver fast responses to queries on the graph is higher than ever before, as it is demanded by many applications (e.g. online recommendations, auctions, terrorism protection, etc.). In addition, graphs change continuously (so do the real world entities that typically represent). Systems must be ready for both: near real-time and dynamic masive graphs. We survey systems taking their scalability, real-time potential and capability to support dynamic changes to the graph as driving guidelines. The main techniques and limitations are distilled and categorised. The algorithms run on top of graph systems are not ready for prime time dynamism either. Therefore, a short overview on dynamic graph algorithms has also been included.

show abstract

AROM: Processing big data with Data Flow Graphs and functional programming

Cited by 6 publications

References 15 publications

Geospatial big data handling theory and methods: A review and research challenges

Geospatial big data handling theory and methods: A review and research challenges

Arcan: A Tool for Architectural Smells Detection

Systems for Near Real-Time Analysis of Large-Scale Dynamic Graphs

Contact Info

Product

Resources

About