On the analysis of big data indexing execution strategies

Siddiqa, Aisha; Karim, Ahmad; Saba, Tanzila; Chang, Victor

doi:10.3233/jifs-169269

Cited by 4 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 5 displays the frequency of publications for each classification cluster proposed by [8]. It is worth mentioning that four papers on the Database Layer did not fall into the predefined clusters, given their genericity [7], or as an evaluation of different techniques [47]- [49].…”

Section: B Study Data Extractionmentioning

confidence: 99%

Interactive Data Exploration of Distributed Raw Files: A Systematic Mapping Study

2019

View full text Add to dashboard Cite

When exploring big amounts of data without a clear target, providing an interactive experience becomes really difficult, since this tentative inspection usually defeats any early decision on data structures or indexing strategies. This is also true in the physics domain, specifically in high-energy physics, where the huge volume of data generated by the detectors are normally explored via C++ code using batch processing, which introduces a considerable latency. An interactive tool, when integrated into the existing data management systems, can add a great value to the usability of these platforms. Here, we intend to review the current state-of-the-art of interactive data exploration, aiming at satisfying three requirements: access to raw data files, stored in a distributed environment, and with a reasonably low latency. This paper follows the guidelines for systematic mapping studies, which is well suited for gathering and classifying available studies. We summarize the results after classifying the 242 papers that passed our inclusion criteria. While there are many proposed solutions that tackle the problem in different manners, there is little evidence available about their implementation in practice. Almost all of the solutions found by this paper cover a subset of our requirements, with only one partially satisfying the three. The solutions for data exploration abound. It is an active research area and, considering the continuous growth of data volume and variety, is only to become harder. There is a niche for research on a solution that covers our requirements, and the required building blocks are there.INDEX TERMS Big data applications, data analysis, data engineering, data exploration, database systems, interactive systems, systematic mapping study. APPENDIX RESULTS OF THE MAPPING STUDYSee Tables.

show abstract

Section: B Study Data Extractionmentioning

confidence: 99%

Interactive Data Exploration of Distributed Raw Files: A Systematic Mapping Study

2019

View full text Add to dashboard Cite

show abstract

“…Sood et al [28] made an experimental evaluation on meteorological data collected that indicated the effectiveness of the proposed architecture. Siddiqa et al [29] analyzed advanced big data techniques to improve accuracy and performance of indexing mechanisms. In dam operations, Coppolino et al [30] presented a Security Information and Event Management system to perform analysis of reports generated by security devices of dam infrastructure, and Coppolino et al [31] investigated a framework for event collection and correlation to process heterogeneous data and spot evidence of security issues.…”

Section: Introductionmentioning

confidence: 99%

Economic, Environmental and Social Gains of the Implementation of Artificial Intelligence at Dam Operations toward Industry 4.0 Principles

et al. 2020

View full text Add to dashboard Cite

Due to the increasing demand for water supply of urban areas, treatment and supply plants are becoming important to ensure availability and quality of this essential resource for human health. Enabling technologies of Industry 4.0 have the potential to improve performances of treatment plants. In this paper, after reviewing contributions in scientific literature on I4.0 technologies in dam operations, a study carried out on a Brazilian dam is presented and discussed. The main purpose of the study is to evaluate the economic, environmental, and social advantages achieved through the adoption of Artificial Intelligence (AI) in dam operations. Unlike automation that just respond to commands, AI uses a large amount of data training to make computers able to take the best decision. The current study involved a company that managed six reservoirs for treatment systems supplying water to almost ten million people at the metropolitan area of São Paulo City. Results of the study show that AI adoption could lead to economic gain in figures around US$ 51,000.00 per year, as well as less trips between sites and less overtime extra costs on the main operations. Increasing gates maneuvers agility result in significant environmental gains with savings of about 4.32 billion L of water per year, enough to supply 73,000 people. Also, decreasing operational vehicle utilization results in less emissions. Finally, the AI implementation improved the safety of dam operations, resulting in social benefits such as the flood risk mitigation in cities and the health and safety of operators.

show abstract

A survey on indexing techniques for mobility in Internet of Things': Challenges, performances, and perspectives

2020

View full text Add to dashboard Cite

SummaryMobile nodes arouse new challenges that expand the performance in those environments. The nodes of Internet of Things (IoT) generate a large amount of data, which have to be stored and processed in a seamless and interpretable form by indexing. Therefore, indexing is a challenge in IoT, particularly when the nodes are mobile. The current indexing techniques dedicated to mobile environments are unsatisfying because the data are transmitted from different locations within different time periods and randomness of sensor movement. Although huge research efforts have been dedicated to this subject amid the last decades, little consideration has been paid for the research summarization and guidance. The objective of this survey is to identify the relationship between activities of mobile sensors in the context of IoT, that concerned on transferring and collecting data as well as the effectiveness of indexing techniques. The contribution of this review is to investigate the techniques of mobile IoT nodes, to find the source of challenges that adversely affect the index effectiveness. In‐depth investigation and analyses of approaches to apply IoT mobile nodes will enable the researcher to understand the behavior of mobile environments to extract the deformities that adversely affect the indexing effectiveness. The analyzed approaches are meta‐heuristic, spatial‐temporal indexing, and prediction model approach. Each approach is analyzed by discussing the features and limitations from the mobility perspectives. Furthermore, the indexing techniques are analyzed according to their features and limitations and mobile IoT indexing requirements. In conclusion, recognize and contemplate different open issues that got little focus or still unknown at this point.

show abstract

On the analysis of big data indexing execution strategies

Cited by 4 publications

References 18 publications

Interactive Data Exploration of Distributed Raw Files: A Systematic Mapping Study

Interactive Data Exploration of Distributed Raw Files: A Systematic Mapping Study

Economic, Environmental and Social Gains of the Implementation of Artificial Intelligence at Dam Operations toward Industry 4.0 Principles

A survey on indexing techniques for mobility in Internet of Things': Challenges, performances, and perspectives

Contact Info

Product

Resources

About