Positive and negative association rule mining in Hadoop’s MapReduce environment

Bagui, Sikha; Dhar, Probal Chandra

doi:10.1186/s40537-019-0238-8

Cited by 24 publications

(16 citation statements)

References 26 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because the data size and complexity increase rapidly, an efficient method is required for mining such rules. Most of the past studies focused on positive association rule mining; therefore, S. Bagui [26] proposed a method to efficiently discover both positive and negative association rules, simultaneously, from big data, using Hadoop's MapReduce architecture.…”

Section: B Distributed Algorithms For Discovery Of Frequent Patternsmentioning

confidence: 99%

A Distributed Method for Fast Mining Frequent Patterns From Big Data

et al. 2021

View full text Add to dashboard Cite

In recent years, knowledge discovery in databases provides a powerful capability to discover meaningful and useful information. For numerous real-life applications, frequent pattern mining and association rule mining have been extensively studied. In traditional mining algorithms, data are centralized and memory-resident. As a result of the large amount of data, bandwidth limitation, and energy limitations when applying these methods to distributed databases, especially in this era of big data, the performance is not effective enough. Hence, data mining on distributed environments has emerged as an important research area. To improve the performance, we propose a set of algorithms based on FP growth that discover FPs that are capable of providing fast and scalable service in distributed computing environments and a brief data structure to store items and counts to minimize the data for transmission on the network. To ensure completeness and execution capability, DistEclat and BigFIM were considered for the experiment comparison. Experiments show that the proposed method has superior cost-effectiveness for processing massive datasets and good capabilities under various experiment conditions. The proposed method on average required only 33% of the execution time and 45% of the transmission cost of DistEclat. Compared to BigFIM, The proposed method on average required 23.3% of the execution time and 14.2% of the transmission cost of BigFIM.

show abstract

Section: B Distributed Algorithms For Discovery Of Frequent Patternsmentioning

confidence: 99%

A Distributed Method for Fast Mining Frequent Patterns From Big Data

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In this study, we evaluated whether co-injections were associated with low SORs and if production regions were associated with high solution gas oil ratios (SGORs). The association rule had three parameters: support, confidence, and lift [46,47]. In this context, the association rule can be written as:…”

Section: Association Rulementioning

confidence: 99%

Discovering Energy Consumption Patterns with Unsupervised Machine Learning for Canadian In Situ Oil Sands Operations

Bai²,

2021

Sustainability

View full text Add to dashboard Cite

Canada’s in situ oil sands can help meet the global oil demand. Because of the energy-intensive extraction processes, in situ oil sands operations also play a critical role in meeting the global carbon budget. The steam oil ratio (SOR) is an indicator used to measure energy efficiency and assess greenhouse gas (GHG) emissions in the in situ oil sands industry. A low SOR indicates an extraction process that is more energy efficient and less carbon intensive. In this study, we applied machine learning methods for data-driven discovery to a public database⁠, Petrinex⁠, containing operating data from 2015 to 2019 extracted from over 35 million records for 20 in situ oil sands extraction operations. Two unsupervised machine learning methods, including clustering and association rules, showed that the cyclic steam stimulation (CSS) recovery method was less efficient than the steam-assisted gravity drainage (SAGD) recovery method. Chi-square tests showed a statistically significant association between the CSS recovery method and high SOR (p <0.005). Two association rules suggested that the occurrence of non-condensable gas (NCG) co-injection produced a low SOR. Chi-square tests on the two rules identified a statistically significant relationship between gas co-injection and low SOR (p < 0.005). Association rules also indicated that there was no association between the production regions and SORs. For future in situ oil sands development, decision-makers should consider SAGD as the preferred method because it is less carbon intensive. Existing in situ oil sands projects and future development should explore the possibility of NCG co-injection with steam to reduce steam consumption and consequently reduce GHG emissions from the extraction processes.

show abstract

“…The data integration capability of Hadoop enables it to integrate data from different sources, in different formats and characters into data lakes that support data input and output between multiple data sources and databases [21]. Bagui and Dhar [22] created a data warehouse using the Hadoop MapReduce framework on the Amazon AWS EMR. The data warehouse had 1.5 GB real life transactional dataset from 1.7 million web html documents mainly written in English and sourced from several websites.…”

Section: Big Data Modelsmentioning

confidence: 99%

A data model for enhanced data comparability across multiple organizations

PhD

Ogbuju

2020

J Big Data

View full text Add to dashboard Cite

Organizations may be related in terms of similar operational procedures, management, and supervisory agencies coordinating their operations. Supervisory agencies may be governmental or non-governmental but, in all cases, they perform oversight functions over the activities of the organizations under their control. Multiple organizations that are related in terms of oversight functions by their supervisory agencies, may differ significantly in terms of their geographical locations, aims, and objectives. To harmonize these differences such that comparative analysis will be meaningful, data about the operations of multiple organizations under one control or management can be cultivated, using a uniform format. In this format, data is easily harvested and the ease with which it is used for cross-population analysis, referred to as data comparability is enhanced. The current practice, whereby organizations under one control maintain their data in independent databases, specific to an enterprise application, greatly reduces data comparability and makes cross-population analysis a herculean task. In this paper, the collocation data model is formulated as consisting of big data technologies beyond data mining techniques and used to reduce the heterogeneity inherent in databases maintained independently across multiple organizations. The collocation data model is thus presented as capable of enhancing data comparability across multiple organizations. The model was used to cultivate the assessment scores of students in some schools for some period and used to rank the schools. The model permits data comparability across several geographical scales among which are: national, regional and global scales, where harvested data form the basis for generating analytics for insights, hindsight, and foresight about organizational problems and strategies.

show abstract

Positive and negative association rule mining in Hadoop’s MapReduce environment

Cited by 24 publications

References 26 publications

A Distributed Method for Fast Mining Frequent Patterns From Big Data

A Distributed Method for Fast Mining Frequent Patterns From Big Data

Discovering Energy Consumption Patterns with Unsupervised Machine Learning for Canadian In Situ Oil Sands Operations

A data model for enhanced data comparability across multiple organizations

Contact Info

Product

Resources

About