Selective and Recurring Re-computation of Big Data Analytics Tasks: Insights from a Genomics Case Study

Cała, Jacek; Missier, Paolo

doi:10.1016/j.bdr.2018.06.001

Cited by 3 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dealing with all this data is challenging as traditional methods of analysis such as guessing, constructing hypothesis and testing with data based experiments do not perform well [22] because of the sheer volume and variety of data, hence new methods uncovering the insights in data must be devised. Research in Big Data is considering new techniques [23][24][25][26][27] with the core challenge of how to process data to extract useful information. Computational resources seem to be second runners-up in this race with Big Data leading the way.…”

Section: The Inherent Problemmentioning

confidence: 99%

See 1 more Smart Citation

Countering Terrorism on Social Media Using Big Data

Al-Zahrani¹,

Bajwa²,

Alghamdi³

et al. 2018

Computer Science &Amp; Information Technology (CS &Amp; IT)

View full text Add to dashboard Cite

Terrorism and violence are used by miscreant groups and individuals to disrupt the normal course of events. While not a new phenomenon, the information age offers new and innovative methods for spreading messages related to terrorism and expansion through recruitment on social media. These observations are alarming due to the broad reach and speed of propagation made available by social media. To ensure safety, harmony, and peace, it is important that the use of social media for terrorism is minimised. We discuss the various methods used by terrorists on social media to increase exposure and identify how the inherent structure of social media, the amount of data available, and language understanding pose challenges as well as opportunities for control efforts. We propose a strategy for restraining terrorist activities through data mining methods on big data created by social media in combination with natural language processing for language understanding and social network analysis for uncovering the underlying structure and association of terrorist groups and their activities

show abstract

Section: The Inherent Problemmentioning

confidence: 99%

“…Computational resources seem to be second runners-up in this race with Big Data leading the way. Classic learning and intelligence methods also fail to perform well on large data set, and new "Deep Learning" techniques [23,24] are being devised to take advantage of the available information.…”

Section: The Inherent Problemmentioning

confidence: 99%

Countering Terrorism on Social Media Using Big Data

Al-Zahrani¹,

Bajwa²,

Alghamdi³

et al. 2018

Computer Science &Amp; Information Technology (CS &Amp; IT)

View full text Add to dashboard Cite

show abstract

Provenance Annotation and Analysis to Support Process Re-computation

Cała

Missier

2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Many resource-intensive analytics processes evolve over time following new versions of the reference datasets and software dependencies they use. We focus on scenarios in which any version change has the potential to affect many outcomes, as is the case for instance in high throughput genomics where the same process is used to analyse large cohorts of patient genomes, or cases. As any version change is unlikely to affect the entire population, an efficient strategy for restoring the currency of the outcomes requires first to identify the scope of a change, i.e., the subset of affected data products. In this paper we describe a generic and reusable provenance-based approach to address this scope discovery problem. It applies to a scenario where the process consists of complex hierarchical components, where different input cases are processed using different version configurations of each component, and where separate provenance traces are collected for the executions of each of the components. We show how a new data structure, called a restart tree, is computed and exploited to manage the change scope discovery problem.

show abstract

Report on the First International Workshop on Incremental Re-computation

2019

View full text Add to dashboard Cite

In the last decade, advances in computing have deeply transformed data processing. Increasingly systems aim to process massive amounts of data efficiently, often with fast response times that are typically characterised by the 4V's, i.e., Volume, Variety, Velocity, and Veracity. While fast data processing is desirable, it is also often the case that the outcomes of computationally expensive processes become obsolete over time, due to changes in inputs, reference datasets, tools, libraries, and deployment environment. Given massive data processing, such changes must be carefully accounted for, and their impact on original computation assessed, to determine how much re-computation is needed in response to changes.

show abstract

Selective and Recurring Re-computation of Big Data Analytics Tasks: Insights from a Genomics Case Study

Cited by 3 publications

References 36 publications

Countering Terrorism on Social Media Using Big Data

Countering Terrorism on Social Media Using Big Data

Provenance Annotation and Analysis to Support Process Re-computation

Report on the First International Workshop on Incremental Re-computation

Contact Info

Product

Resources

About