Repairing and Optimizing Hadoop hashCode Implementations

Kocsis, Zoltan A.; Neumann, Geoffrey; Swan, Jerry; Epitropakis, Michael G.; Brownlee, Alexander E. I.; Haraldsson, Sæmundur Ó.; Bowles, Edward

doi:10.1007/978-3-319-09940-8_22

Cited by 13 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kocsis et al [107], [108] and Burles et al [109] proposed to use semantics-preserving transformations within their improvement framework in order to retain full functionality of the original code. Therefore, this work also fits within the field of program transformation [110], [111] (see Section V-C for details).…”

Section: Survey Methodologymentioning

confidence: 99%

See 1 more Smart Citation

Genetic Improvement of Software: A Comprehensive Survey

Petke

Haraldsson

Harman

et al. 2018

IEEE Trans. Evol. Computat.

171

126

View full text Add to dashboard Cite

Genetic improvement uses automated search to find improved versions of existing software. We present a comprehensive survey of this nascent field of research with a focus on the core papers in the area published between 1995 and 2015. We identified core publications including empirical studies, 96% of which use evolutionary algorithms (genetic programming in particular). Although we can trace the foundations of genetic improvement back to the origins of computer science itself, our analysis reveals a significant upsurge in activity since 2012. Genetic improvement has resulted in dramatic performance improvements for a diverse set of properties such as execution time, energy and memory consumption, as well as results for fixing and extending existing system functionality. Moreover, we present examples of research work that lies on the boundary between genetic improvement and other areas, such as program transformation, approximate computing, and software repair, with the intention of encouraging further exchange of ideas between researchers in these fields.Index Terms-genetic improvement, survey 1089-778X (c)

show abstract

Section: Survey Methodologymentioning

confidence: 99%

“…Kocsis et al [107] exploited available contracts for the Java equals and hash methods to probe existing Hadoop code for violations. Simple program transformations were used to repair violations, before a metaheuristic was used to further optimise their repairs, in order to improve the quality of hash functions.…”

Section: B Software Repairmentioning

confidence: 99%

Genetic Improvement of Software: A Comprehensive Survey

Petke

Haraldsson

Harman

et al. 2018

IEEE Trans. Evol. Computat.

171

126

View full text Add to dashboard Cite

show abstract

“…Automatic program repair can be considered an improvement of a functional property, like improving the quality of hash code functions in Hadoop [15] or grafting new features to existing software [14,27]. However, optimizing attributes like execution time, memory consumption and power consumption is generally considered an improvement of a non-functional property which spans another big part of the GI literature.…”

Section: Related Workmentioning

confidence: 99%

Exploring Fitness and Edit Distance of Mutated Python Programs

Haraldsson

Woodward

Brownlee

et al. 2017

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Genetic Improvement (GI) is the process of using computational search techniques to improve existing software e.g. in terms of execution time, power consumption or correctness. As in most heuristic search algorithms, the search is guided by fitness with GI searching the space of program variants of the original software. The relationship between the program space and fitness is seldom simple and often quite difficult to analyse. This paper makes a preliminary analysis of GI's fitness distance measure on program repair with three small Python programs. Each program undergoes incremental mutations while the change in fitness as measured by proportion of tests passed is monitored.We conclude that the fitnesses of these programs often does not change with single mutations and we also confirm the inherent discreteness of bug fixing fitness functions. Although our findings cannot be assumed to be general for other software they provide us with interesting directions for further investigation.

show abstract

“…Recent work in this area includes Kocsis et al [16], which yield a 10,000-fold speedup of database queries on terabyte datasets within the Apache Spark analytics framework by eliminating redundant database joins and other transformations. Kocsis et al also automatically repaired 451 systematic errors in the implementation of the Apache Hadoop HPC framework [17], whilst simultaneously significantly improving performance. In addition to the work improving Quicksort for energy efficiency mentioned in Section 2, Burles et al [6], also obtained a 24% improvement in energy consumption by optimising a single widely-used class, ImmutableMultimap, in Google's Guava collection library.…”

Section: Threats To Validitymentioning

confidence: 99%

Deep Parameter Tuning of Concurrent Divide and Conquer Algorithms in Akka

White

Joffe

Bowles

et al. 2017

Applications of Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

Abstract. Akka is a widely-used high-performance and distributed computing toolkit for fine-grained concurrency, written in Scala for the Java Virtual Machine. Although Akka elegantly simplifies the process of building complex parallel software, many crucial decisions that affect system performance are deferred to the user. Employing the method of Deep Parameter Tuning to extract embedded 'magic numbers' from source code, we use the CMA-ES evolutionary computation algorithm to optimise the concurrent implementation of three widely-used divide-and-conquer algorithms within the Akka toolkit: Quicksort, Strassen's matrix multiplication, and the Fast Fourier Transform.

show abstract

Repairing and Optimizing Hadoop hashCode Implementations

Cited by 13 publications

References 5 publications

Genetic Improvement of Software: A Comprehensive Survey

Genetic Improvement of Software: A Comprehensive Survey

Exploring Fitness and Edit Distance of Mutated Python Programs

Deep Parameter Tuning of Concurrent Divide and Conquer Algorithms in Akka

Contact Info

Product

Resources

About