Log-based slicing for system-level test cases

Messaoudi, Salma; Shin, Donghwan; Panichella, Annibale; Bianculli, Domenico; Briand, Lionel C.

doi:10.1145/3460319.3464824

Cited by 11 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Log slicing differs in that it actively tries to reduce the contents of a log. Finally, Messaoudi et al [8] have proposed a log-based test case slicing technique, which aims to decompose complex test cases into simpler ones using, in addition to program analysis, data available in logs.…”

Section: Related Workmentioning

confidence: 99%

Towards Log Slicing

Dawes

Shin

Bianculli

2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

This short paper takes initial steps towards developing a novel approach, called log slicing, that aims to answer a practical question in the field of log analysis: Can we automatically identify log messages related to a specific message (e.g., an error message)? The basic idea behind log slicing is that we can consider how different log messages are “computationally related” to each other by looking at the corresponding logging statements in the source code. These logging statements are identified by 1) computing a backwards program slice, using as criterion the logging statement that generated a problematic log message; and 2) extending that slice to include relevant logging statements.The paper presents a problem definition of log slicing, describes an initial approach for log slicing, and discusses a key open issue that can lead towards new research directions.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Log Slicing

Dawes

Shin

Bianculli

2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…An assertion-aware test-suite reduction technique has been proposed by Chen et al [12]. Messaoudi et al [49] use assertion-based backward slicing to decompose complex system test cases into smaller, separate ones. Petke and Blot [60] suggest to consider the output of test case assertions in fitness functions for test-based program repair using genetic algorithms.…”

Section: Specific Applicationsmentioning

confidence: 99%

“…Employing a search-based algorithm for improving assertions, Jahangirova [34] combine test case generation to reveal false positives and mutation testing to reveal false negatives. Test case execution logs are used in DS3 [49] to determine dependencies among test slices. Mutation analysis has been also used by Fraser and Zeller [28] to improve the fault detection capability of test oracles, by Knauth et al [39] to assess the quality of the assertions, and by Xie et al [87] for analyzing PUTs written by developers and identifying likely locations in PUTs for improvement.…”

Section: Techniquementioning

confidence: 99%

A Literature Survey of Assertions in Software Testing

Taromirad,

Runeson

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Assertions are one of the most useful automated techniques for checking program's behaviour and hence have been used for different verification and validation tasks. We provide an overview of the last two decades of research involving 'assertions' in software testing. Based on a term-based search, we filtered the inclusion of relevant papers and synthesised them w.r.t. the problem addressed, the solution designed, and the evaluation conducted. The survey rendered 119 papers on assertions in software testing. After test oracle, the dominant problem focus is test generation, followed by engineering aspects of assertions. Solutions are typically embedded in tool prototypes and evaluated throughout limited number of cases while using large-scale industrial settings is still a noticeable method. We conclude that assertions would be worth more attention in future research, particularly regarding the new and emerging demands (e.g., verification of programs with uncertainty), for effective, applicable, and domain-specific solutions.

show abstract

“…post-process optimization. Techniques like test slicing (Messaoudi et al 2021), carving (Elbaum et al 2006), and purification (Xuan and Monperrus 2014) have been applied to manually-written tests and could be potentially included in test case generation tools. 2.…”

Section: Research Opportunity In Test Case Generationmentioning

confidence: 99%

Test smells 20 years later: detectability, validity, and reliability

Panichella

Sawant

et al. 2022

Empir Software Eng

Self Cite

View full text Add to dashboard Cite

Test smells aim to capture design issues in test code that reduces its maintainability. These have been extensively studied and generally found quite prevalent in both human-written and automatically generated test-cases. However, most evidence of prevalence is based on specific static detection rules. Although those are based on the original, conceptual definitions of the various test smells, recent empirical studies indicate that developers perceive warnings raised by detection tools as overly strict and non-representative of the maintainability and quality of test suites. This leads us to re-assess test smell detection tools’ detection accuracy and investigate the prevalence and detectability of test smells more broadly. Specifically, we construct a hand-annotated dataset spanning hundreds of test suites both written by developers and generated by two test generation tools (EvoSuite and JTExpert) and performed a multi-stage, cross-validated manual analysis to identify the presence of six types of test smells in these. We then use this manual labeling to benchmark the performance and external validity of two test smell detection tools—one widely used in prior work and one recently introduced with the express goal to match developer perceptions of test smells. Our results primarily show that the current vocabulary of test smells is highly mismatched to real concerns: multiple smells were ubiquitous on developer-written tests but virtually never correlated with semantic or maintainability flaws; machine-generated tests actually often scored better, but in reality, suffered from a host of problems not well-captured by current test smells. Current test smell detection strategies poorly characterized the issues in these automatically generated test suites; in particular, the older tool’s detection strategies misclassified over 70% of test smells, both missing real instances (false negatives) and marking many smell-free tests as smelly (false positives). We identify common patterns in these tests that can be used to improve the tools, refine and update the definition of certain test smells, and highlight as of yet uncharacterized issues. Our findings suggest the need for (i) more appropriate metrics to match development practice, (ii) more accurate detection strategies to be evaluated primarily in industrial contexts.

show abstract

Log-based slicing for system-level test cases

Cited by 11 publications

References 32 publications

Towards Log Slicing

Towards Log Slicing

A Literature Survey of Assertions in Software Testing

Test smells 20 years later: detectability, validity, and reliability

Contact Info

Product

Resources

About