An Interview Study of how Developers use Execution Logs in Embedded Software Engineering

Yang, Nan; Cuijpers, Pjl Pieter; Schiffelers, R.R.H.; Lukkien, Johan J.; Serebrenik, Alexander

doi:10.1109/icse-seip52600.2021.00015

Cited by 13 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from them, we focus on the latest deep learning-based approaches and investigate more practical issues such as unprecedented logs in testing data and inevitable anomalies in training data. More recently, Yang et al [49] presented an interview study of how developers use execution logs in embedded software engineering, which summarizes the major challenges of log analysis. He et al [15] conducted a comprehensive survey on log analysis for reliability engineering, which covers the entire lifecycle of logs, including logging, log compression, log parsing, and various log mining tasks.…”

Section: Related Workmentioning

confidence: 99%

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

Chen¹,

Liu²,

Gu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Logs have been an imperative resource to ensure the reliability and continuity of many software systems, especially large-scale distributed systems. They faithfully record runtime information to facilitate system troubleshooting and behavior understanding. Due to the large scale and complexity of modern software systems, the volume of logs has reached an unprecedented level. Consequently, for log-based anomaly detection, conventional methods of manual inspection or even traditional machine learning-based methods become impractical, which serve as a catalyst for the rapid development of deep learning-based solutions. However, there is currently a lack of rigorous comparison among the representative log-based anomaly detectors which resort to neural network models. Moreover, the re-implementation process demands non-trivial efforts and bias can be easily introduced. To better understand the characteristics of different anomaly detectors, in this paper, we provide a comprehensive review and evaluation on five popular models used by six state-of-the-art methods. Particularly, four of the selected methods are unsupervised and the remaining two are supervised. These methods are evaluated with two publicly-available log datasets, which contain nearly 16 millions log messages and 0.4 million anomaly instances in total. We believe our work can serve as a basis in this field and contribute to the future academic researches and industrial applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

Chen¹,

Liu²,

Gu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To answer our research questions, in our previous conference paper (Yang et al 2021) we reported on an exploratory case study at ASML, a company that develops lithography systems for the semiconductor industry. We conducted a series of semi-structured interviews with 25 software developers.…”

Section: Introductionmentioning

confidence: 99%

An interview study about the use of logs in embedded software engineering

Yang

Cuijpers²,

Hendriks³

et al. 2023

Empir Software Eng

Self Cite

View full text Add to dashboard Cite

published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the "Taverne" license above, please follow below link for the End User

show abstract

“…There is a need for a tool to identify meaningful differences between executions. They also report the need for a top-down approach, where a high-level hypothesis is verified by a lower level of analysis [3].…”

Section: Introductionmentioning

confidence: 99%

Software Failure Log Analysis for Engineers—Review

2023

View full text Add to dashboard Cite

The use of automated methods for log analysis is unavoidable in any large company; therefore, it has attracted attention from engineers and researchers. As a result, the number of articles in the field grows yearly and new approaches are frequently proposed. Unfortunately, published research works only sometimes meet the needs of engineers wishing to apply the methods in real-life systems. A common issue is that the method’s benefits often do not compensate for the effort required for its implementation and maintenance. Therefore, engineers must understand the pros and cons of full-scale applications, including the implementation details and the required effort. This work provides a comprehensive review of automated log analysis methods and aims to offer a guide for software engineers who fix integration and production failures. The article categorizes and provides an overview of existing methods and assesses their implementation and maintenance costs, as well as the feasibility of the methods. The article also identifies and describes the shortcomings of existing methods, including concept drift, which is not addressed with sufficient attention, as well as the lack of online benchmarks and the interpretation of the log sequence as a language, without an in-depth analysis of its properties. Despite growing efforts to provide feasible and widely adopted solutions, many reference implementations are unavailable. Consequently, the time and computation complexities differ between various implementations of the same approaches, making the results of research work difficult to replicate in real-life scenarios.

show abstract

An Interview Study of how Developers use Execution Logs in Embedded Software Engineering

Cited by 13 publications

References 38 publications

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

An interview study about the use of logs in embedded software engineering

Software Failure Log Analysis for Engineers—Review

Contact Info

Product

Resources

About