Fault Triggers in the TensorFlow Framework: An Experience Report

Du, Xiaoting; Xiao, Guanping; Sui, Yulei

doi:10.1109/issre5003.2020.00010

Cited by 20 publications

(7 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…44,45 Additionally, there are empirical studies involving bug triggering, based on TensorFlow analysis of bug types, distribution, etc. 46 While static code metrics have also contributed to fault prediction, their individual predictive performance is relatively poor. Zhou et al applied various complexity metrics to logistic regression models and found that NCLOC outperformed other metrics.…”

Section: Effects Of Smell On Faults and Securitymentioning

confidence: 99%

Security‐based code smell definition, detection, and impact quantification in Android

Zhong,

Shi,

et al. 2023

Softw Pract Exp

View full text Add to dashboard Cite

Android's high market share and extensive functionality make its security a significant concern. Research reveals that many security issues are caused by insecure coding practices. As a poor design indicator, code smell threatens the safety and quality assurance of Android applications (apps). Although previous works revealed specific problems associated with code smells, the field still lacks research reflecting Android features. Moreover, the cost and time limit developers to repairing numerous smells timely. We conducted a study, including Definition, Detection, and Impact Quantification for Android code smell (DefDIQ): (1) define 15 novel code smells in Android from a security programming perspective and provide suggestions on how to eliminate or mitigate them; (2) implement DACS (Detect Android Code Smell) to automatically detect the custom code smells based on ASTs; (3) investigate the correlation between individual smells with DACS detection results, select suitable code smells to construct fault counting models, then quantify their impact on quality, and thereby generating code smell repair priorities. We conducted experiments on 4575 open‐source apps, and the findings are: (i) Lin's CCC between DACS and manual detection results reaches 0.9994, verifying the validity; (ii) the fault counting model constructed by zero‐inflated negative binomial is superior to negative binomial (AIC = 517.32, BIC = 522.12); some smells do indicate fault‐proneness, and we identify such avoidable poor designs; (iii) different code smells have different levels of importance and the repair priorities constructed provide a practical guideline for researchers and inexperienced developers.

show abstract

Section: Effects Of Smell On Faults and Securitymentioning

confidence: 99%

Security‐based code smell definition, detection, and impact quantification in Android

Zhong,

Shi,

et al. 2023

Softw Pract Exp

View full text Add to dashboard Cite

show abstract

“…Moreover, non-silent bugs have been considered with obvious symptoms like crash or build failure but we study silent bugs inside TensorFlow in this paper which are more difficult to investigate. Moreover, faults triggers in TensorFlow have been investigated recently [15]. Faults triggers are defined as the set of conditions that activated a fault leading to a failure.…”

Section: Related Workmentioning

confidence: 99%

“…There are several empirical studies on bugs in DL software developed using various libraries [11], [12], [13]. Researchers also investigated symptoms, root causes, and repair patterns of bugs inside TensorFlow [7], [14] and their fault triggering conditions [15]. However, we are not aware of existing studies on silent bugs inside DL frameworks, bugs that do not result in obvious symptoms or misbehaviours.…”

Section: Introductionmentioning

confidence: 99%

Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow

Tambon¹,

Nikanjam²,

An³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep Learning (DL) frameworks are now widely used, simplifying the creation of complex models as well as their integration to various applications even to non DL experts. However, like any other programs, they are prone to bugs. This paper deals with the subcategory of bugs named silent bugs: they lead to wrong behavior but they do not cause system crashes or hangs, nor show an error message to the user. Such bugs are even more dangerous in DL applications and frameworks due to the "black-box" and stochastic nature of the systems (the end user can not understand how the model makes decisions). This paper presents the first empirical study of Keras and TensorFlow silent bugs, and their impact on users' programs. We extracted closed issues related to Keras from the TensorFlow GitHub repository. Out of the 1,168 issues that we gathered, 77 were reproducible silent bugs affecting users' programs. We categorized the bugs based on the effects on the users' programs and the components where the issues occurred, using information from the issue reports. We then derived a threat level for each of the issues, based on the impact they had on the users' programs. To assess the relevance of identified categories and the impact scale, we conducted an online survey with 103 DL developers. The participants generally agreed with the significant impact of silent bugs in DL libraries and acknowledged our findings (i.e., categories of silent bugs and the proposed impact scale). Finally, leveraging our analysis, we provide a set of guidelines to facilitate safeguarding against such bugs in DL frameworks.

show abstract

“…Several works in the most recent years have investigated the effects of attacks and faults on ML-applications and have provided evidence that even small perturbations caused by hardware or software faults can deceive the ML-based application, to the extent that an incorrect output is produced [25], [24], [29]. While it is possible that a single transient fault does not lead to wrong decisions or it is masked through the activation of the different layers of the neural network [24], there is still the risk that residual software faults (when not hardware faults) manifest into an observable output corruption [26], [27], [29].…”

Section: Introductionmentioning

confidence: 99%

“…However, as in the recent history we have witnessed also remote hacking of vehicles [21], such attack surface should not be neglected a-priori, and we believe our work can contribute as a warning in this direction. Failures instead may be the consequence of bugs in neural network software [26], [27] or accelerators faults (GPUs) [23], [25]. Several recent works have raised a warning about their dangerousness [29], overall describing the possible occurrence of software and hardware faults in a similar fashion as for other safety-critical hardware and software parts of the system.…”

mentioning

confidence: 99%

Attack and Fault Injection in Self-driving Agents on the Carla Simulator – Experience Report

Piazzesi

Massimo

Ceccarelli

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Machine Learning applications are acknowledged at the foundation of autonomous driving, because they are the enabling technology for most driving tasks. However, the inclusion of trained agents in automotive systems exposes the vehicle to novel attacks and faults, that can result in safety threats to the driving tasks. In this paper we report our experimental campaign on the injection of adversarial attacks and software faults in a self-driving agent running in a driving simulator. We show that adversarial attacks and faults injected in the trained agent can lead to erroneous decisions and severely jeopardize safety. The paper shows a feasible and easily-reproducible approach based on open source simulator and tools, and the results clearly motivate the need of both protective measures and extensive testing campaigns.

show abstract

Fault Triggers in the TensorFlow Framework: An Experience Report

Cited by 20 publications

References 35 publications

Security‐based code smell definition, detection, and impact quantification in Android

Security‐based code smell definition, detection, and impact quantification in Android

Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow

Attack and Fault Injection in Self-driving Agents on the Carla Simulator – Experience Report

Contact Info

Product

Resources

About