Design Automation for Intelligent Automotive Systems

Lan, Shuyue; Huang, Chao; Wang, Zhilu; Liang, Hengyi; Su, Wencong; Zhu, Qi

doi:10.1109/test.2018.8624723

Cited by 8 publications

(7 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Odena and Goodfellow [24] apply the approximate nearest neighbors algorithm to guide their tests generation, but it is not clear, in a DNN, what the maximum number of nearest neighbors are. As shown in [35], quantitative DNN coverage criteria can be applied to the design and certification of automotive systems with deep learning components.…”

Section: Generation Of Adversarial Examples For Dnnsmentioning

confidence: 99%

Concolic testing for deep neural networks

Sun

Ruan

et al. 2018

Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

296

249

View full text Add to dashboard Cite

Concolic testing combines program execution and symbolic analysis to explore the execution paths of a software program. This paper presents the first concolic testing approach for Deep Neural Networks (DNNs). More specifically, we formalise coverage criteria for DNNs that have been studied in the literature, and then develop a coherent method for performing concolic testing to increase test coverage. Our experimental results show the effectiveness of the concolic testing approach in both achieving high coverage and finding adversarial examples.

show abstract

Section: Generation Of Adversarial Examples For Dnnsmentioning

confidence: 99%

Concolic testing for deep neural networks

Sun

Ruan

et al. 2018

Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

296

249

View full text Add to dashboard Cite

show abstract

“…by conducting empirical studies like surveys [224] or reporting industry experiences [73]. The scope of the described design challenges varies greatly and covers areas such as intelligent automotive systems [119],…”

Section: Software Design (34 Studies)mentioning

confidence: 99%

“…via StackOverflow questions [90,223], surveys [224], theoretical analyses of the AI development process [119,214], or case studies [13,140].…”

Section: Software Construction (23 Studies)mentioning

confidence: 99%

“…A total of 12 papers studied the challenges in software testing for AI. Nine of them discussed the challenges, issues, and needs in AI software testing based on the current state of the art, either for generic AI systems [29,62,81,150,175] or focusing on the particular challenges for autonomous vehicles or other safety-critical systems [105,117,119,170]. Finally, three proposals identified the challenges for generic AI or ML systems through empirical methods like questionnaire surveys with practitioners [79,88,224].…”

Section: Software Testing (115 Studies)mentioning

confidence: 99%

See 1 more Smart Citation

Software Engineering for AI-Based Systems: A Survey

Martínez-Fernández,

Bogner,

Franch

et al. 2021

Preprint

View full text Add to dashboard Cite

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image-and speechrecognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems.To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula. CCS Concepts: • Software and its engineering → Software creation and management; • Computing methodologies → Machine learning;

show abstract

“…The adversarial test generation in [36,40] is applied to evaluate DNN-based control systems in autonomous vehicles. As shown in [19,35], quantitative DNN coverage criteria can be applied to support the design and certification of automotive systems with deep learning components.…”

Section: Related Workmentioning

confidence: 99%

Structural Test Coverage Criteria for Deep Neural Networks

Sun

Huang

Kroening

et al. 2019

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Deep neural networks (DNNs) have a wide range of applications, and software employing them must be thoroughly tested, especially in safety-critical domains. However, traditional software test coverage metrics cannot be applied directly to DNNs. In this paper, inspired by the MC/DC coverage criterion, we propose a family of four novel test coverage criteria that are tailored to structural features of DNNs and their semantics. We validate the criteria by demonstrating that test inputs that are generated with guidance by our proposed coverage criteria are able to capture undesired behaviours in a DNN. Test cases are generated using a symbolic approach and a gradient-based heuristic search. By comparing them with existing methods, we show that our criteria achieve a balance between their ability to find bugs (proxied using adversarial examples and correlation with functional coverage) and the computational cost of test input generation. Our experiments are conducted on state-of-the-art DNNs obtained using popular open source datasets, including MNIST, CIFAR-10 and ImageNet.

show abstract

Design Automation for Intelligent Automotive Systems

Cited by 8 publications

References 65 publications

Concolic testing for deep neural networks

Concolic testing for deep neural networks

Software Engineering for AI-Based Systems: A Survey

Structural Test Coverage Criteria for Deep Neural Networks

Contact Info

Product

Resources

About