The functional testing technique has been widely applied to reveal unknown faults in the software caused by programmer mistakes. Nevertheless, in autonomous data processing systems with highly variable inputs and outputs, such as embedded applications, data streams, and machine learning algorithms, the non-functional testing helps to reveal unknown-nontrivial faults in the deployment of software products. This paper addresses the detection of unknown code-faults by employing performance analysis as a nonfunctional requirement without predefined test cases, test oracles, or source-code analyses. The premise is that codefaults change demands for hardware resources during software execution, and a novel testing methodology can automatically detect them. This paper proposes the Tricorder testing methodology for automating workload characterization and detecting potential performance anomalies, caused by code-faults in autonomous data processing systems. Tricorder evaluates the performance profiles of hardware regarding the detection of the source code-faults. DAMICORE, a non-parametric multipurpose clustering methodology, enables Tricorder to group performance profiles of the software under testing and identify performance anomalies using non-parametric data in unsupervised learning based on Normalized Compression Distance (NCD). Tricorder reveals unknown source code faults, with no specialist to determine standards for input and output data, previous models inherent to architecture, test case creation, or workload characterization. We evaluate the capability of Tricorder in revealing faults through experiments based on three benchmarks: cryptography system, machine learning algorithm, and data stream processing server. Tricorder detects faults even under various workloads for different applications in our experiments. Tricorder helps the maintenance phase of the software development life cycle, providing additional information regarding the proper functioning of the application release before and after the updating process. This work contributes to the cost reduction of regression testing during the maintenance phase of autonomous data processing applications and can be used as a complementary testing technique.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.