On Testing Data-Intensive Software Systems

Felderer, Michael; Russo, Barbara

doi:10.1007/978-3-030-25312-7_6

Cited by 22 publications

(12 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MLware combines data and conventional source code, thus we propose the amalgamation of corresponding quality definitions from the IEC/ISO 25000 series. Our proposal is in line with discussions by Felderer et al in the context of testing data-intensive systems [11].…”

Section: Aiq -An Ai Meta-testbedsupporting

confidence: 88%

The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs

Borg

2021

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

AI solutions seem to appear in any and all application domains. As AI becomes more pervasive, the importance of quality assurance increases. Unfortunately, there is no consensus on what artificial intelligence means and interpretations range from simple statistical analysis to sentient humanoid robots. On top of that, quality is a notoriously hard concept to pinpoint. What does this mean for AI quality? In this paper, we share our working definition and a pragmatic approach to address the corresponding quality assurance with a focus on testing. Finally, we present our ongoing work on establishing the AIQ Meta-Testbed.

show abstract

Section: Aiq -An Ai Meta-testbedsupporting

confidence: 88%

The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs

Borg

2021

Lecture Notes in Business Information Processing

View full text Add to dashboard Cite

show abstract

“…However, keeping the same level of software quality becomes challenging due to shorter testing cycles. Run-time monitoring of services in operations [5], which is the focus of this study, is of high importance for gaining confidence in a software system and providing feedback to the development.…”

Section: Introductionmentioning

confidence: 99%

Closing the Feedback Loop in DevOps Through Autonomous Monitors in Operations

2021

View full text Add to dashboard Cite

DevOps represent the tight connection between development and operations. To address challenges that arise on the borderline between development and operations, we conducted a study in collaboration with a Swedish company responsible for ticket management and sales in public transportation. The aim of our study was to explore and describe the existing DevOps environment, as well as to identify how the feedback from operations can be improved, specifically with respect to the alerts sent from system operations. Our study complies with the basic principles of the design science paradigm, such as understanding and improving design solutions in the specific areas of practice. Our diagnosis, based on qualitative data collected through interviews and observations, shows that alert flooding is a challenge in the feedback loop, i.e. too much signals from operations create noise in the feedback loop. Therefore, we design a solution to improve the alert management by optimizing when to raise alerts and accordingly introducing a new element in the feedback loop, a smart filter. Moreover, we implemented a prototype of the proposed solution design and showed that a tighter relation between operations and development can be achieved, using a hybrid method which combines rule-based and unsupervised machine learning for operations data analysis.

show abstract

“…The definition they provided was also instrumental in deriving the major factors which are used to characterize such type of systems and also was crucial in driving the further investigation. They also further discussed the state of art and new possible and probable frontiers on testing of the data in these types of software systems [8].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning System and It’s Automatic Testing: An Approach

Khan¹

2021

Ann. Data. Sci.

View full text Add to dashboard Cite

The process of testing conventional programs is quite easy as compared to the programs using Deep Learning approach. The term Deep learning (DL) is used for a novel programming approach that is highly data centric and where the governing rules and logic are primarily dependent on the data used for training. Conventionally, Deep Learning models are evaluated by using a test dataset to evaluate their performance against set parameters. The difference in data and logic handling between programs using conventional methods and programs using the DL approach makes it difficult to apply the traditional approaches of testing directly to DL based programs. The accuracy of test data is currently the best measure of the adequacy of testing in the DL based systems. This poses a problem because of the difficulty in availability of test data that is of sufficient quality. This in turn restricts the level of confidence that can be established on the adequacy of testing of DL based systems. Unlike conventional applications, using the conventional programming approaches the lack of quality test data and the lack of interpretability makes the system analysis and detection of defects a difficult task in DL based systems. So testing of DL based models can be done automatically with a different approach compared to normal software.

show abstract

On Testing Data-Intensive Software Systems

Cited by 22 publications

References 24 publications

The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs

The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs

Closing the Feedback Loop in DevOps Through Autonomous Monitors in Operations

Deep Learning System and It’s Automatic Testing: An Approach

Contact Info

Product

Resources

About