Metamorphic Testing of a Deep Learning based Forecaster

Dwarakanath, Anurag; Ahuja, Manish; Podder, Sanjay; Vinu, Silja; Naskar, Arijit; MV, Koushik

doi:10.48550/arxiv.1907.06632

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the applicable MRs, three are concerned about accuracy: the follow-up test case introduces outliers [28], makes arbitrary affine transformations [29], or removes one category from the image data [32]. The two completenessrelated MRs introduce missing values [28] and add uninformative attributes [29] in the follow-up test cases. The MRs about consistency both rest on linear scaling of feature values [28,31].…”

Section: Results and Analysismentioning

confidence: 99%

See 1 more Smart Citation

Deep learning's fitness for purpose: A transformation problem frame's perspective

Gudaparthi

Niu

Yang

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

Combined sewer overflows represent significant risks to human health as untreated water is discharged to the environment. Municipalities, such as the Metropolitan Sewer District of Greater Cincinnati (MSDGC), recently began collecting large amounts of water-related data and considering the adoption of deep learning (DL) solutions like recurrent neural network (RNN) for predicting overflow events. Clearly, assessing the DL's fitness for the purpose requires a systematic understanding of the problem context. In this study, we propose a requirements engineering framework that uses the problem frames to identify and structure the stakeholder concerns, analyses the physical situations in which the highquality data assumptions may not hold, and derives the software testing criteria in the form of metamorphic relations that incorporate both input transformations and output comparisons. Applying our framework to MSDGC's overflow prediction problem enables a principled way to evaluate different RNN solutions in meeting the requirements. K E Y W O R D S deep learning, deep neural networks, software engineeringThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Section: Results and Analysismentioning

confidence: 99%

“…The two completenessrelated MRs introduce missing values [28] and add uninformative attributes [29] in the follow-up test cases. The MRs about consistency both rest on linear scaling of feature values [28,31]. The latency impacting currentness is instrumented in…”

Section: Results and Analysismentioning

confidence: 99%