Robustness testing oracle using a sequence alignment algorithm

Lemos, Gizelle Sandrini; Martins, Eliane

doi:10.1145/1868048.1868051

Cited by 3 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous works [8,9] presented the application of local SqAA [29], which is useful to determine whether a certain property (invariant) was satisfied by the trace. In this work we propose the use of global SqAA to allow the identification of deviations from normal behavior when in presence of faults.…”

Section: Introductionmentioning

confidence: 99%

Specification-guided Golden Run for Analysis of Robustness Testing Results

Lemos

Martins

2012

2012 IEEE Sixth International Conference on Software Security and Reliability

Self Cite

View full text Add to dashboard Cite

Comparison with a golden run is commonly used as an oracle in robustness testing based on fault injection. However, traditional comparison algorithms present, among other limitations, requires the system under test to present, for the same workload, the same behavior, either in presence or in absence of faults. We present an approach that uses a pairwise sequence alignment algorithm in the comparison allowing faulty traces to have some regions of dissimilarity regarding the golden run. This is possible because the algorithm is based on inexact matching and aggregates to the search semantic aspects based on SUT specification. The approach can obtain the degree of similarity with the golden run, and visually presents similarities and differences, which helps in diagnosing. This paper illustrates the application of the approach on traces collected during robustness testing of the Wireless Transaction Protocol (WTP).

show abstract

Section: Introductionmentioning

confidence: 99%

Specification-guided Golden Run for Analysis of Robustness Testing Results

Lemos

Martins

2012

2012 IEEE Sixth International Conference on Software Security and Reliability

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nosso trabalho [61,62,63] também apresentou a mesma evolução na definição do sistema de pontuação que verificamos em [25,26]. Inicialmente, trabalhamos apenas com sistemas de pontuação fixa (com apenas três valores, para pontuação de matches, mismatches e espaços), então passamos a pontuar diferentemente alguns mismatches, que chamamos de good mismatches.…”

Section: Considerações Finaisunclassified

Alinhamento de sequências na avaliação de resultados de teste de robustez

Lemos¹,

Martins²

View full text Add to dashboard Cite

Robustness, which is the ability of a system to work properly in unexpected situations, is an important characteristic, especially for critical systems. A commonly used technique for robustness testing is to inject faults during the system execution and to observe its behavior. A frequent problem during the tests is to determine an oracle, i.e., a mechanism that decides if the system behavior is acceptable or not. Oracles such as the golden run comparison -system execution without injection of faults -consider all different behaviors from the golden run as errors in the system under test (SUT). For example, the activation of exception handlers in the presence of faults could be considered as errors. Safety property searching approach is also used as oracle and it can show the presence of non-robustness in the SUT. If there are events in the SUT execution that are semantically similar to the property they are not taken into account (unless they have been explicitly defined in the property). The objective of this work is to develop specific oracles to evaluate results of robustness testing in order to minimize the deficiences in the current oracles. The main difference between our solutions and the existing approaches is the type of algorithm that we used to compare the sequences. We adopted sequence alignment algorithms commonly applied in Bioinformatics. These algorithms are a kind of inexact matching, allowing some variations between the compared sequences. First approach is based on the tradicional golden run comparison, but applies global sequence alignment of sequences to compare traces collected during fault injection and traces collected without fault injection. The use of this algorithms allows that traces with some differences of the golden run also being classified as robust allowing it use in non-deterministic systems evaluation which is not possible currently. The second approach works with comparison of patterns derived from safety properties and traces collected during robustness testing. However, differently from the first approach, the second one use of local sequence alignment algorithm to search for subsequences. Besides the advantages of the inexact matching, these algorithms use a scoring system based on information obtained from SUT specification to guide the alignment of the sequences. We show the results of the approaches application through case studies. ix A Deus em primeiro lugar por me dar saúde, capacidade e força necessárias para o desenvolvimento deste trabalho.À minha orientadora Eliane Martins que é um exemplo de competência e de dedicação ao trabalho acadêmico, por seu suporte em todos os momentos, por seus conselhos e ideias.Ao meu marido Mario Prado que, nestes anos em que eu estive imersa neste trabalho, traduziu de maneira perfeita a palavra "companheiro".Aos meus filhos Marinho e Pedro pela compreensão nas muitas ausências da mamãe durante as brincadeiras e os passeios.Aos meus pais, Ernani e Cidinha pelo suporte incondicional que eu nunca esquecerei. A ajuda deles foi decisiv...

show abstract

Golden-run alignment for analysis of robustness testing results: dealing with diagnostics issues

Lemos

Martins

2013

2013 43rd Annual IEEE/IFIP Conference on Dependable Systems and Networks Workshop (DSN-W)

View full text Add to dashboard Cite

Robustness testing oracle using a sequence alignment algorithm

Cited by 3 publications

References 9 publications

Specification-guided Golden Run for Analysis of Robustness Testing Results

Specification-guided Golden Run for Analysis of Robustness Testing Results

Alinhamento de sequências na avaliação de resultados de teste de robustez

Golden-run alignment for analysis of robustness testing results: dealing with diagnostics issues

Contact Info

Product

Resources

About