Alinhamento de sequências na avaliação de resultados de teste de robustez

Abstract

Robustness, which is the ability of a system to work properly in unexpected situations, is an important characteristic, especially for critical systems. A commonly used technique for robustness testing is to inject faults during the system execution and to observe its behavior. A frequent problem during the tests is to determine an oracle, i.e., a mechanism that decides if the system behavior is acceptable or not. Oracles such as the golden run comparison -system execution without injection of faults -consider all different behaviors from the golden run as errors in the system under test (SUT). For example, the activation of exception handlers in the presence of faults could be considered as errors. Safety property searching approach is also used as oracle and it can show the presence of non-robustness in the SUT. If there are events in the SUT execution that are semantically similar to the property they are not taken into account (unless they have been explicitly defined in the property). The objective of this work is to develop specific oracles to evaluate results of robustness testing in order to minimize the deficiences in the current oracles. The main difference between our solutions and the existing approaches is the type of algorithm that we used to compare the sequences. We adopted sequence alignment algorithms commonly applied in Bioinformatics. These algorithms are a kind of inexact matching, allowing some variations between the compared sequences. First approach is based on the tradicional golden run comparison, but applies global sequence alignment of sequences to compare traces collected during fault injection and traces collected without fault injection. The use of this algorithms allows that traces with some differences of the golden run also being classified as robust allowing it use in non-deterministic systems evaluation which is not possible currently. The second approach works with comparison of patterns derived from safety properties and traces collected during robustness testing. However, differently from the first approach, the second one use of local sequence alignment algorithm to search for subsequences. Besides the advantages of the inexact matching, these algorithms use a scoring system based on information obtained from SUT specification to guide the alignment of the sequences. We show the results of the approaches application through case studies. ix A Deus em primeiro lugar por me dar saúde, capacidade e força necessárias para o desenvolvimento deste trabalho.À minha orientadora Eliane Martins que é um exemplo de competência e de dedicação ao trabalho acadêmico, por seu suporte em todos os momentos, por seus conselhos e ideias.Ao meu marido Mario Prado que, nestes anos em que eu estive imersa neste trabalho, traduziu de maneira perfeita a palavra "companheiro".Aos meus filhos Marinho e Pedro pela compreensão nas muitas ausências da mamãe durante as brincadeiras e os passeios.Aos meus pais, Ernani e Cidinha pelo suporte incondicional que eu nunca esquecerei. A ajuda deles foi decisiv...