Trustworthy Evaluation of a Safe Driver Machine Interface through Software-Implemented Fault Injection

Abstract

Abstract-Experimental evaluation is aimed at providing useful insights and results that constitute a confident representation of the system under evaluation. Although guidelines and good practices exist and are often applied, the uncertainty of results and the quality of the measuring system is rarely discussed. To complement such guidelines and good practices in experimental evaluation, metrology principles can contribute in improving experimental evaluation activities by assessing the measuring systems and the results achieved. In this paper we present the experimental evaluation by software-implemented fault injection of a safe train-borne Driver Machine Interface (DMI), to evaluate its behavior in presence of faults. The measuring system built for the purpose and the results obtained on the assessment of the DMI are scrutinized along basic principles of metrology and good practices of fault injection. Trustfulness in results has been estimated satisfactory and the experimental campaign has shown that the safety mechanisms of the DMI correctly identify the faults injected and that a proper reaction is executed.

On trustworthy measurements when testing dependable systems: a discussion and experiences

“…The EVC Packet Generator severely affects repeatability: despite it is supposed to generate always the same workload with the same exact timing, such exact timing is not guaranteed because of the non-real time OS (Microsoft Windows) in use. Running our experiments, we noted that experiments performed while the DMI is in start-up mode were highly repeatable, showing compatible measurement results in the various runs of the same experiment [15]. This is mainly due to the fact that in start-up mode the EVC Packet Generator is not active, thus not introducing variability.…”

“…Only in one case a slight violation of the requirement was observed, in which the reaction time (time to transit to safe state) was slightly exceeding the maximum allowed time of 100 milliseconds. In such case, it was suggested to system designers to shorten the period of the thread devoted to manage transition to safe state [15].…”

A survey of evaluation platforms for ad hoc routing protocols: A resilience perspective

“…Although many authors in the domain of ad hoc networks present measures that appear generally correct and sensible, sometimes the approach followed to obtain them varies from a paper to another [93]. This fact is due to the lack of widespread rules and practices to conduct their assessment following reference or standardised criteria, which hinders the comparison of measures.…”

A Software-Implemented Fault Injection Toolkit for Dependency Analysis of Large Scale Distributed Applications

“…The effects of faults on the distributed system can be also uncovered while fault injection tool executes realistic programs. Fault injection method has become an attractive way of validating specific fault tolerance mechanisms and allowing the estimation of fault tolerant system measures [5,6], According to the way of injecting faults and errors into target, these methods can be classified into two categories which are hardware and software implemented fault injections. And network connection fault could be triggered by either hardware errors or software errors, so the fault injection methods are generally distinguished by hardware and software.…”