The numerical format used for representing weights and activations plays a key role in the computational efficiency and robustness of CNNs. Recently, a 16-bit floating point format called Brain-Float 16 (bf16) has been proposed and implemented in hardware accelerators. However, the robustness of accelerators implemented with this format has not yet been studied. In this paper, we perform a comparison of the robustness of state-of-the art CNNs implemented with 8-bit integer, Brain-Float 16 and 32bit floating point formats. We also introduce an error detection and masking technique, called opportunistic parity (OP), which can detect and mask errors in the weights with zero storage overhead. With this technique, the robustness of floating point weights to bit-flips can be improved by up to three orders of magnitude.
Deep Neural Networks (DNNs) are increasingly used in safety critical autonomous systems. In this paper, we present MOZART, a DNN accelerator architecture which provides fault detection and fault tolerance. MOZART is a systolic architecture based on the Output Stationary (OS) variant, as it is the one that inherently limits fault propagation. In addition, MOZART achieves fault detection with on-line functional testing of the Processing Elements (PEs). Faulty PEs are swiftly taken off-line with minimal classification impact. The implementation of our approach on Squeezenet results in a loss of accuracy of less than 3% in the presence of a single faulty PE, compared to 15-33% without mitigation. The area overhead for the test logic does not exceed 8%. Dropout during training further improves fault tolerance, without a priori knowledge of the faults.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.