Metric-Driven Verification Methodology with Regression Management

Abstract

This paper discusses the regression testing in digital integrated circuit verification. The aim of the metric-driven verification (MDV) and its role in modern verification methodology is presented. According to recognized limitations, it is proposed to include the regression testing strategy in verification environment itself. The data collected from the verification metrics can be used to adjust the testing procedures during the simulation. This approach allows for a dynamic management of the regression testing structure and may result in a significant reduction of the simulation time. The presented solution introduces a concept of the test segments. They may be started at arbitrary simulation point related to the DUT internal state (checkpoint). Usage of such segments, in the controlled regression testing strategy, may prevent from repeating the stimuli which is not contributing to the pre-defined verification metrics.

Fuzzing Hardware Like Software

“…To overcome CRV shortcomings, researchers have proposed CDG [15]- [18], [22], [24], [25], [28]- [32], or using test coverage feedback to drive future test generation. Unlike CRV, CDG does not randomly piece input sequences together in hopes of exploring new design state.…”

“…First, how should we format our grammar to enable the fuzzer to learn it quickly? To facilitate widespread deployment of Hardware Fuzzing, it is imperative DV engineers do not have to tailor fuzzing harnesses (testbenches) to specific designs, as is the case with existing CDG methods [15], [22], [24], [25], [28]- [30]. Lastly, how does Hardware Fuzzing perform in practice on real hardware IP cores?…”

“…At one extreme, dynamic verification involves driving concrete input sequences into a Design Under Test (DUT) during simulation, and comparing the DUT's behavior to a set of invariants, or gold model. The most popular dynamic verification technique in practice today is known as Constrained Random Verification (CRV) [24]- [27]. CRV attempts to decrease the manual effort required to develop simulation test cases by randomizing input sequences in the hopes of automatically maximizing exploration of the DUT statespace.…”

“…In search of a hybrid approach to bridge these DV extremes, researchers have ported software testing techniques to the hardware domain in hopes of improving hardware test generation to maximize coverage. In the hardware domain, these approaches are referred to as CDG [15], [18], [22], [24], [25], [28]- [32]. Like their software counterparts, CDG techniques deploy coverage metrics-e.g., Hardware Description Language (HDL) line, Finite State Machine (FSM), functional, etc.-in a feedback loop to generate tests that further increase state exploration.…”