A Model Checking-Based Method of Functional Test Generation for HDL Descriptions

. Automated test generation is a promising direction in hardware verification research area. Functional test generation methods based on models are widespread at the moment. In this paper, a functional test generation method based on model checking is proposed and compared to existing solutions. Automated model extraction from the hardware design’s source code is used. Supported HDLs include VHDL and Verilog. Several kinds of models are used at different steps of the test generation method: guarded action decision diagram (GADD), high-level decision diagram (HLDD) and extended finite-state machine (EFSM). The high-level decision diagram model (which is extracted from the GADD model) is used as a functional model. The extended finite-state machine model is used as a coverage model. The aim of test generation is to cover all the transitions of the extended finite state machine model. Such criterion leads to the high HDL source code coverage. Specifications based on transition and state constraints of the EFSM are extracted for this purpose. Later, the functional model and the specifications are automatically translated into the input format of the nuXmv model checking tool. nuXmv performs model checking and generates counterexamples. These counterexamples are translated to functional tests that can be simulated by the HDL simulator. The proposed method has been implemented as a part of the HDL Retrascope framework. Experiments show that the method can generate shorter tests than the FATE and RETGA methods providing the same or better source code coverage.


Introduction
Functional verification is an expensive and time-consuming stage of hardware design process [1].Due to hardware designs increasing complexity, automated test generation seems to be important and challenging.To avoid design complexity, automated verification methods often utilize mathematical abstractions of system properties and behavior, so-called models.Models can be created manually or automatically extracted from the system's source code.Automated verification methods based on model extraction from the HDL (Hardware Description Language -a collective name for several languages described below) source code are considered in this paper.Models can be based on the following formal descriptions: finite-state machines, decision diagrams, Petri nets [2], etc. Model checking [3] is an approach to set up the correspondence between the model of the system and formal conditions (specifications).For every specification a model checker tries to produce a counterexample -an input stimuli sequence that leads the system into a specification-contradicting state.Counterexample construction is often used for functional test generation purposes.Proof of equivalence of a model and the corresponding system is an important issue when model checking is used for hardware verification.There is no need in such proof when the model is automatically extracted from the system's source code and translated into the model checker's format.A method of functional test generation for hardware is proposed in this paper.It is based on automatic extraction of High-Level Decision Diagrams (HLDD) [4] from the system's source code.Synthesizable sets of VHDL [5] and Verilog [6] HDLs are supported.Extracted models are then automatically translated into SMV (Symbolic Model Verifier) language supported by the nuXmv [7] model checker.Extended Finite State Machine (EFSM) transition constraints are used as specifications for model checking.EFSM model is also extracted from the system's source code.Counterexamples built by the nuXmv model checker are then translated into an HDL testbench which can be simulated by an HDL simulator.The rest of the paper is organized as follows.Section 2 contains a review of functional test generation methods based on model extraction from the HDL source code.In Section 3 basic definitions are given.Section 4 contains HLDD construction and test generation methods.Section 5 is dedicated to the experimental results.Section 6 concludes the paper.

Related works
The idea of model extraction from the HDL source code and following test generation is not new.A prototype of CV tool for VHDL description model checking is presented in [8].The tool's execution process consists of five stages.On the first stage, a VHDL description is parsed and an internal representation is constructed.A Binary Decision Diagram (BDD) based model is built on the second stage.On the third stage a CTLbased specification is parsed.The specification language syntax is described in the paper.On the fourth stage the specification parsing result and the BDD-based model are passed to the CBMC [9] model checker.On the final stage, the model checker output is translated into tests that can be executed by the HDL simulator.It is stated that BDD-based model size plays the key role in the model checking process.Model size reduction heuristics usage is suggested to avoid state space explosion but no heuristics are offered in the paper.In [10], extraction of the EFSM model and generation of tests that cover all the model transitions are described (so-called FATE method).This method assumes that the user provides additional information for the tool about signal semantics (for example, which of the signals encodes state).The EFSM extraction process contains several stages of transition structure simplification (embedded conditions elimination, compatible transitions union, dataflow dependency analysis).The test generation method is based on the state graph traversal through random walk and backjumping techniques.
In [11] an improved modification of method [10] is proposed.Optimizations described concern path reachability (weakest precondition [12] is used instead of the approximate approach) and test filtering tasks.A new functional test generation method called RETGA is also proposed in [11].This method is based on the algorithm [13] for automated EFSM model extraction from HDL descriptions.The algorithm does not require additional information about signals\variables semantics; for state and clock-like variable detection it uses heuristics based on dataflow dependencies.Experiments have shown that RETGA method produces shorter tests with higher HDL code coverage than FATE and even improved FATE do.It should be noted, that state graph traversal techniques (that FATE and RETGA methods use) do not guarantee coverage of all the EFSM model transitions.One of the problems concerns counter [11] variables that are defined in transition loops and used in transition guards, so an EFSM simulation engine needs to recognize at which value of the counter it is going to finish the loop execution.

Basic definitions
Suppose that all models described below run in discrete time that implies clock presence.Clock C is a set of events {c 1 ,…,c k } where an event c = {signal, edge} is a pair, consisting of a one-bit signal (so-called clock pulse) and a type of registration called edge (i.e.positive edge when signal changes its value from 0 to 1 and negative edge otherwise).Let V be a set of variables.A valuation is a function that associates a variable v  V with a value [v] from the corresponding domain.Let Dom V be a set of all valuations of V. A guard is a Boolean function defined on valuations (Dom V → {true, false}).An action is a transform of valuations (Dom V → Dom V ).A pair γ → δ, where γ is a guard and δ is an action, is called a guarded action (GA).It is implied that there is a description of every function in some HDL-like language (thus, we can reason about not only semantics, but syntax).Let guarded actions be synchronized [14], if each GA is associated with a clock.A system {C (i) ,  (i)   (i) } i=1,l of synchronized guarded actions can be represented by an oriented acyclic graph G = (N, E, C) called Guarded Actions Decision Diagram (GADD).Here N is a set of graph nodes, E is a set of graph edges, and C is a clock.
N contains two non-intersecting subsets: a set N s of non-terminal nodes n s that are marked by expressions γ(n s ); a set N t of terminal nodes n t that are marked by actions δ(n t ).Graph edges can start from non-terminal nodes only and finish either in terminal or in non-terminal nodes.Edges e  E are marked by sets Val(e, n) of accepted values γ(n) (here edge e is an outgoing edge for the node n, e  Out(n)).The node n  N s can have no more than one e d  Out(n) that is marked by default keyword -it means that for this path in G an expression γ(n) equals to a value that does not belong to any marking sets of the other edges outgoing from the node n.Supposing that the GADD contains exactly one root node n root (the node without incoming edges, In(n root ) = ), a set of paths from the root node to all the terminal nodes produces a system of synchronized guarded actions.For example, the i th path, including n 1 (i) ,…n m (i) nodes and e 1 (i) ,…, e m-1 (i) edges (n , defines a guarded action with p 1 (i) …p m-1 (i) (p r (i) = AND(γ(n r ) == q), r = 1,…,m-1, q  Val(e r , n r )) conjunction as a guard and δ(n m (i) ) as an action.The guarded action clock is a subset of the GADD clock.
In [4] a definition of a high-level decision diagram (HLDD) is given and is shown that every variable of an HDL description can be represented by a function Usually some of these functions are trivial and equal either to variables v k  V or to constants.All the edges are marked by sets of accepted values of variables in the same manner as in the GADD definition; the semantics of the default edges is also similar.On every tick of the clock, the HLDD H v assigns a value to the target variable v through an activation procedure.Starting from the initial node m 0 it calculates values of the variables which mark non-terminal nodes.For a value e of the variable v(m c ), e  Z(v(m c )), the corresponding edge from the node m c  M to the subsequent node m e  Γ(m c ) is activated.A vector V t of variable values activates the path l(m 0 , m k ) from m 0 to the terminal node m k marked by the function f k (V k ) that determines the value of the target variable v.

HLDD model construction and test generation method
The proposed test generation method consists of the following steps:  HDL (VHDL/Verilog) description parsing and GADD model construction. HLDD model construction using the GADD model. HLDD model and specification translation into the nuXmv model checker input language (SMV model) [16]. SMV model checking by the nuXmv model checker and translation of counterexamples into HDL tests.
The first step has been implemented in [13] so we will start from the second step.Note that all the actions, which mark the terminal nodes of the GADD model, are represented in the static single assignment (SSA) [15] form.

HLDD model construction
GADD and HLDD models preserve the module structure of the original HDL description.Every HDL description process is represented by a single GADD.The GADD G = (N, E, C) is used as a basis for HLDD construction for every non-input variable of the process.HLDD construction algorithm pseudo code is listed below: 0 shows the GADD model of the process.Non-terminal nodes of the GADD are shown as diamonds and correspond to branch expressions.Terminal nodes are shown as rectangles and correspond to basic blocks.Outgoing edges from the non-terminal nodes are marked by possible values of the branch expressions.Note that the default edge on 0 is unreachable because the state variable can only take the value of 0 or 1.The clock of the GADD is formed by events of clk, rst, x and y signals.

SMV model construction and checking
The constructed HLDD model is translated into an SMV language description.Hardware design module structure is preserved.Any variable constraints (like the range of possible values that is specified for the state variable) and their initial values described in the HDL description are added to the SMV model.Specification construction is based on the EFSM model extracted from the same HDL description.Formal definition of the EFSM model and its extraction algorithm from an HDL description are presented in [13].Here we provide only the informal definition.Extended finite-state machine is a special case of an ordinary finite-state machine (FSM).It contains sets of inputs, outputs and internal variables.EFSM transitions are marked by guard expressions, which depend on input and internal variable values, and by actions that can change internal and output variable values.A transition can be enabled only if its guard becomes true.When a transition is enabled, its action is executed.Specifications used by the proposed method are represented as negations of the EFSM transition guards.Negation is used to make the model checker build a counterexample -a sequence of data states and input stimuli that contradicts the specification (and thus satisfies the corresponding guard).The first state shows the initial values assigned to the variables.The second state shows only the values that have changed.We can see that the second state contradicts the given SMV specification: clk is equal to 1, while the rst and state variables are equal to 0.

Experimental results
The proposed test generation method was implemented as a part of the HDL Retrascope 0.2.1 software tool [17].Java language was used for development along with the Fortress formulae manipulation library [18].Some HDL descriptions from the ITC'99 benchmark [19] were used for testing of the proposed approach.The nuXmv model checker supports both symbolic model checking and bounded model checking [21] methods.In some cases, symbolic model checking needed too much time and computer resources because of the state explosion (for example, B04, B10 and B11 designs).Bounded model checking could manage this problem by exploring the model state space only up to some bound.However, bound value affects the model checking results (not all the counterexamples may be obtained at the specified bound).Therefore, in some cases the bound size was iteratively increased in order to get all possible counterexamples.Generated tests were simulated by the QuestaSim HDL simulator [20].Test properties (length and source code coverage) were compared to existing test generation methods like FATE [10], RETGA [11] (these methods are based on EFSM model extraction from the HDL descriptions and are targeted at covering the EFSM model transitions) and random test generation.0contains information about the ITC'99 designs that were used for test generation: their source code size and the corresponding SMV model size (without specifications).Size is given in lines of code.The proposed method achieved the same code coverage as the RETGA method at the specified set of HDL descriptions.B07 and B11 HDL description coverage is less than 100% because of the unreachable code in these designs.

Conclusion and future work
The functional test generation method based on automated HLDD model extraction and checking with nuXmv is presented in this paper.The main advantage of this method is its flexibility in choosing a test target (through using different kinds of specifications).EFSM transition coverage is presented for comparison to the other test generation methods (FATE, RETGA).Any other specifications can be formulated and checked in order to get a test aimed at covering the corresponding property of a model.The presented implementation of the proposed approach does not produce shorter tests than existing approaches on the chosen hardware design set.Simple optimizations (like test filtering) can be helpful and are going to be implemented in the nearest future.
Future work is focused on applying the method to more complex hardware designs (including Verilog-based).In this case complexity is defined by the number of execution paths in processes and the number of processes and modules in an HDL description.Process decomposition using dataflow analysis methods and predicate abstraction [22] test generation methods are under research now.

Fig. 2 .
Fig.2.HLDD prototype.Consider the HLDD construction for the cnt variable.Terminal nodes marked by cnt are highlighted in grey on 0. Terminal nodes, which are not marked by this variable, are removed.New terminal nodes marked by cnt are added to the free non-terminal
.state = 0sd32_0 SAMPLE.process.cnt= 0ud1_0 SAMPLE.process.guard2= 0sd32_0 SAMPLE.process.guard1= FALSE SAMPLE.process.guard0= FALSE SAMPLE.process.res= 0ud1_0 clk At the first step the HLDD prototype proto is created.GADD nodes are transformed into HLDD nodes with the help of the transform_node method and added to the prototype.Terminal GADD nodes n t  N t are transformed into terminal HLDD nodes m k  M t .Every terminal node n t marked by multiple assignment action δ(n t ) is transformed into a sequence of nodes.Every node in this sequence is marked by a corresponding single assignment a k .Every terminal HLDD node is marked by a target variable v k (which is the left-hand side of a k ) and a function f k (V k ) (which is the righthand side of a k ).Non-terminal GADD nodes n s  N s are transformed into non-terminal HLDD nodes m c  M n .Guard γ which marks the node n s is replaced by a new variable guard(m c ) which marks the node m c .The new HLDD that contains a single terminal node marked by γ is created for this variable (create_variable_from_switch method).GADD edges are transformed into HLDD edges by the copy_edges method.The corresponding values are not changed.Then for every non-input variable v the HLDD hldd is created which is actually a modified copy of proto.The keep_assigns method removes from M t the terminal nodes which are not marked by v.After that the add_missing_terminals method adds new terminal nodes marked by f(v) = v to the edges which lack the subsequent terminal nodes.This means that the value of v does not change if any path to such node is activated.The transform_identical_assigns method searches for such nonterminal nodes m c whose reachable terminal nodes are marked by the same function f k (v k ), and replaces m c and its reachable subgraph with the only terminal node marked by f k (v k ).Consider an example of the HLDD model construction for a simple VHDL description.This description contains a single module and a single process.The module interface consists of input variables clk, rst, x, y and an output variable res (all of 1-bit size).The process contains two internal variables: a 1-bit size vector cnt and an integer state (that can be assigned either 0 or 1).The source code of the process is listed below: proto = new; for node  N do hldd_node = transform_node(node); proto.add(hldd_node);end copy_edges(E, proto); for (v : non_input_variables(G)) do hldd = proto.keep_assigns(v);hldd.add_missing_terminals();hldd.transform_identical_assigns();end

Table 3 .
Source code statement coverage

Table 4 .
Source code branch coverage