Verification coverage
Verification coverage attempts to answer the question: “How do you know you are finished verifying?” In reality, coverage can only provide a partial answer but sensible use of coverage strategies and metrics can provide SoC design teams with a good sense of their progress towards takeout.
The strategy in use today among a number of teams is to feed as much coverage data as possible into a single unified database that can be queried by a variety of users and tools to provide dashboard functions that show progress towards a given target or to outline which parts of the design need to be exercised more comprehensively. An example of a design-wide database and access software is vManager from Cadence Design Systems.
Image A verification database such as Cadence vManager allows multiuser access to coverage data
Coverage can be measured in many different ways. The most common is code coverage, a technique common to software as well as hardware development. Hardware code coverage involves recording statistics from simulation on how many times a particular statement in the code has been exercised, if at all. This form of coverage is effective for isolating unreachable code, which may indicate the presence of a redundant block, such as a feature from a reused IP block that is not required in the design, that access to a vital feature has not been implemented correctly, or that the testbench is not exercizing the code as well as it might.
Code coverage is one of the easiest forms of coverage to implement and understand but can give a distorted view of the readiness of a design as simple executing a statement a few times is unlikely to explore boundary conditions that may indicate aberrant behavior. There are a number of different variants of code coverage that range from individual statement execution to the traversal of a complete logic path.
Code coverage techniques
Statement coverage is similar but not the same as block or segment coverage, where the measured unit of code is a sequence of non-branching code that executes within a single tick of simulation.
Decision coverage, which is also used in the coverage of high-reliability software, measures the coverage of each branch of an if or case statement. Decision coverage in hardware is slightly more accurate than statement coverage because of the presence of implied else conditions, such as if (reset) which may be used to ensure that an output is set to a known value after reset but that the output remains unchanged if the code is executed under normal conditions. Statement coverage would report full coverage if the simulation only exercized the reset condition, whereas decision coverage would recognize the presence of the implied else.
Path coverage is an extension to decision coverage in that it treats a series of decisions as paths through a section of logic. The problem with path coverage is the explosion of paths that need to be considered for all but very small blocks.
Image Coverage managers show which parts of a module have been exercised (Source: Cadence)
Expression coverage can be considered to be a more granular form of statement coverage, examining how variables or sub-expressions are evaluated. A typical use of this is for complex logic evaluations with multiple inputs to test that the combinations match the design intent by, with the right stimuli, exercizing covering the truth table for the expression.
Another popular form of coverage is for finite state machine (FSM) implementations. This examines the ability of the stimulus to traverse the state-transition graph during simulation. As with path coverage, FSM coverage can become highly complex with large state spaces.
Other techniques include observability-based coverage, toggle coverage, and variable coverage, as well as assertion-driven and functional coverage.
Operating at a very low level – that of individual bits – toggle coverage suits gate-level analysis but has become a major component of power-analysis tools. Variable coverage is an extension of toggle coverage that considers bits to be grouped into larger variables. Coverage metrics for this analyze which values of the variable have been visited.
Coverage interoperability
Code coverage in hardware verification began with simulation. The trend in recent years is to expand the usage of coverage to encompass a wider variety of tools, such as formal verification programs that can exercise entire blocks in a fraction of the time of simulation, either through integration in single-company flows or through standards such as the Unified Coverage Interoperability Standard (UCIS), released mid-2012 by Accellera.
Other tools and techniques that can feed data to a common coverage database include static RTL analysis, otherwise known as linting, and graph-based verification technologies, such as that donated by Mentor Graphics to Accellera for possible standardization in March 2014.
Synopsys, on the other hand, used its purchase of SpringSoft to bring support for coverage into the Verdi debug tool, allowing engineers to see the coverage data for lines of code or simulation traces within the debug environment.
