Open and proprietary verification tools home in on RISC-V core quality

With the power to customize comes responsibility and in the case of RISC-V that means doing more checks than designers used to have to do with processor cores where suppliers with control over both the macro and micro-architecture provided the their RTL with compliance guarantees. There is also the flexibility to do more checks on safety and security aspects, such as a core’s susceptibility to timing-based side-channel attacks that use inconsistencies in pipeline behavior to glean information about the data being processed.

The technical sessions at this year’s Design Automation Conference (DAC) as well as several product launches focused on the additional verification that SoC integrators need to do to incorporate RISC-V processors that will have significant customisation or where they have taken and used existing implementations but want to check their internal behavior.

“While the RISC-V [instruction set architecture] is free and open source it doesn’t mean you can get a processor core with mass-production quality. Unlike Arm or x86 processors which are backed by large companies, you need to take extra precautions if you want to take an open-source RISC-V processor to mass production,” said Google senior hardware engineer Tao Liu in a presentation at DAC.

“Traditionally, you had one vendor that would make sure it’s all compliant and so all you have to do is the integration testing,” said Simon Davidmann, founder and CEO of UK-based processor-modeling specialist Imperas.

Random instruction testing

This need to check architectural compliance and processor behavior motivated engineers at the search-engine giant to help build an open-source verification platform.

“The center of the platform is a random assembly instruction generator,” Liu said, which is built on the Unix tool YAML to help with the streaming of instructions into a variety of simulators. The typical setup for the RISC-DV framework is to run an instruction-set simulator (ISS) alongside an RTL simulation and then simply compare the results to look for deviations from the golden ISS model. A common choice for the golden ISS model is the one produced by Imperas, which has built high-speed instruction-level models for a variety of cores alongside scriptable analysis and debug tools.

“The generated program is extremely random and stressful [for the target],” Liu claimed, noting that it has been successful in catching corner-case bugs.

The first half of 2019 saw the first release, focused on the core instruction sets with support for extensions coming in the second half of last year. “In the first half of 2020 we formed a workgroup under the Chips Alliance to continue this effort,” Liu said.

A major part of the effort at the moment is to convert the generator into Python. The current generator was built using SystemVerilog. Liu said the Python option would make it possible to not have to use a commercial SystemVerilog simulator to run the generator code. However, the current need for SystemVerilog may not affect many commercial SoC integrators as they will most likely run their RTL testbenches using the same simulation engine.

That is not to say there are not teams aiming to provide commercial-grade open-source cores. The OpenHW Group, for example, is one that has taken processors designed in academia, such as those from the PULP project. “The OpenHW Group was established to create a viable set of cores based on open source. We are developing a full suite of verification tests for these cores,” said director of engineering Mike Thompson, adding that many of the tests the teams are using in the group use the RISC-DV generator.

Step and compare

Davidmann said the ISS-enabled step-and-compare approach is commonly used in both fully commercial testbenches and in those that use open-source frameworks such as RISC-DV. Typically, the ISS will not model the pipeline behavior directly so the comparison takes places after each instruction is finally completed. Thompson said the OpenHW Group’s testbench does not just model continuous instruction streams but also the effects of taking interrupts, invoking debug modules and microarchitectural issues. “We have assertions embedded that look for things like pipeline stalls,” Thompson said.

Customer funding to EDA companies has helped build out the tools infrastructure for RISC-V. Davidmann said one such project based on mutation testing has helped bolster the development of test harnesses. “The mutation testing tool lets us see which values propagate to the outputs, providing important information for improving test quality,” he said.

At DAC, India-based Valtrix Systems described the extension of its portable-stimulus testbench platform STING to cover the RISC-V vector and bit-manipulation instruction extensions. The platform supports a variety of verification approaches, from directed test through constrained-random to fully random instruction generation to exercise processor models. The tool works by injecting snippets of generated code into streams of assembly that perform setup or directed tests.

OneSpin has applied its formal-verification technology to a number of RISV-C cores as well as developing support to help optimize implementations. At DAC, OneSpin field application engineer Salahedden Hetalani said the company had found a number of issues in the 32bit RISCY processor: one of the portfolio supported by OpenHW. OneSpin has extended the support with tools that examine finite state machines in custom extensions for reachability as well as dead-code analysis, which can help support area-limited designs, he said. Mutation-based analysis provides another method to help determine whether the RTL has hidden issues.

Linting support

Aldec is using another static technology, linting, to help improve the code used for synthesis before it goes into the simulation and verification loop. Last week, the company launched a kit of checks for its ALint-Pro tool that are aimed at RISC-V projects. Louie De Luna, director of marketing at Aldec, said: “The RISC-V rule set is based on our expertise and our analysis of the available open-source RISC-V cores in the market including Ariane, Pulp, SweRV, and Hummingbird e203. We analyzed these cores and found common code inefficiencies.”

The linting checks look for problems such as signals that have been assigned but not used in the actual design, common finite state-machine problems, potential mismatches between synthesis and simulation and SystemVerilog constructs such as wildcard connections that often cause problems during debug.

As RISC-V gains ground and teams choose it as the substrate for their custom processor and accelerator, the roster of tools that focus on it seems to set to grow in both the proprietary and open-source domains, with projects now beginning to make use of machine-learning techniques to help make testing more productive.