Benchmarking the network-on-chip

From its inception, the OCP standard was designed to address the advent of heterogeneous processors and multicore SoC development. Since the OCP-IP organization opened for business in December 2001, it has established eight technical working groups (WGs) to develop tools, technologies and products that support the standard, leading in turn to the release of a wide range of valuable ‘deliverables’ that are offered to members free-of-charge.

The OCP efficiently supports and ensures re-use of the most complex IP cores deployed in diverse SoC design flows and methodologies, while elegantly and frugally supporting the needs of less complex functions in simple design applications and FPGAs.

Among OCP-IP’s working groups, five (Specification, Cache Coherence, Debug, NoC Benchmarking and Meta Data) directly address multicore processing requirements. Of these, the NoC Benchmarking WG was formed in the second half of 2006 in specific recognition of the need for benchmarking to advance beyond that for the stand-alone embedded processor.

Today’s challenge is to address performance from a broad perspective, acknowledging that multiple and diverse processors (the embedded processor, DSP, media processor, graphics engines etc.) can and do exist in the same design. The issue is no longer the performance of any individual processor, but of the system – and in today’s world of SoCs, the network is the system such that NoCs increasingly fulfill this system role.

The NoC BWG is drawing on skills from both academia and industry. Researchers from the University of British Columbia, Carnegie Melon University, the Royal Institute of Technology, Tampere University of Technology, and Washington State have teamed up with experts from such companies as GreenSocs, Nokia, Sonics, Synopsys, Texas Instruments, and Toshiba. This group has already delivered a landmark white paper, “An initiative towards open network-on-chip benchmarks” designed to help create a consensus and consolidate group and industry thinking around the topic. The paper met these objectives and has subsequently enjoyed thousands of downloads from the OCP-IP website (you can still access the document for free at www.ocpip.org/socket/whitepapers/). With this ground-work in place, the NoC BWG team developed a two-part specification that is now complete and freely available to members and researchers (under a clickthru license) from OCP-IP.

The first part of the specification addresses the needs and formatting of the NoC itself around standardized configurations for benchmarking. It details requirements and features for application programs, synthetic micro-benchmarks, and abstract benchmark applications. It also discusses ways to measure and benchmark reliability, fault tolerance, and testability of the on-chip communication fabric.

The second part of the specification describes the configurations and processing required to format micro-benchmarks so that they can be readily deployed in a broader NoC benchmarking scheme. It contains sections on architecture definition, traffic configuration, measurement and the micro-benchmarks set.

‘Architecture Definition’ covers the elements of the NoC model under evaluation, communication entities (transaction, transfer and packet) and standard communication interfaces. ‘Traffic Configurations’ discusses the temporal and spatial distribution of NoC traffic data for performance evaluation. The ‘Measurement’ section details workload and metrics. Finally, the ‘Microbenchmarks Set’ section covers naming conventions for microbenchmarks, and micro-benchmark implementations.

With the philosophy behind and challenge in the work fully described in the original white paper and the two-part specification in place, it is now possible for researchers and industry experts alike to more readily explore and exchange ideas and directly compare NoC architectures and configurations via a standardized approach.

The NoC BWG itself continues to meet regularly and is exploring plans for sharing benchmarks with the broader academic and industrial community. If you are interested in joining this work, contributing or learning more about our activity, please contact us at admin@ocpip.org.

Ian Mackintosh is president and chairman of OCP-IP