Building faster data centers with 25G Ethernet
John Swanson is senior marketing manager for DesignWare IP at Synopsys. He has been working in the IP business since 1990. The data center market is expected to grow 20% a year until 2021, according to analysts Research and Markets, to $90.56bn, due to the increasing number of connected devices in use and their need for fast cloud-based data processing. This is creating pressure on the Ethernet interconnect standard widely used in data centers to evolve to move data more quickly and efficiently. This has driven the development of a 25Gbit/s version of Ethernet.
Before 25G Ethernet was proposed, the next speed upgrade for data centers was expected to be 40G Ethernet (using four lanes of 10G) with a path to 100G defined as using 10 lanes of 10G. As Table 1 shows, however, the 25G Ethernet standard can provide a path to 100G and achieve higher total bandwidth than 40G.
Table 1 A 25G port speed provides a scalable path to 100G with additional lanes per port (Source: Synopsys)
The 25G/50G Ethernet Consortium initially defined the 25G Ethernet standard and promoted it through its member companies. Based on the Consortium’s initial work, IEEE standards for 25G Ethernet are now defined for both single-lane and four-lane versions of 25G. Table 2 shows these standards and the related electrical specifications.
Table 2 IEEE defined standards for 25G Ethernet and its target interfaces (Source: Synopsys)
The existence of clearly defined standards has unlocked rapid development of supporting hardware. The standard supports various hardware interfaces such as chip-to-chip, chip-to-module and backplane. Running chip-to-chip and chip-to-module connections at 25G should improve data-center system performance, while the backplane version of Ethernet will help the blade server market to evolve from 1G though 10G to 25G interconnect.
Figure 1 shows how 25G Ethernet can be used to interconnect different chips/modules, as well as providing connections for modules via passive or active cables at the port side. It also shows the 25G Ethernet standards that can be used to build a single-lane 25G Ethernet connection, a 2 x 25G set-up for 50G Ethernet or a 4 x 25G configuration for 100G Ethernet. This flexibility gives designers interfaces for use in everything from switching fabrics through to inter-module connections.
Figure 1 Pictorial view of 25G Ethernet applications on a data plane (Source: Synopsys)
25G Ethernet IP
Achieving the system throughput of 25G Ethernet, while meeting its timing budget and supporting selected 25G Ethernet features, is complex. Designers who want to work with the standard but don’t want to design it for themselves need subsystem IP that includes the media access controller (MAC), physical coding sublayer (PCS), and serializer/deserializer (SerDes).
The DesignWare Enterprise Ethernet MAC IP from Synopsys supports data rates from 1G to 100G, and is designed for three different system-level configurations: MAC only, MAC with a memory controller, and direct memory access (DMA) using an ARM AXI interface. The IP also offers native 128bit FIFO interface for minimal latency, Jumbo frame support and various application-specific configurations.
The DMA interface is included to ease the migration of 10G designs to 25G, as the first generation of mass-deployed 10G designs often use ARM’s AMBA on-chip interconnect. This is why the DesignWare Enterprise MAC’s optional AMBA AXI interface supports 128bit data transfers with up to 16 transmit and receive channels.
The MAC has a separate DMA channel in the transmit path for each queue in the MAC transaction layer (MTL). There are also single or multiple DMA channels for any number of queues in the MTL Receive path, and individual programmable burst sizes for Tx DMA and Rx DMA engines for optimal host bus utilization. Other options in the descriptor include:
- Byte-aligned addressing
- Dual-buffer descriptor ring and support for 64bit addressing
- Descriptor architecture to enable large blocks of data transfer with minimum CPU intervention
- TCP Segmentation Offload and IEEE 1588 Timestamps
- One-step PTP time correction in Tx
- Option to provide 64bit Timestamp for received packets
- Option to give up to two VLAN tags in descriptor for insertion or replacement in Tx and stripped VLAN tags in Rx
- Option to split the packet header (Layer 3 and Layer 4) and Layer 4 payload in different buffers
- Delivery of RSS hash information for packets received with TCP/UDP over IP payload
The DesignWare Enterprise MAC works with the Synopsys PCS layer, which is configurable and supports both IEEE and consortium 25G Ethernet specifications. Key optional modules include the Reed Solomon Forward Error Correction (RS-FEC) block defined by Clause 74 or Clause 91 of the IEEE 802.3 specification, link training support, and auto-negotiation. The PCS supports multiple interfaces including single 25G, 2 x 25G for 50G Ethernet, and 4 x 25G for 100G Ethernet.
The Enterprise MAC and PCS work with the DesignWare Multi-Protocol 25G PHY IP, which is optimized for signal integrity and jitter performance. It also supports Energy Efficient Ethernet strategies. The configurable transmitter and receiver equalizers enable users to control and optimize signal integrity and at-speed performance, and continuous calibration and adaptation features provide robust performance across voltage and frequency variations.
With Synopsys’ multi-protocol 25G PHY, all the components needed for a 25G Ethernet solution are available and fully tested. This means designers can take on tasks such as full subsystem simulations, synthesizing the subsystem with a variety of target process libraries, ensuring timing closure, and placement and routing. Synopsys also provides implementation and verification scripts to ease the configuration of the subsystem for the target application.
Author
John A Swanson is senior marketing manager, DesignWare IP, Synopsys. He has been working in the IP business since 1990 when he joined Logic Automation/Modeling, which was later acquired by Synopsys. Swanson has worked in the design, verification, integration and implementation aspects of complex IP in engineering methodology, and business development and marketing. He has been working on system-on-a-chip technologies and methodologies for over ten years with Synopsys in a variety of assignments. Currently he is the product line manager for the DesignWare Ethernet family of Digital cores as well as JPEG, 1394 and the IP reuse tools.
Prior to joining Synopsys he worked for Amoco Oil Company designing wellhead automation and control systems. He is an Honor graduate from DeVry Institute of Technology where he completed his engineering degree with Presidents List honors.
