How to achieve 200-400GE network buffer speeds

As network line rates and packet rates are growing, the need for high efficiency, reduced latency, fine granularity interfaces to memory and coprocessors has become critical. Buffer traffic at 400GE will require approximately 900 I/O pins at 3.2Gbit/s to DDR4 memory. Any additional off-chip memory operations for header processing would require as many more pins again.

Many designers will try to integrate all the memory on chip, and this will put challenges on how much computing resources can also be included on the same die in the face of requirements to improve computation by 4x. Even with advanced packaging, these pin counts are not achievable when the line interface and power pins are included. I/O pins exact a cost not only in larger packages, die area, but also power. Being I/O efficient is an important aspect of today's architecture. Protocols play a large role in efficiency of information transfer.

However, currently available device-to-device serial interfaces used to deal with such latencies suffer from several shortcomings including channelised one-way transport or they target specific applications, such as memory. These interfaces may also be optimised for large data packets, with the result that they may suffer from inefficiency due to the structure of their transactions with an ASIC or FPGA. Inefficiency arises because load/store transactions to and from memory occur in small synchronous transfers of data, such as 72 bits (64 bits + 8 bits of Electronic Dispersion Compensation). Such inefficiency incurs costs in the form of extra memory, additional traces, and therefore increased board real estate.

To meet these challenges, MoSys has developed a serial chip-to-chip transport protocol that operates over OIF standard CEI SerDes. It is touted to achieve 90% efficiency. The protocol, called the GigaChip Interface (GCI), can be scaled to 1, 2, 4 or 8 SerDes lanes as well as multiples of 8s. It targets computational and memory solutions with serial interfaces for networking equipment such as the Bandwidth Engine. Operating on existing devices with 16 lanes at 15Gbit/s, the GCI provides enough bandwidth to support 4.5B read/write transactions and sufficient bandwidth to buffer full duplex 200GE. Doubling the pins or doubling the line rate (30Gbps) achieves full duplex 400GE.

After briefly describing the two most common transmission protocols – packets and data word translations—this article will provide details of the GCI protocol and its various layers and show how it can be used to achieve performance improvements in a typical system.

Channelised packets versus data words
The most common transmission protocols serve one of two categories: Packets or Data Word transactions. Packets are transported through multiple devices, one of which is often a switching device. Due to multiple end points and potential congestion in switches, packets face the reasonable possibility that they will be dropped. To address this problem, any serial interface protocol requires more complex error checking and flow control mechanisms. As shown in figure 1, the protocol must provide the means to communicate multiple fields, including individual packet ID, priority levels, packet types, and end-to-end port ID fields.

Packet transmissions through the core electronics of networking equipment typically exhibit the following characteristics:
???Data rates of n x 1/10/40/100Gbit/s
???Variable length packets from 64B to 1.5KB
???Packet arrival rate varies based on Data Rate and Packet Length
???Asynchronous transfer mode
???Reach of 8-30 inch possibly through connectors
???ASSP/ASIC/FPGA to and from network PHY or back plane