Global Sources
EE Times-Asia
Stay in touch with EE Times Asia
EE Times-Asia > Networks

Driving 10Gb Serial ATCA backplanes

Posted: 01 Jun 2005 ?? ?Print Version ?Bookmark and Share

Keywords:backplane ethernet task force? ieee 802.3? ethernet? backplane? 1000base-kx?

By John D'Ambrosia, Tyco Electronics

Riccardo Badalone, Diablo Technologies

The Backplane Ethernet Task Force was formed in May 2004 by the IEEE 802.3 to standardize transmission of Ethernet traffic across a backplane. The task force is now focused on developing three new physical layers (PHYs): 1000BASE-KX (serial 1G PHY), 10GBASE-KX4 (4-lane 10G PHY), and 10GBASE-KR (serial 10G PHY). The 10GBASE-KR is of particular interest because there is no comparable technique available. Of course, the task force is also challenged to live up to the Ethernet motto of "10x the performance at 2x the cost." Undoubtedly, developers of advanced telecom computing architecture (ATCA) platforms are watching the 802.3ap efforts closely, with the intent of leveraging the standard to lower development costs and deliver on the promise of more cost-effective system solutions.

Total solution cost is the major issue when developing a PHY capable of delivering 10Gbps across a backplane. One industry position argues that significant improvements to the channel would simplify the task for the serializer/deserializer (serdes) developer, thereby reducing power, and ultimately saving costs in thermal management. Another perspective argues to make the channel as inexpensive as possible and force the serdes developers to work smarter. Consensus will likely be reached by finding a middle ground, as today's market will not allow system vendors to develop solutions that are too expensive.

Boosting density per signal layer

Developing the lowest-cost channel capable of supporting higher serial speeds requires an understanding of the Active Interconnect, which looks at the total system by analyzing the synergistic performance of the active and passive components of the channel. Two years ago, the QuadRoute technique was introduced, which uses the Tyco Electronics' Z-PACK HM-Zd connector for ATCA backplanes. The technique essentially doubles the routing density per signal layer, which helps to offset increases in material costs by decreasing processing costs. For instance, with this technique it is possible to route a full-mesh 14-slot ATCA backplane in 8 signal layers. (The standard implementation of a full-mesh ATCA backplane requires 16 signal layers, for a total layer count of 36 layers.) The decrease in signal layers helps to reduce the overall backplane thickness to 0.125 inch, which helps to shift the null effect (associated with top layer connections in a backplane) to higher frequencies (Figure 1 below). As a result, overall cost of the backplane and performance is improved.

Developing viable test cases

To create viable test cases for examining 10Gbps serial speeds, full mesh and dual star ATCA backplanes based on the QuadRoute technique were obtained. The full mesh backplane uses Nelco 4000-13SI, and has a maximum differential pair trace length of approximately 20 inch (508mm). The dual-star backplane uses Nelco 4000-13, and has a maximum trace length of approximately 254mm. Line cards of different lengths and materials were created to develop a suite of different test cases that are representative of real-world channel scenarios. Four-port s-parameter characterization of the forward channel response, direct adjacent near-end crosstalk (NEXT) aggressors, and direct adjacent far-end (FEXT) using the Agilent 8720ES with N4418A test set was completed by the University of New Hampshire Interoperability Laboratory.

Within the various system configurations, two different channel scenarios were then chosen:

Long versus short channels (Case 1 versus Case 4)

Comparison between top and bottom layer routing (Case 2 versus Case 3)

Table 1: Summary of the specific parameters for the different test case scenarios.

Figure 1: Diagram showing examples of top and bottom layer connections.

The test scenarios were designed to capture the various situations that challenge backplane and system designers who are targeting data rates above 10Gbit/s. For each case, one or more channels was fully characterized and included in a high-level simulation environment. The simulations sought to capture the effects of the entire data path, including the effects of the package, connectors, traces, and various other discontinuities or imperfections. This data path from chip-to-chip is illustrated in Figure 2. The "channel" would be considered the total length from the ball of one device to the ball of the other, and would include segments N2, B, and H. Models for the package and various aspects of the silicon design would be cascaded with the channel to complete the data path.

Figure 2: The simulations sought to capture the effects of the entire data path, including the effects of the package, connectors, traces, and various other discontinuities or imperfections.

Analyzing performance

The channels were selected to represent the challenges that developers of ATCA backplanes face. Longer channels have often been thought to be the most difficult to make work because of small signal-to-noise ratios. Shorter channels, however, can also be very challenging. For instance, consider the forward channel responses of Case 1 and Case 4 (Figure 3). Case 1, due to its bottom layer connection has minimal stub, therefore the discontinuity is limited to the plated through-hole itself. As a result, the primary distortion of the channel is attenuation due to skin effect and dielectric losses. In Case 4, however, there is some stub left over. The overall short distance of the net and the 1 inch (25.4mm) adjacent slot spacing causes multi-path reflections that result in a "ripple" in the channel.

Figure 3: In Case 1, the primary distortion of the channel is attenuation due to skin effect and dielectric losses. In Case 4, adjacent slot spacing causes multi-path reflections that result in a "ripple" in the channel.

The problem with analysis exclusively in the frequency domain is that there are no real insight as to what the actual signal looks like. Therefore, impulse or pulse responses are often used to characterize the channel. Figure 4 shows 10 Gbit/s pulse responses for Cases 1 and 4. Normalized to baud intervals, t0 is set to the midway point in time between the 50% crossings of each pulse.

Figure 4: Cases 1 and 4: 10Gbps pulse response.

The first noticeable difference is the disparity in the peak at t0 between the two pulses. Case 1 (which is the longer channel) experiences a greater reduction in its peak value at t0, as compared to Case 4. Furthermore, Case 1 is experiencing more "blurring" so there is also significant precursor contribution at t-1 and t-2. The reduction in amplitude at t0 makes the percent impact of the pre-cursor and post-cursor contribution even more significant. The tail of Case 1, however, is relatively smooth will little indication of reflections. On the other hand, while Case 4 has higher amplitude, it also shows the reflections that occur between the partial stubs of the adjacent slots on the backplane. Furthermore, there are small reflections that are happening twenty to twenty-five bits later. Note that these are pulse responses of the channel by itself without any packaging or IC effects. Additional reflections would be anticipated with their addition, and it is expected that Case 4, because of the partial stub on the backplane, would be more sensitive to their addition than Case 1.

In Cases 2 and 3, the overall channel lengths are similar with the only real difference being the backplane layer connection. (Case 2 is a bottom layer connection on the backplane, while Case 3 is a top layer connection) Figure 5 shows the SDD21 profile of the two cases. For those familiar with the classic top versus bottom layer connection, this is a very familiar graph. However, note that the null has been pushed out in frequency, as compared to what one would find with a typical 0.225inch thick backplane, which can be attributed to the thinner backplane thickness that the QuadRoute technique enabled.

Figure 6 shows 10Gbps pulse responses for Cases 2 and 3. Normalized to baud intervals, t0 is set to the midway point in time between the 50% crossings of each pulse.

Figure 6: 10-Gbit/s pulse response for Cases 2 and 3

In this instance, the responses of the two channels seem similar, but there are three distinct differences. First, despite the fact that the two nets are similar in length, Case 3 experiences a reduction in its t0 value, as compared to Case 2. Next, the stub of Case 3 causes an increase in the pre-cursor contribution at t-1, as compared to Case 2. The last difference to note is that for Case 3, reflections are occurring approximately 27 to 35 bits later. The difference in time in comparison to Case 4 can be attributed to the fact that Case 3 is longer than Case 4, thus requiring more time for the reflections to occur.

Correcting for effects

As noted so far, being able to compensate for pre-cursors for data rates beyond 5Gbps becomes an important aspect of the equalization solution. The introduction of decision feedback equalization (DFE) to modern backplane serdes solves the post-cursor equalization issue as well as most reflections, but, since it works based on decisions, it cannot compensate for the effects of data that has not yet been sampled. A DFE with six taps can correct for the effects of inter-symbol interference (ISI) and any reflections that may be located within 6 unit intervals of the cursor sampling point. All the simulation results in this report are based on a DFE whose performance is consistent with that of a Diablo DFE implementation (a combination of a continuous time filter [shaper] and a DFE). This includes the effects related to finite resolution, finite tap weights, limited loop bandwidth, and propagation delay.

Both components of the equalizer are adapted by the same algorithm in tandem, and the use of transmit pre-emphasis can be completely avoided. The simplified architecture is highlighted in Figure 7, where it should be noted that there is no inclusion of a transmit filter in the system. The transmitter needs only to transmit the binary data at the same frequency as that which is expected by the receiver. The shaper is a fully continuous circuit that is constructed with programmable gain amplifiers. There are no latching mechanisms or clock requirements in the shaper.

Figure 7: Block diagram of the Diablo receiver-based equalizer for lossy backplanes

Figure 8 shows the output of the shaper overlaid with the output of the channel. The yellow pulse response is the un-emphasized output of the channel. The purple pulse response is the filtered output of the shaper. The shaper provides benefits beyond pre-cursor ISI compensation. It also significantly shortens the post-cursor tail, which reduces the magnitude of the DFE taps and may also require fewer DFE taps if reflections are not considered. This effect may significantly reduce the probability of a burst error caused by the DFE.

Figure 8: Pulse response overlay of channel output and shaper output

Using the NEXT and FEXT crosstalk full characterization data, the equalizer's proprietary simulation bench was used to determine if the channels in question would support the target data rate of 10 Gbit/s. Random and deterministic jitter was also injected into the simulations. To approximate the bit error rate (BER) performance of the simulation, a method was employed that used a statistical analysis of sampling point data from the simulations. At every sampling clock edge, the amplitude of the equalized signal is compared to the target signal reference. The true mathematical error of the signal level with respect to the desired signal reference is obtained, and is used to obtain a vector of errors for N decisions (Figure 9).

Figure 9: Obtaining signal "error" for BER approximation

Figure 10 shows the simulated results for Cases 1 and 4 with all signal impairments (crosstalk, jitter, etc) added. Figure 11 shows the simulated results for Cases 2 and 3 with all signal impairments (crosstalk, jitter, etc) added. The BER values generated by the simulator are 9.01-16, 9.3-16, 1.06-16 and 2.6-16 for Case 1, Case 2, Case 3, and Case 4 respectively. Assuming a fully Gaussian distribution, all channels met the target BER of 1-15.

Figure 11: Simulation results for Cases 2 and 3

For Cases 1 and 2 (which were both bottom layer connections) there was minimal difference in the reported eye-openings or BER. This is extremely interesting; given the fact that there is an 18-inch difference in the lengths of the two cases. This is an example of how ISI is correctable.

Cases 3 and 4, which had approximately 0.100 inch and 0.075 inch stub respectively, had a 15 percent to 20 percent reduction in eye height. (Recall that Case 3 is only 22 inch while Case 4 is 13 inch) Therefore, loss was not the issue, but reflections were caused by impedance discontinuities (associated with plated through holes that were not corrected) and higher power (delivered through the channel and to the receiver due to shorter channel lengths).

Keys to success

There has been a prevailing opinion in the industry that better materials can be employed to achieve greater reach. While there is a certain logic to this, it must also be noted that in a backplane environment all routing channels are typically used to minimize layer count, so on a given layer, minimum and maximum trace lengths could most likely co-exist. Therefore, the use of better materials to achieve longer reaches could be detrimental to the performance of shorter links by increasing the delivered power through the channel and increasing the magnitude of subsequent reflections.

Therefore, the key is to understand the problem and design accordingly. Understanding the passive channel drove the design of the cost-effective QuadRoute technique with the Z-PACK HM-Zd connector used in ATCA. Understanding the problems inherent in a lower-cost channel led to a unique equalization solution. Bringing together active and passive expertise allows the total system problem to be addressed. This has helped to drive the development of a cost-effective solution that has been demonstrated in the ATCA form factor to support 10Gbps serial speeds while meeting a target BER requirement of 1-15.

About the author

John D'Ambrosia is the manager of semiconductor relations for Tyco Electronics. He received a B.S. in Electrical Engineering Technology from the Pennsylvania State University and a Master's Degree in Engineering Management from the National Technology University. Riccardo Badalone is the CTO of Diablo Technologies. He received a B.S in Electrical Engineering from Concordia University in Montreal, Canada.

Article Comments - Driving 10Gb Serial ATCA backplanes
*? You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.

Back to Top