Global Sources
EE Times-Asia
Stay in touch with EE Times Asia
EE Times-Asia > Interface

Infiniband delivers flexibility

Posted: 01 Sep 2004 ?? ?Print Version ?Bookmark and Share

Keywords:infiniband? backplane? data center? server? interconnect?

Infiniband as a backplane technology offers an opportunity to relieve bottlenecks in scaling and managing systems in the data center. However, incorporating this technology requires a new generation of servers, switches and software tools that leverage the capabilities of the interconnect.

In recent years, data centers have aggressively moved toward scaling up distributed networks of servers. While arrays of rack-mounted servers have grown rapidly, they have not scaled efficiently. That fact has resulted in a decline in management ease, performance, resource utilization and flexibility. Also, total cost of ownership has become higher than expected. Infiniband-based architectures combined with virtualized software for installation and configuration will dramatically decrease costs while still meeting required service-level and business objectives. These systems offer new flexibility that, when coupled with an appropriate architecture, will allow for rapid remapping of applications to meet business needs.

Blade servers will be one of the first areas to benefit. The separation and virtualization of the storage from the compute elements transform boot images and data into network fabric services that can be associated with server blades rapidly, flexibly and transparently. This results in the capability to quickly map and remap workload requirements onto available resources, driving up utilization, availability and agility.

The consolidation of cabling, managed cards and ports, and the elimination of proprietary clustering interconnects significantly lower costs in the data center. The number of system components is reduced as well, increasing availability.

Cable management is simplified and the number of cables reduced using Infiniband. Although each server in a data center must be connected to the Infiniband switch, no further server-side connections need be made. Cabling effort can be cut in half in a simple installation.

With Infiniband, cabling to shared storage and service networks becomes rationalized to be a function of bandwidth requirements rather than the number of servers in the rack. As this occurs, the amount of Fibre Channel switch ports drops. Companies can buy fewer Fiber Channel switches or reduce the storage charge that system administrators pay, often calculated on a per-port basis.

Performance is enhanced because Infiniband supports remote DMA. Certain applications create significant TCP/IP overhead on the processor. Infiniband standards support direct-memory read/write by the host adapter into application memory. CPU cycles do not need to be consumed verifying data integrity and demultiplexing messages. This allows the data center manager to purchase fewer processors and fewer software licenses.

Infiniband opens the door to a new flexibility for using systems more efficiently. Businesses typically buy systems capable of handling their peak demands, resulting in unused off-peak processing cycles reported to be typically below 30 percent due to the inability to access unused processing resources across partitions. This is a result of architectures that strictly define and allocate servers to run specific applications and services.

To solve the problem of underused servers, computer makers have introduced clustering, load balancing, partitioning and several other solutions. However, each falls short of allowing data center managers to dynamically allocate servers from one partition to another. Also, the man-hours necessary to provision a server to a different application space with these tools can be prohibitively high.

Infiniband enables the data center manager to dynamically change the workload assigned to a server by provisioning the identity of the machine and logically connecting the server to the boot image of the new workload and associated data stores. This flexibility is a result of Infiniband's core architectural building block--the concept of disaggregation and recomposition of resources. Traditional servers with PCI are inherently more difficult to disaggregate. PCI was designed as a multimaster bus. Systems using a CompactPCI backplane have neither a failure model nor a programming model that provides isolation and a method for sharing I/O--issues that can result in system faults.

Infiniband defines a communications mechanism called queue pair that allows either server nodes or I/O gateways to communicate in a one-to-one or a one-to-many mode. Rather than establishing a PCI-like relationship where a server "owns" the adapter, Infiniband lets a server communicate with the adapter. This enables separation of the I/O from the server and effectively virtualizes the I/O. The virtualization, in turn, enables diverse forms of recomposition, allowing the dynamic change of system resources.

Users will need integrated system-management tools to take advantage of these advances. The tools will make it possible to manage disaggregated resources as a single entity, since the reconfiguration of the environment will no longer be traceable by cable labels or managed through sticky notes placed on a physical device, and because there are simply more pieces to manage due to disaggregation.

These management tools should include system configuration, provisioning and deployment, inventory control, patch and firmware management, as well as monitoring and alerts. Such functionality should be integrated with the servers' fault-management capabilities to ensure implementation of fail-over policies and proper reporting of faults, and to ultimately deliver a higher level of availability than would be implemented through manual processes.

This is the type of system Sun Microsystems is developing with its N1 Grid software.

With N1 Grid, Sun is working on management tools to allow the allocation of system resources by a simple point-and-click. N1 Grid enables system administrators to manage the system from one console rather than separate server, network and storage consoles.

Infiniband, coupled with N1 Grid, frees administrators from manual handling of hardware and software details, and lets them concentrate on resource utilization and system maximization. These features provide the capability for simpler and faster server deployment as low-level provisioning activities are automated.

- Subodh Bapat

VP and Chief Technologist

Volume Systems Products Division, Sun Microsystems Inc.

Article Comments - Infiniband delivers flexibility
*? You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.

Back to Top