Big Iron Secrets

Currently, the key differentiators from your GigE NIC in the back of your motherboard and the kit you’d find in a rack in a cold room somewhere are very few. Technologies such as TCP/IP Offloading Engines (TOE), stack compression and hardware decompression are nothing new. Where the enterprise market edges forward however, is in the 10GigE and Infiniband market for both high performance SAN technologies and HPC/Compute dense applications. One such example of the use of an Infiniband/10GigE Fabric put to good use is TACC Ranger, currently the most powerful supercomputing centre.

TACC Ranger, currently the most powerful supercomputer in the world

The Constellation data centre switch in this building (Figure 8) was designed with 2,048 Infiniband ports to accommodate a total of 49,152 Gbit/sec throughput. To see the Constellation switch in action take a look at the Ranger tour videos.





Both Infiniband and 10GigE FC links are still at a price point where only enterprise computing budgets can afford such throughput. These technologies are hard to put close to the home user’s budget because of semiconductor fabrication costs and operating system incompatibilities/scalability issues. When we consider that the average 10GigE optical link over long wave small form-factor pluggable transceivers (SFP) is capable of ~1000MB/sec and that generally, these connections will connect to a SAN environment or 4Gig/sec switching HBA (~400MB/sec), you’ve still got throughput that consumer level hard disk and storage appliances are not capable of performing at.

The Sun Constellation Infiniband switch