FPGA use in HPCs plagued by bottlenecks

Gregg Cooke, a director in the emerging-technologies investigation team at a large investment bank in the United States, is not your typical user of FPGA design tools. But he's representative of a new breed of customers in fields such as finance, biosciences, and oil and gas exploration, where people with no FPGA hardware experience are trying to use the devices in high-performance computing systems (HPC)—and are often struggling with FPGA design tools.

FPGAs can potentially provide orders-of-magnitude speedups over conventional processors for compute-intensive algorithms, with greatly reduced power consumption. So it's no surprise that FPGA design is going far outside its traditional domain into a growing number of HPC systems, modules and boards.

The untapped market is potentially huge, but there's a problem. Current FPGA synthesis, placement and routing tools are written for hardware designers, not software programmers simply trying to accelerate an algorithm. A new generation of ESL C-language compilers is attempting to bridge the gap, but there's still a lot of work involved in moving C-language algorithms onto FPGAs.

Cooke is working with a large compute grid that's running bond risk calculations. Bonds have 30-year lives, and the calculations are extremely computationally intensive. "Before we knew it, we were shoveling blades into our data center just to feed this beast," Cooke said. "We were casting about for ways to reduce heat in our data centers, and FPGA technology seemed like a good bet."

Cooke is working on pilot projects that use FPGAs to accelerate the Black-Scholes algorithm, a stochastic process frequently used in equities trading. One goal is to allow software people to port existing C-language code directly to FPGAs, without having to touch VHDL.

New demand on vendors
Current C-language FPGA compilers don't support the full syntax of C, Cooke said. This includes "structs," or constructs that are heavily used when C++ is brought into C, and "switch" statements. "Hardware engineers don't use these constructs, so we're placing new demands on C-to-gates compiler vendors," he said.

FPGA placement and routing is another bottleneck. When Cooke tried placing and routing a large piece of code, his 16Gbyte machine ran out of memory. The FPGA vendor predicted the job would take a week and consume all 16Gbytes. "That's impossible," Cooke said. "We need to be able to make changes to code, compile and run, sometimes multiple times within a day."

Cooke has used the Impulse C compiler from Impulse Accelerated Technologies Inc. It's close to ANSI C and supports portability across devices, he said, but it could benefit from more efficient pipeline generation and smaller code size.

'There are stumbling blocks all along the way,' says Messmer.

Peter Messmer, deputy VP for accelerator technology at Tech-X Corp., a contract engineering company involved in computational physics, is running "particle push" simulations in which particles are accelerated and then evaluated for force, velocity and position. Messmer is also using the Impulse C compiler, along with FPGA boards from Pico Computing Inc. "We are desperate to get those particles pushed faster on an Opteron, a single particle push costs us on the order of 0.7?s," he said. Messmer is not yet sure what FPGAs can do, because he hasn't yet succeeded in getting the algorithm to run, but he's hoping for a five- to tenfold speedup.

Robert Shuler, an engineer at NASA working on software radio, wanted to use a C compiler to put sequential parts of his signal-processing code into an FPGA, and then found that FPGA tools and conventional C-language programs use different memory models. Thus, programmers can't really reuse existing C-language code in FPGA applications. "In most cases, it amounts to starting over and reprogramming the application," he said.

Market potential
The HPC market potential is not lost on FPGA vendors. "You start out with applications like financial or oil and gas, and they could start buying thousands of FPGAs," said Steve Lass, senior director of software product marketing at Xilinx Inc. "What I see is that this will eventually expand to accelerate pretty much anything." Right now, said Lass, Xilinx is mostly seeing people who are "kicking the tires" for FPGA-based HPC, mainly in the financial area. Current tools, he said, are just about good enough for that. "But when it comes to doing something like straight ANSI C with no modifications, the tools aren't there."

Lass acknowledged that FPGA placement and routing runtimes have to be a lot faster for HPC users. "The runtimes are in hours," he said. "I think if we can get it down to 10mins, that's what these guys would like to see." Lass also noted that the placement and routing tools are written for hardware designers, with error messages no software engineer is going to understand.

Similarly, one approach to FPGA programming is to construct APIs that hide the hardware details from the programmer, said Misha Burich, senior VP of software and system engineering at Altera Corp. "Somebody would develop functions on the FPGA and provide an API to the developer of the application," he said. In this sense, "the FPGA is not really programmed by the developer of the application." One solution, he said, is for ESL tools to hide the details.

Compiling into FPGAs; Virtual processor is built.

Compiler providers
Several companies provide compilers for FPGA-based HPC applications. Celoxica Ltd offers its DK Design Suite for C-based algorithmic entry, simulation and synthesis. The goal, said Jeff Jussel, VP of marketing, is to make FPGAs as easy to program as general-purpose CPUs. "The perfect thing is to let them write any type of C code," Jussel said. "But the reality is that you can write unsynthesizable C code. So there are certain things we have to ask them to do."

For handling parallel algorithms, Celoxica provides its proprietary Handel-C language. And to ease the programming burden, it offers specific compiler capabilities and libraries for selected algorithms. "If it's in the finance space and we know what you're doing, we can take raw C," he said. "For oil and gas, we know what an acoustic-wave algorithm looks like, and we can take that and process it."

"All tools, ours included, are making significant improvements," said Dave Pellerin, CTO of Impulse Accelerated Technologies.

- Richard Goering
EE Times