Nvidia's CUDA

Nvidia’s sidestep from gaming could predict a brave new world for domestic computing.

A new use for your graphics card is emerging. It’s known as GPGPU or General Purpose computing on Graphics Processing Units, and the idea is to harness the immense power of consumer graphics hardware for computation.

And it’s gaining momentum. In February last year, Nvidia released CUDA - or Compute Unified Device Architecture - an interface for programmers to write software to execute on its Geforce 8 series (or better) GPUs. This name is a cute backronym, but the floodgate it’s opened is much more serious.

The way GPUs process data is slightly different to x86 CPUs and their relatives. In a CPU cycle a couple of things happen: the CPU loads and decodes its next instruction, the Arithmetic/Logic Unit (ALU) does the grunt work of performing the operation and then spits out the result which then needs to be stored somewhere.

Say you have one million data points which all require an “add this to that” operation performed – for example increasing the brightness of an image in Photoshop. Processing the pixels one at a time will take at least a million cycles. If, however, you have lots and lots (a technical term) of ALUs, each performing “add this to that” on some of those million data points, the number of cycles is massively reduced. There are other delays involved -- such as fetching from memory -- however the principal is the same. Correctly channelled, it’s an epic amount of power compared to the piss-weak four cores on a Core 2 Quad. The CUDA approach has a massive amount of things going on at very high speed, like a leopard in a blender.

You might ask why CPUs aren't built this way. The catch with GPGPU is that the threads are specialised and each needs to be running the same instruction. This model of computing is known as SIMD – or Single Instruction, Multiple Data. For multimedia acceleration, a very specialised and limited set of SIMD operations have found their way into CPUs in the form of Streaming SIMD Extensions (better known as SSE), but these are very limited in terms of versatility and still operate in a serial manner.

Unfortunately, not everything is nicely parallelisable, and for normal office and internet use massively parallel processing doesn't make any sense at all. For the casual or business user, x86/x64 is still better suited to the task.

So GPGPU processing isn't for everyone, but there are a lot of very specific applications where massively parallel processing beats the mostly serial x86 processing we’re used to. Multiple core CPUs are trying to introduce parallel computing to the desktop, but can only process a handful of threads at a time. A GPU core is comprised of lots of little specialised processors each capable of running multiple threads. The word on the street is that a Geforce 8800 only reaches saturation when at least about 25,000 threads are running simultaneously. Perfect for massively parallel number crunching.

In the real word, that means a massive speed boost for data-heavy operations like simulating organic molecules, weather, image recognition, anything to do with multimedia and Newtonian physics. While the first three only help blast through someone’s PhD research, multimedia is everywhere and Newtonian physics will help you blast through a friend and admire the view as their entrails realistically splatter throughout the game world.