Aquamark 3 - Feature #33

With 3DMark03 weathering a storm of controversy, James Wang discovers a new DirectX 9 benchmarking hope deep beneath the ocean waves.

For better or for worse, 3DMark is now an entrenched part of 3D benchmarking. Some people like it for its pretty pictures, some enjoy watching the demos, while others simply want that magic number at the end. In any case, it has reigned undisputed as the industry standard 3D benchmark, until now. Sure, there have always been other alternatives available: GameGauge, VillageMark and Codecreatures are all well known, but none offered any tangible threat to 3DMark.  The exclusive preview we are about to present to you with represents possibly the first -- Massive Development's 'Aquamark 3'.

Krass –- made in Germany
Whereas 3DMark was designed based on the philosophy of combining synthetic and real world tests, Aquamark 3 concentrates purely on in-game testing. The krass graphics engine has been used in-house at JoWood productions (owner of Massive Development) for many years. With each technology leap, the krass engine has always been at the frontier of 3D. It was the first engine to claim DirectX8 compliance in 2001 and the game using this technology, AquaNox, was also the first DX8 title released worldwide.

Aquamark 2 was also one of the most elusive benchmarks around. Shown behind doors to showcase the power of NVIDIA's GeForce 3, it was one of the first demos to utilise pixel shading models. Now, in 2003 and the era of DX9, krass has been fully boosted to take advantage of conditional shaders, high precision colour and advanced physics. Like all the fine technology we've previewed recently, krass utilise the DX9 Higher Level Shading Language (HLSL) to produce fast and efficient code.

When it comes to shaders, people love comparing numbers. “How long is yours?” as if it's a testament of one's manhood. Which version is yours? What's the precision? How many registers does it use? At the end of the day, even the savviest 3D geek is not sure which measurement matters. Refreshingly, Aquamark 3 has taken a most sensible path in measuring shader performance. Real world fillrate still holds the key (which is good), along with some other interesting tests which will expose what's beneath the skin of 3D rendering.

Recall that shaders are merely descriptions of real world materials (Atomic 32, scanner). If a scene contains only a bare room, then only the brick shader would matter and hence there will only be one shader at work. Using a sensible solution like multi-texturing, the brick shader would be fairly simple. If the whole scene was instead replaced with water, then the number of shaders remains at one, but the complexity (number of instructions in the shader) has jumped dramatically. One can think of the number of shaders used as a measure of how many different types of materials are simulated while the complexity of the shader as an indication of how difficult this material is to model.

During the making of Aquamark 3, the developers realised that the number of shaders used bore little relevance to performance, but rather which shaders took up the majority of the scene space was the dominating factor. For example, if you implemented extremely complex shaders for killer piranhas but only a handful of them swam in the distance, the amount of pixels they account for is perhaps only 10 percent of the frame. With this in mind, Aquamark 3 uses a special measuring technique that only counts the pixels which contribute to the final framebuffer. In other words, pixels that are removed, culled or hidden will not be accounted for. It's good to see Aquamark 3's emphasis on actual output over theoretical prowess.

Meet Ingo
With such high emphasis of efficient shaders, wouldn't it be natural to write all shaders in HLSL? Not so according to Ingo Frick, Co-founder and technical director at Massive Development. 'HLSL implementations are not used for all shaders. There are still a substantial number of shaders being implemented traditionally, i.e. by assembly language or even by the classical texture stage combiner concept,' he says. The fact remains, for certain tasks, using shaders is simply overkill. Most of the objects and materials in today's rendering system tend to be simple, 'In AquaMark3, the number of pixels generated with ps2.0 shaders cover approximately 30% of the screen space. If you consider the total number of pixels rendered in the benchmark, only 10% are rendered with a ps2.0 shader.' Don't fret if you don't have ps2.0 hardware, fallback shaders have also been implemented which achieve the same effect with multiple passes but less precision.

Wrapping off this topic, we ask Ingo what shaders are easy to write yet profound in impact: 'The most easy to write yet beautiful shaders implement improved illumination models on a per pixel basis rather than on a per vertex basis. These models allow a more realistic material rendering than before and all this [works] in conjunction with novel shadow generation methods for soft shadows under extended and multiple light sources.'

ABOVE: Aquamark 3 emphasises commonly occuring rendering situation like overdraw or environment mapping.

He sure hit the right spot -- just about every FPS under development is pushing this rendering model. Ingo concludes by reiterating the fundamental shift to away from classic fillrate, 'the number of pixels will not be the limiting factor in future rendering process, but the complexity of the pixel shader behind each pixel.'

Since the arrival of DX9, higher precision colour has gone through fuelled debate in the online community. While more precision is most definitely useful, it is only beneficial if used with the right shader. Simple shaders will produce identical results for integer or floating point colour while some Renderman shaders will not tolerate anything less than full 32-bit floating point (FP128). Aquamark 3 uses the whole range from FP64 to FP128 but the results do not depend on specific FP precision. It simply means that if your hardware supports higher precision, it will have enhanced colour accuracy, but lower-end gear can still get by.

When we speak about code optimisation, Ingo also has positive comments for Microsoft's HLSL compiler, 'From our experience it [HLSL compiler] produces short and efficient code which seems to work well on every type of hardware without the need for specific code paths.' In fact, without some intimate knowledge of the underlying hardware, it is hard to beat the compiled code, 'With its pipeline-oriented approach and the relatively small instruction set, the compilers find preconditions which allow for highly optimised code generation'.

Let the games begin!
Aquamark 3's default test is roughly three minutes long. It is essentially one test but with different scenes at each point. At each stage, a particular subsystem of the graphics pipeline is stressed. The unique property of Aquamark 3 is that while the whole test is conducted in one seamless level, the different stages are radically different. For example, the polygon scene features hectares of what would appear as underwater mushrooms while the particle test has two giant machines churning out black smoke. Most impressive is probably the pixel shader test. The unveiled view shows vessels far in the distance about to engage in an underwater fire fight. The camera suddenly accelerates and we are taken