Building a Better (Linux) GPU Benchmark

IQ Testing

To go further along the lines of the extreme "Average" frames per second scenario, we also need to consider Image Quality. Do we really want to play a game that runs faster on one card than another if it has poor shading techniques or unknowingly (or knowingly?) clips lighting effects? ATI and NVIDIA have both been batted around over "Optimziation" issues. Although the optimization accusations are more or less behind us now, a particular graphics manufacturer will release drivers soon that purposely enable optimizations (but will force you to enable them yourself). As new extra-tweaked driver sets begin to take advantage of these new open optimizations, knowing the image quality between different graphics cards becomes much more critical.

Again, we were extremely blessed with some very bright coders fully capable of abusing every last ounce of the OpenGL and SDL libraries. Below, you can see two side by side images of NVIDIA graphics cards captured at the same frame. On the top is a GeForce FX 5950 Ultra with single pass lighting, and on the bottom is the same card with multipass lighting on low quality. Click to enlarge either image.

Click to enlarge.

Click to enlarge.

Notice that these frames are slightly different. We have one or two bugs to work out in the frame capture program. In any case, we can still make some comparisons. All of our sample frames are captured in a lossless format similar to BMP, and then compressed with another lossless codec, PNG.


The lighting and textures speak volumes alone. Granted, these are extreme cases that we should not see from card to card. When it comes to comparing different card architectures, these are the kind of comparisons that we will take.

Final Words

Hopefully, you feel that our approach to benchmarking Linux games is an improvement over traditional methods. We have put a lot of work into preparing this new utility, and we will continue to perfect it in the future as well. The point here is not to eliminate our old benchmarks, but to augment average frames per second with thorough IQ and time-delimited data to give readers a better way to judge which products are better.

And of course, be prepared for our massive Linux GPU roundup in the coming weeks.