Futuremark, developer of benchmark tests for measuring graphics-chip performance, has claimed that it found code in NVidia and ATI drivers that it says 'cheated' when tested.
NVidia's code allegedly detected certain tests that altered the graphics chip performance to inflate results. Futuremark claims that some NVidia drivers have "detection mechanisms" that trigger a higher level of performance when certain tests within its 3DMark03 benchmark are detected. ATI Technologies, NVidia's rival, is also alleged to have altered its drivers in order to boost test results, Futuremark said.
Futuremark claims it identified eight instances where NVidia software detected specific 3DMark03 tests and inflated the results to the detriment of overall image quality.
3DMark03 has been criticized by some as favoring products from ATI over those of NVidia, said Peter Glaskowsky, editor in chief of the Microprocessor Report. The benchmark contains certain DirectX calls that favour ATI, he said, and NVidia appears to have been addressing this by altering its drivers to convert that type of DirectX call into one that works better on NVidia hardware, Glaskowsky said.
A new version of 3DMark03 with slightly different code that blocks the NVidia drivers from detecting specific tests reduced the benchmark score of a system with Nvidia's GeForceFX 5900 Ultra and the 44.03 driver by 24 per cent, Futuremark said.
Chip makers often cite benchmark results in marketing materials when announcing new chips. Benchmarks are virtually the only method of comparing performance of competing chips or systems, but are often criticized for not mirroring real-world performance.