Author Topic: NVIDIA cheats on Benchmark scores... (Read 382 times)

Terror · « **on:** May 23, 2003, 03:04:33 PM »

FutureMark has reported that NVIDIA is using "driver cheats" to get higher scores in the 3DMark03 benchmark.

Slashdot Article

TSK..TSK...

Terror

bloom25 · « **Reply #1 on:** May 23, 2003, 04:07:13 PM »

Both ATI and nVidia have done this in the past, and I'll bet both are still doing it. Does anyone else remember the Radeon 8500 Quake3 - Quack3 episode? Futuremark and nVidia aren't getting along well recently either...

Mini D · « **Reply #2 on:** May 23, 2003, 04:30:37 PM »

Yeah... I remember that quite well Bloom. It was hilarious.

I've never really put much stock in benchmarks. Especially when manufacturers started to cater to them.

AMD is running into the same situation with its new 3200+ processor. This happens when you rely on benchmarks to quantify your processors performance.

MiniD

ALF · « **Reply #3 on:** May 23, 2003, 06:46:59 PM »

DIRECT LINK TO PDF FILE

This latest fiasco is a clear attempt to defraud the public, nothing less. Its just pathetic that these companies tollerate and/or condone such blatent lack of ethics. Ive seen issues before over optimzasions that were unfair and clearly designed to give a better score on commonly used games, but now Nvidia has gone a step further, and made deliberate attempts to defraud the consumer by way of undermining the credibility of a popular benchmark. There can be no excuse for this incident, as the drivers apparently have code designed specifically to optimize very specific rander paterns only present in the benchmark and no where else.

I wish Futurmark could sue the crap out of nvidia for dilluting the value of their benchmark program with nvidias attempts to defraud the consumer.

Reschke · « **Reply #4 on:** May 23, 2003, 08:14:50 PM »

Why sue? Just because they are not part of the "beta group" for Futuremarks benchmarking software? Seriously though. I don't put much/if any stock in the 3DMark test. Real world applications and games instead of "synthetic" applications and games built specifically for the eye candy factor. While I admit that it is nice to see those types of screenshots out there. They are not something that we will be seeing in a real game for quite some time.

So the reason Futuremark should sue is because someone tweaked their drivers to show faulty scores on a test that other companies had previous access to and were "allowed" to build their drivers for pre-release. Should make you wonder.

Sox62 · « **Reply #5 on:** May 23, 2003, 10:46:02 PM »

3dmark has patched both 2001 and 2003 so the Driver shortcuts will no longer give an inflated score.

ALF · « **Reply #6 on:** May 24, 2003, 12:29:51 AM »

Quote

Originally posted by Reschke
Why sue? Just because they are not part of the "beta group" for Futuremarks benchmarking software? Seriously though. I don't put much/if any stock in the 3DMark test. Real world applications and games instead of "synthetic" applications and games built specifically for the eye candy factor. While I admit that it is nice to see those types of screenshots out there. They are not something that we will be seeing in a real game for quite some time.

So the reason Futuremark should sue is because someone tweaked their drivers to show faulty scores on a test that other companies had previous access to and were "allowed" to build their drivers for pre-release. Should make you wonder.

Nvidia made clear and deliberate attempts to falsify results of a specific benchmark with clear mallace of forthought....we are not talking about drivers that "just happen" to inflate a benchmark, or even drivers that have the clear effect of scewing results...we are refering to delibrate programing, line by line with no other possible result but the falsification of data from a well known and respected benchmark with full knowlege and contempt for futuremark and consumers.

There are MANY reasons to run benchmarking programs as opposed to just a mishmash of other programs, not the least of which is that cheating and fraudulent driver issues are much easier to detect, not to mention that those wishing to see what specific strengths and weaknesses of a system are can be determined in no other way.

It was WRONG, it was FRAUDULENT and it should be ACTIONABLE.

Mini D · « **Reply #7 on:** May 24, 2003, 03:30:25 AM »

Nothing says costumizing your drivers for benchmark performance is wrong ALF.

Perhaps placing any real weight on "benchmarks" might be the culprit?

MiniD

Skuzzy · « **Reply #8 on:** May 24, 2003, 08:43:14 AM »

Regardless of the program being run on a video card, the video card/drivers should render what is being sent down the pipeline without any alterations.

This is pretty fundamental. To do anything else is just wrong.

AKS\/\/ulfe · « **Reply #9 on:** May 26, 2003, 11:27:05 PM »

I stopped using benchmark programs several years ago, was still running a 486 DX4 100mhz at the time. For whatever reason, it was doing as well as Pentium 90s through 120s in the database in the benchmark scores.

I soon found out how completely f*cked those scores were, I upgraded to a Pentium 90 (overclocked to 120Mhz) and the same game I played on the 486 (Fighter Duel, which ran around 8-15fps on that system) was now running at over twice the framerate, and the systems ability to count hits was multiplied as well.

Benchmarks are pretty much worthless, if you are looking to see improvements in your system performance after an upgrade- just run a really intensive(CPU, GPU, and uses lots of system RAM) game and use Fraps to get the FPS.

Do you a lot better than a "benchmarking" program.
-SW