All,
Well, I submitted my security suite software review to the magazine today, and boy am I glad its over. Look for the full article in the July issue of CPU. At newsstands everywhere.
Just doing the full suite of performance tests took about 3.5 hours for each product, and I looked at 9 products. You do the math.
Anyway, I'm going to reveal some information about benchmarks and such.
For the past few years, I've done performance benchmarks on an old and slow machine, to artificially increase the differences between products. But not anymore. The Steam Hardware Survey shows that 70% of players have 2GB of RAM or greater, 75% are running XP, and 50% have CPUs ranging between 2.5GHz and 3.3GHz, so I tested on a machine that more representative of modern gaming machines: a Core2Duo at 3.0 GHz, 4GB or RAM, and Windows XP.
The short story is that machines of this caliber are finally strong enough to run security suites without being significantly bogged down. All the security products had 3DMark06 benchmark scores within 2.5% of the baseline machine, coming down to a .2% difference for CounterStrike:Source framerates.
Though not part of the article, I was curious if benchmarks for standalone AV products were significantly different that the full security suites, so "off the clock" I ran the same benchmarks for the standalone AV products too.
So here are some results for the top two products in my review, along with their AV-only cousins. These results are averages over multiple runs (at least 3, sometimes more), with a reboot between each run.
Boot Times:
Clean: 32 seconds
Eset Smart Security: 35
Norton Internet Security 2009: 41
Eset NOD32: 34
NAV2009: 38 seconds
3dMark06:
Clean: 11759
ESS: 11753
NIS: 11747
NOD32: 11756
NAV: 11743
PCMark05:
Clean: 8969
ESS: 8992
NIS: 8953
NOD32: 8993
NAV: 8922
CounterStrike:Source
Clean: 279.24 fps
ESS: 277.97
NIS: 278.20
NOD32: 279.05
NAV: 278.38
You'll notice that occasionally a benchmark ran faster with a security product installed than with a clean run. It sometimes happened. I tested repeatedly in these cases and usually had the same result repeatibly.
Conclusions: NOD32 is always the fastest product, and faster than its full Security Suite cousin, but the differences between all products here are almost within the margin of error, if you ask me. I don't think its worth selecting an AV product or security suite based solely on these benchmarks.
Base your selection on how not-annoying the security product is (and that includes game interference), then its detection rates (as determined by a very large malware sample), then its useful features, then its benchmarked speed, and then (optionally) its price. Using this criteria, Eset Smart Security and Norton Internet Security are the two best products for power users.
I encourage your questions...
-Llama
