Lots of questions from lots of folks: I'll answer what I can.
Vulcan: "That odd with the drive by downloads, I see loads of stuff blocked on active sites."
Maybe with more advanced SonicWall gear, but not with this one. I was really looking forward to this level of blocking, but Norton blocked the exact same number of drive-bys when the test PC was hooked up behind the sonicwall and when it wasn't. To be fair, no home-level UTM did very well.
Vulcan: "What do you mean about closing vs stealthing outbound ports? If spyware or a signature triggers a ips block outbound it sends a tcp rst."
I mean when you do a portscan of an IP address where the Sonicwall is connected, all the inactive ports are reported as "they exist and are closed" rather than "these ports don't exist, so there's no point in looking for a response."
Vulcan: "The AV sig set is supposed to reflect the active stuff out there. Even the bigger boxes only have around 28000 signatures. And to be honest the active stuff is probably less than 50 virus's, most of those being variants of about 5 core virus's."
Yes, it is supposed to, but even variants of the Storm worm were allowed through. I had a mix of both old and very VERY new viruses, and I can't say that any of the units did very well. Bottom line: 8000 signatures just isn't enough, even when focusing on new threats. Oddly, the SonicWall blocked a virus when it was zipped up, but let it through when it was a straight uncompressed EXE.
Vulcan: "The boxes are always sold as a secondary line of AV/AS defence (if you search my posts you'll see that, and that I always recommend Nod32)."
I got these lines from the vendors after sharing my results with them. When I was acquiring them, they really were being sold as a primary line of defense. That's disingenuous. More on this later.
2BigHorn: "Aye, Both manufacturer (Zyxel and Sonicwall) advertise virus defense for "in the wild" viruses."
True. Sonicwalls' homepage says "SonicWALL's family of network security appliances combines robust UTM security services with high-speed deep packet inspection to provide small, mid-size and enterprise-class organizations the best protection possible." CheckPoint says "Safe@Office keeps your network safe with proven technology. " Stonger statements are made deeper in their websites. That's the standard I held these devices to.
2BigHorn: "Also, all 4 products are not even closely in the same class, nor the features correspond the price listed."
Also agreed. But we sent the same parameters to many vendors, and these are the 4 that responded in time to meet deadlines, and these are the products they sent in response to the testing parameters. On top of that, most of the vendors knew what the other vendors were sending me when they sent me theirs. In other words, I didn't select these products. The vendors selected them based on our review criteria, which they knew about ahead of time.
2BigHorn: "Don't know why are you listing number of CPUs (all different specs), but not the memory (very important)"
"CPUS" refers to the CPU Magazine scoring system, where 1 star is terrible, 5 stars are perfect, and 2.5 stars average. The magazine refers to the scoring in language like "This product earns 3.5 CPUs." Not my system, BTW, but its is required in all reviews.
2BigHorn: "Testing throughput on 6Mbps connections is not adequate. Many users have 10Mbps down or more. Would be bugger to buy TZ180 just to find out it limits your 15Mbps connections to 10Mbps when you turn everything on."
I initially tested throughput speed reductions by setting up servers on a 100 MB internal LAN, and partially though testing, editors wanted my tests changed to a "real" internet connection. The neighborhood where I can test with a FIOS connection was having problems due to water damage that month from a sewage problem (murphy's law), so a 6mbit DSL connection was it. I agree that a 6MBit connection is not the fastest connection a power user might have in this day and age. This is indeed a shortcoming of the review.
Vulcan: "Wow just scanned the review, who did it needs a kick in the pants : 'Content filtering is based on the administrator (that is, you) entering URLs and IP addresses rather than selecting topics and letting the UTM decide what’s acceptable.' That is in blatantly incorrect."
The categorized content filter/monitor was not included in the unit I received from SonicWall, where I understand it to be an extra-cost option and subscription at this price point. The lack of its presence was noted, but it didn't affect scoring. Our initial request for units did not specify the need for this feature, but when it was present in all the other models, is absence was merely noted, as the initial paragraphs stated it would.
Needless to say, all the vendors got in touch with us after the review went out. None were very happy, as you can imagine. Some requested changes, and where they were right, we made them. Sonicwall did not request a change to the content filtering statement.
Not everyone was unhappy, however.
Watchguard sent me a similar unit, but it arrived too late for the review. Here's what one of their product managers wrote me when I said I would be happy to play with it, but it probably wouldn't lead to a printed review:
"Yes, I did read the article, and it looks as if you pulled no punches,
which is exactly why I'd like to share any feedback you have with our PM
team. Even if it doesn't make it into print, it's always helpful for us
to get real-world feedback. And, especially since you beat up the other
players, I know you'll be giving us the same, fair pounding!"I'll close with this: tech journalists and reviewers are normally bashed for not testing thoroughly enough, or going too easy on non-performing products. It was nice not to be accused of this for a change. ;-)
-Llama
