 |
|
|
VA Test Results Symantec Enterprise Security Manager We did not include Symantec ESM in the performance test runs, since it is purely a host-based scanner, more concerned with enterprise-wide policy auditing and enforcement. It performed well in our overall evaluations, however, proving itself to be powerful, flexible and easy to use, with excellent reporting capabilities. It also supports a wide range of host platforms, and would make an ideal companion to any of the more �conventional� VA products tested here. Click here to return to Symantec ESM Review NetRecon was the product that gave us most cause for concern. Concern, that is, in that it made our test systems appear almost trivial to break into, reporting, as it did, over 1700 vulnerabilities on the first run!
Time taken to scan 3 machines: 11 minutes 41 seconds The problem with being so thorough, of course, is that the resulting report is simply too large to handle and so is almost as bad as having no report at all. To be fair to Symantec, it was our methodology that was largely at fault here, since we insisted on running the heaviest scan available on all products. Symantec recommends you start with the Light scan, fixing the problems found there before moving on to the Medium scan. Having fixed the problems discovered there, only then should you attempt to run the Heavy scan, by which time the resulting report should be much smaller. When we re-ran the tests using the Medium scan instead, we came up with the following results:
This proved to be a much more manageable report, and NetRecon found all the obvious security flaws we had left in place following our default installations. The reports were clear and easy to read (if somewhat extensive on occasion) and the simple operation makes it straightforward to use by those administrators who are not security specialists. Click here to return to Symantec NetRecon Review BindView bv-Control for Internet Security With bv-Control we created a new Security Check group that included every check in every category except for the unsafe Denial of Service checks. On running this we were presented with a fairly short report, compared to the other products on test, with the following results:
Time taken to scan 3 machines: 12 minutes 14 seconds These results are a little worrying, and indicate that BindView still has some way to go to increase the bv-Control vulnerability database to a similar standard to the competition. The reports were accurate in the information they returned, however, and were very clear and easy to read. bv-Control also provides an auto-fix capability for some vulnerabilities, and that feature, combined with the simple reports and user interface, may appeal to the less security-literate administrator (although it would have to be compared carefully against NetRecon if such a consideration were paramount). Click here to return to Bindview bv-Control Review With CyberCop Scanner we created a new Template containing all the �default� vulnerability checks � this is a �safe� setting within CyberCop that selects every check except those which are likely to crash the machine being scanned. The following results were obtained:
Time taken to scan 3 machines: 2 minutes 55 seconds This report was clear, easy to read, and contained all the vulnerabilities we expected to find. In addition, CyberCop Scanner allows a high degree of �tweaking� of the scan configuration as well as a scripting language that can be used to create custom attacks, thus making it much more attractive to the security professional (though it is no more difficult to use than any of the other products tested). The most outstanding feature of this set of results is the time in which they were achieved � less than three minutes to scan all three machines. Click here to return to NAI CyberCop Scanner Review For the SecureScan NX test we selected the default policy named �Safe Scan�. This includes all Test Cases (vulnerability checks) except those that will cause Denial of Service on the machine being scanned. The following results were obtained:
Time taken to scan 3 machines: 11 minutes 33 seconds There has been a significant amount of work done on this product since we last evaluated it a year ago, and increases in performance and number of test cases are both apparent. Performance is even better when port scans are omitted, turning in a time of 5 minutes 10 seconds when running our test suite � less than half the time takes when port scans are included. Although not the fastest product on test (CyberCop Scanner would be hard to beat) the recent performance enhancements see it push all the way up into second place, and it returns all the vulnerabilities we expected to find. The reports are clear and easy to read, and SecureScan NX also provides the most complete on-screen monitoring and analysis capability of the products tested here. In addition, it is the only product we tested that is capable of using remote scanning engines to perform scans behind firewalls in a distributed environment, as well as determine the exact firewall filter rules in effect between the scanner console and remote firewall probe. That, together with the high degree of flexibility in configuring test parameters, should make SecureScan NX of great interest to most security professionals. The fact that the more esoteric scanning parameters are well hidden behind a Wizard-type interface, will also make SecureScan NX attractive to the less security-literate administrator. Click here to return to VIGILANTe Review Summary � Performance Testing The range of results returned in the various reports make it impossible for us to reproduce a straight comparison of the products tested in a document such as this. We felt that of all the products tested, only bv-Control under performed, finding just 33 vulnerabilities across all three machines. It could be argued that Symantec�s NetRecon �over performed� on the �Heavy� scan setting, returning far too much information to be really useful. However, in selecting the �Medium� scan, we were presented with a far more usable report that more closely matched that of CyberCop Scanner and VIGILANTE�s SecureScan NX. We also particularly liked the Progressive Scan feature that attempted to use information from one exploit to perpetrate another. In extreme cases, this could actually map out a potential route from low-level access on one machine to administrator access on another, and this could obviously prove very useful. CyberCop Scanner and SecureScan NX produced the most usable and accessible results, identifying all the most important vulnerabilities that would allow external hackers to gain access to your systems (writeable FTP root directories, vulnerable Web and mail servers, etc.). However, neither of these were as good at auditing NT Servers as bv-Control or NetRecon. Where they had the edge was in their more advanced features that are simply not included in other VA products. CyberCop Scanner, for example, provides the built-in IDS testing capability and the CASL scripting language. The latter feature in particular would be of interest to many security professionals who would like to script their own attacks, and a number of CASL scripts aimed at exploiting firewall filtering rules are included, together with a remote �listening� component which can be installed inside a firewall. SecureScan NX takes this latter feature to its logical conclusion and provides a true multi-tiered architecture with remote scanning engines that can be deployed on multiple subnets, and behind firewalls, throughout a corporate network. Not only can SecureScan NX then run remote scans from any or all of those distributed engines, but the firewall probe component provides the ability to initiate a full analysis of the effectiveness (or otherwise) of the firewall rules in effect between the scanner and the probe. In our opinion, this makes SecureScan NX a very desirable product indeed. At the end of the day, there was a huge difference in the range, accuracy and content of the reports across all the products tested. Judgement of the effectiveness of these products is therefore largely subjective, and we would recommend that you evaluate carefully in your own environment to determine if the results that are being returned from scanning your own machines are useful. As with anti-virus products, there is often considerable overlap between VA products, but each one has specific strengths and weaknesses. To provide maximum coverage, most organisations would be well advised to consider the purchase of more than one VA product, or to supplement a commercial offering with one of the various �underground� tools that are freely available on the Internet (not one of the tools on test, for example, possessed a port scanning capability that could in any way be considered a match for nmap). One product tested as part of the VA evaluation which should be considered by every organisation with more than a handful of desktops to administer is Symantec�s Enterprise Security Manager. This is a fairly unique product amongst those tested in that it focuses more on cross-platform policy auditing and enforcement, and would thus make an ideal companion product to any of the VA scanners. Click here to return to the VA Index Section |
Security Testing |
Send mail to webmaster
with questions or
|