 |
|
|
In addition to thoroughly evaluating each product in a controlled environment that was as close to a real-life network as we could make it, we also performed a number of tests against the VA scanner products. We evaluated each VA product carefully paying particular attention to a range of key features. The feature set we evaluated against also represents the sort of questions you should be asking your VA vendor:
We put these questions directly to each of the vendors, and the replies are reproduced unedited in Appendix A. Our standard test bed for this project consisted of Pentium III 700-1000MHz PCs each with 256MB RAM running Windows 2000 SP1, Windows NT Server 4.0 SP6a, or Red Hat Linux 6.2 (depending on the requirements of the product under test). All installations were on �clean� machines, restored between tests from a standard Symantec Ghost image. The network was 100Mbit Ethernet with CAT 5 cabling, Intel NetStructure 480T Routing Switches, and Intel auto-sensing 10/100 network cards installed in each host. In all cases, the Intel drivers that were provided with the network cards were used during the test. A range of scanning hosts were installed across multiple subnets behind a router and an open firewall in order to test deployment, management and scanning features. Three test hosts were installed �out of the box� including as many of the �default� services bundled with the product as possible (Web server, mail server, FTP server, DNS server, etc.):
We then performed scans from the scanning hosts using the default settings and the �heaviest� scan mode available, recording the time taken to complete the scan and the number of vulnerabilities discovered. This was an extremely difficult test to evaluate, since we found we were not really comparing �apples with apples� in most cases. Some VA products are designed as �active� � or network-oriented - scanners, behaving like a �hacker in a box� in trying to discover and exploit as many vulnerabilities as possible. Others were more passive � or host-oriented - concerned mainly with highlighting incorrect or weak configurations in the host operating systems. Symantec Enterprise Security Manager BindView bv-Control for Internet Security V3
The range of results returned in the various reports make it impossible for us to reproduce a straight comparison of the products tested in a document such as this. We felt that of all the products tested, only bv-Control under performed, finding just 33 vulnerabilities across all three machines. It could be argued that Symantec�s NetRecon �over performed� on the �Heavy� scan setting, returning far too much information to be really useful. However, in selecting the �Medium� scan, we were presented with a far more usable report that more closely matched that of CyberCop Scanner and VIGILANTE�s SecureScan NX. We also particularly liked the Progressive Scan feature that attempted to use information from one exploit to perpetrate another. In extreme cases, this could actually map out a potential route from low-level access on one machine to administrator access on another, and this could obviously prove very useful. CyberCop Scanner and SecureScan NX produced the most usable and accessible results, identifying all the most important vulnerabilities that would allow external hackers to gain access to your systems (writeable FTP root directories, vulnerable Web and mail servers, etc.). However, neither of these were as good at auditing NT Servers as bv-Control or NetRecon. Where they had the edge was in their more advanced features that are simply not included in other VA products. CyberCop Scanner, for example, provides the built-in IDS testing capability and the CASL scripting language. The latter feature in particular would be of interest to many security professionals who would like to script their own attacks, and a number of CASL scripts aimed at exploiting firewall filtering rules are included, together with a remote �listening� component which can be installed inside a firewall. SecureScan NX takes this latter feature to its logical conclusion and provides a true multi-tiered architecture with remote scanning engines that can be deployed on multiple subnets, and behind firewalls, throughout a corporate network. Not only can SecureScan NX then run remote scans from any or all of those distributed engines, but the firewall probe component provides the ability to initiate a full analysis of the effectiveness (or otherwise) of the firewall rules in effect between the scanner and the probe. In our opinion, this makes SecureScan NX a very desirable product indeed. At the end of the day, there was a huge difference in the range, accuracy and content of the reports across all the products tested. Judgement of the effectiveness of these products is therefore largely subjective, and we would recommend that you evaluate carefully in your own environment to determine if the results that are being returned from scanning your own machines are useful. As with anti-virus products, there is often considerable overlap between VA products, but each one has specific strengths and weaknesses. To provide maximum coverage, most organisations would be well advised to consider the purchase of more than one VA product, or to supplement a commercial offering with one of the various �underground� tools that are freely available on the Internet (not one of the tools on test, for example, possessed a port scanning capability that could in any way be considered a match for nmap). One product tested as part of the VA evaluation which should be considered by every organisation with more than a handful of desktops to administer is Symantec�s Enterprise Security Manager. This is a fairly unique product amongst those tested in that it focuses more on cross-platform policy auditing and enforcement, and would thus make an ideal companion product to any of the VA scanners. Click here to return to the VA Index Section |
Security Testing |
Send mail to webmaster
with questions or
|