 |
|
|
Summary Testing IDS products is never easy, and as network speeds increase, topologies become more complex, and the IDS products themselves try to differentiate themselves with ever more complicated “unique” features and architectural differences, the job of the independent tester can only become more difficult. Our job is to attempt to simulate as close to a real world environment as possible in our labs, whilst keeping the tests completely repeatable. Our most recent methodology, which has been completely revised for this latest Gigabit IDS test, tries to bring elements of real world use to the lab, whilst stressing as many different features of each IDS as is feasible. Testing reliably at Gigabit speeds and beyond continues to present us with new challenges, and pushes our testing equipment to the limits - for Version 3 of our testing methodology we actually had to use two pairs of Caw Web Avalanche/Reflector boxes in order to produce the levels of background traffic we required. At the end of the day, the environment we created was one of the toughest ever likely to be faced by the average IDS product, whilst remaining fair in that certain tests represented an extremely busy “normal” network in terms of background traffic. Pushing the products under test to their limits in a heavily-utilised Gigabit network certainly produced some interesting results, and posed problems for some vendors. A total of five vendors signed up for Edition 3, but three of them failed our stringent tests leaving the two you see in this report. The two remaining products, however, are amongst the most interesting on the market at the moment. Both ISS and Sourcefire have their roots firmly in the IDS space, and have recently moved into the in-line IPS market too. Whereas ISS can claim a lengthy pedigree as a long-established company, Sourcefire may well be considered by many as the “new kid on the block”. However, it should be remembered that the base Snort product has been under development for a lot longer than the relatively brief history of Sourcefire, and this has allowed the company to bring a mature and robust product to market relatively quickly. Both products turned in good performances in terms of both signature recognition and detection rates - in ISS’ case in particular, the signature recognition and false positive rates were outstanding. In our real-world protocol mix tests, both products exhibited 100 per cent detection rates, meaning that on any “normal” network, both would provide adequate performance right up to the rated throughput (600Mbps for the Proventia A604 and 1Gbps for the Sourcefire IS3000). Note, however, that at these levels, performance can be affected significantly by changes in the make-up of the traffic being monitored. If your network has a different average packet size to ours, or a different average HTTP response size, or different connection rates, or is predominantly FTP traffic, then your mileage may vary. Our 1 million open connection tests were handled with ease by both products out of the box, although the Proventia is the only one of the two which allows the administrator to configure the default action when resources are low (age out old connections or refuse new ones). Resistance to evasion techniques was also strong, with both products turning in blemish-free performances in that area. One of the areas that continues to improve is the management and reporting capability. Both of these vendors have made significant improvements to their management systems in recent years, although it should be noted that ISS includes a centralised three-tier management system as standard whereas this is an extra-cost option for Sourcefire 3D. However, at the time of testing, it was not possible to manage the Proventia appliance directly without a centralised management system (this will change in future releases) whereas the Sourcefire appliances do provide Web-based direct management capability, eliminating the requirement for a cumbersome three-tier management system in smaller installations. Multi sensor management and adequate alert handling and forensic reporting become even more critical in a high speed switched network environment. The ability to fine-tune individual signatures or entire policies and then deploy them to multiple sensors at the click of a button is essential. As is the ability to consolidate alerts from multiple sensors, perform extensive drill-down capabilities, detailed alert analysis, and the ability to annotate and group alerts into single incidents for investigation. Both ISS and Sourcefire have developed management systems which are more than capable of handling these requirements. Correlation and more intelligent event handling have been at the forefront of development efforts by those vendors who are “switched on” to the increasing burden of security event analysis in today’s high-speed networks. The more they can automate the decision of which alerts can be reliably grouped together into a single incident, or which events have the highest priority or pose the greatest immediate threat, the more they will become a real help to the administrator. At the moment, Sourcefire and ISS are leading the field in this area. ISS provides advanced correlation features in the optional SecurityFusion module, coupled with the ability to combine active scan information from Internet Scanner to determine which alerts are most likely to pose a real threat. Sourcefire provides something very similar via its RNA product, but using the far less-intrusive passive scanning method. In this case, the RNA Sensor is a passive sensor which sniffs traffic from the wire in the same way as the Intrusion Sensor, monitoring traffic flows between hosts and very quickly detecting when new hosts are added to the network. Having monitored flows between hosts for some time the system quickly builds up a picture of which operating systems and services are running on these hosts, and this is used to determine the likely vulnerabilities on the end-points. As with SecurityFusion, this data is then used to determine which of the intrusion events are likely to have the biggest impact on a given host, thus providing the administrator with a much more accurate, focussed and prioritised list of events to address. Both of these - SecurityFusion and RNA - are extra-cost options, but both could be considered essential purchases in most networks due to the accuracy with which they can identify the most threatening security events. Accurate price comparisons between the two products are difficult, due mainly to the differences in how each company factors in the cost of management software and hardware. As we have already mentioned, centralised management (software and hardware) is an additional cost option for the Sourcefire product, whereas the software (but not hardware) is included with the ISS product. However, a centralised management system is not strictly necessary for the Sourcefire product, which has a Web-based direct management capability built in. At $24,995 for a 1Gbps-capable IS3000 against $29,495 for a 600Mbps-capable Proventia A604 (the G1000 is around $40,000) the cost difference in base hardware pretty much levels out the differences in management costs in most cases (maintenance costs will also end up being fairly similar between the two products). Both RNA and SecurityFusion are extra cost options, but whereas RNA is a one-off cost, SecurityFusion is licensed per-IP address protected. It would be wise to carry out your own cost comparison based on your exact requirements before purchasing. Our feeling is that you will be buying on functionality and confidence in the supplier, and that overall the price differences are unlikely to sway you too much one way or the other. Click here to return to the Gigabit IDS Index Section |
Send mail to webmaster
with questions or
|