NSS Group logo

Intrusion Prevention Systems 

Group Test (Edition 2)

This report has been superseded by IPS Group Test Edition 3 and is no longer available on line.  However, it does include reviews of five IPS products, none of which are included in the current Edition, and is currently the only report in the IPS series which specifically tests both content-based AND rate-based products.  It is available for purchase in PDF, CD or print versions.

Click here to purchase on line using our secure server.

Foreword

Following the huge success the first comprehensive Intrusion Prevention System (IPS) test of its kind, The NSS Group is pleased to present the results of its second IPS Group Test which includes a number of new products not included in the first report. 

As with Edition 1, this exhaustive review will give readers a complete perspective of the capabilities, maturity and suitability for immediate deployment of each of the products tested. The NSS Group established this test as IPS products are being actively deployed as a new layer in defence-in-depth security architectures.

It is interesting to note that between publishing Edition 1 and Edition 2 the analyst groups who were previously so sure that IDS was dead and IPS stillborn have now come around to our way of thinking - while the so-called “deep inspection firewalls” are not ready for prime-time deployments, security administrators need to make the best use of the technology that is available, and for now that means a combination of firewalls, in-line intrusion prevention devices, and intrusion detection systems. They are likely to be in use for quite some time to come, too! 

The NSS IPS Group Test evaluates the performance, reliability, security effectiveness, and usability of Network IPS products. The test consists of seven sections within three primary areas: performance and reliability, security accuracy, and usability.  

Overall, the brand new test suite contains over 800 individual tests, many of which are run multiple times, to provide the most thorough and complete evaluation of IPS products available anywhere today. This edition also sees the introduction of a new Rate-Based IPS methodology to complement our exiting Content-Based IPS methodology used in Edition 1. This has allowed us to more accurately test Rate-Based/Attack Mitigation products, and two devices were tested against this new methodology in the latest report (one of them actually tested against both methodologies - a first). 

It is worth pointing out that not every product submitted for testing receives an NSS Approved award. Standards are very high, and out of nine products signed up for this group test initially, only the five included in the final Edition 2 report have received NSS Approved awards. 

We believe that our IPS test methodologies - which have been updated for this test - will become the de facto standard for testing in-line Intrusion Prevention/Attack Mitigation devices, and the NSS Approved logo an essential item on the list of requirements when purchasing these products.  

We also believe that this report is essential reading for anyone considering deploying Intrusion Prevention Systems in their networks, either in a test or live situation, and we hope that you find it both informative and useful in making your purchasing decisions. The IPS Group Test (Edition 2) report can be viewed on-line at www.nss.co.uk/ips

Bob Walder

Table of Contents

Introduction
Intrusion Prevention Systems (IPS)
Host IPS (HIPS)
Network IPS (NIPS)
Rate-Based IPS (Attack Mitigator)
Implementation Challenges
Requirements for effective prevention
The NSS Intrusion Prevention Group Test
Performance
Security Effectiveness
Usability

Summary
The Market
The Products

Content-Based IPS Product Reviews

BroadWeb NetKeeper NK-3256T V3.6.0
Executive Summary
Architecture
BEMS Management Server
BEMS Management Client
NetKeeper Appliance
Performance
Security Effectiveness
Usability
Installation
Configuration
Policy Management
Alert Handling
Reporting and Analysis
Verdict

Contact Details

Fortinet FortiGate-800
Executive Summary
Architecture
FortiGate-800 Appliance
Web Manager 
FortiManager

Logging & Reporting
Performance
Security Effectiveness
Usability
Installation
Configuration
Policy Management
Alert Handling
Reporting and Analysis
Verdict
Contact Details

SecureSoft Absolute IPS NP5G V1.1
Executive Summary
Architecture
Absolute IPS NP5G
Absolute IPS Manager
Performance
Security Effectiveness
Usability
Installation
Configuration
Policy Management
Alert Handling
Reporting and Analysis
Verdict
Contact Details

Top Layer IPS 5500 V3.3
Executive Summary
Architecture
Management Application
Central Management System (CMS)
IPS 5500 Appliance
High Availability
Performance
Content-Based
Rate-Based (Attack Mitigation)
Security Effectiveness
Content-Based
Rate-Based (Attack Mitigation)
Usability
Installation
Configuration
Policy Management
Alert Handling
Reporting and Analysis
Verdict
Contact Details

Content Based Testing Methodology
The Test Environment
Section 1 - Detection Engine
Section 2 - Evasion
Section 3 - Stateful Operation
Section 4 - Detection/Blocking Performance Under Load
Section 5 - Latency & User Response Times
Section 6 - Stability & Reliability
Section 7 - Management and Configuration

Content Based Test Results

Rate-Based Product Reviews

V-Secure V-100 V7.0
Executive Summary
Architecture
V-Secure Management Studio
V-Secure IPS Appliance
Security Modules
Performance
Security Effectiveness
Usability
Installation
Configuration
Policy Management
Alert Handling
Reporting and Analysis
Verdict
Contact Details

Rate-Based Testing Methodology
The Test Environment
Section 1 - Detection Engine
Section 2 - Evasion
Section 3 - Attack Mitigation Performance Under Load
Section 4 - Latency & User Response Times
Section 5 - Stability & Reliability
Section 6 - Management and Configuration

Rate-Based Test Results

Appendix A - Vendor Questionnaires

Appendix B - The Test Equipment
Spirent Communications SmartBits SMB-6000/sMB-600
SmartBits Applications

Spirent Communications Avalanche and Reflector
Adtech-AX/4000
Cisco Catalyst 6500 Series Switches
Blade Software Informer Suite

Open Source Replay Tools

Top         Home

Certification Programs

Group Test Reports

White Papers

On-Line Store

Contact The NSS Group

Home

Click here to purchase electronic (PDF) and hard copy versions of this report

Send mail to webmaster with questions or 
comments about this web site.

Copyright � 1991-2006 The NSS Group Ltd.
All rights reserved.