 |
|
ISS Proventia A201 Test Results
|
Test 1.1 � Attack Recognition |
Attacks |
Default ARR |
Custom ARR |
|
Test 1.1.1 - Backdoors |
5 |
5 |
5 |
|
Test 1.1.2 - DNS |
2 |
2 |
2 |
|
Test 1.1.3 - DOS |
11 |
9 |
11 |
|
Test 1.1.4 - False negatives (modified exploits) |
7 |
7 |
7 |
|
Test 1.1.5 - Finger |
4 |
4 |
4 |
|
Test 1.1.6 - FTP |
4 |
3 |
4 |
|
Test 1.1.7 - HTTP |
35 |
29 |
35 |
|
Test 1.1.8 - ICMP |
2 |
2 |
2 |
|
Test 1.1.9 - Reconnaissance |
10 |
10 |
10 |
|
Test 1.1.10 - RPC |
2 |
1 |
2 |
|
Total |
82 |
72 / 82 |
82 / 82 |
�
|
Test 1.2 � Resistance to False Positives |
Pass/Fail |
|
Test 1.2.1 - Audiogalaxy FTP traffic |
PASS |
|
Test 1.2.2 - Normal directory traversal (below Web root) |
PASS |
|
Test 1.2.3 - MDAC heap overflow using GET instead of POST |
PASS |
|
Test 1.2.4 - Retrieval of Web page containing �suspicious� URLs |
PASS |
|
Test 1.2.5 - MSTREAM communications using invalid commands |
PASS |
|
Test 1.2.6 - Normal NetBIOS copy of �suspicious� files |
PASS |
|
Test 1.2.7 - Normal NetBIOS traffic |
PASS |
|
Test 1.2.8 - POP3 e-mail containing �suspicious� URLs |
PASS |
|
Test 1.2.9 - POP3 e-mail with �suspicious� DLL attachment |
PASS |
|
Test 1.2.10 - POP3 e-mail with �suspicious� Web page attachment |
PASS |
|
Test 1.2.11 - SMTP e-mail transfer containing �suspicious� URLs |
PASS |
|
Test 1.2.12 - SMTP e-mail transfer with �suspicious� DLL attachment |
PASS |
|
Test 1.2.13 - SMTP e-mail transfer with �suspicious� Web page attachment |
PASS |
|
Test 1.2.14 - SNMP V3 packet with invalid request ID |
PASS |
|
Total Passed |
14 / 14 |
�
Section 2 - NIDS Performance Under Load
|
Test 2.1 � UDP traffic to random valid ports |
� 25Mbps |
� 50Mbps |
� 75Mbps |
� 100Mbps |
� Max |
|
Test 2.1.1 - 64 byte packet test - max 148,000pps |
100% |
100% |
100% |
99% |
95Mbps |
|
Test 2.1.2 - 440 byte packet test - max 26,000pps |
100% |
100% |
100% |
100% |
100Mbps |
|
Test 2.1.3 - 1514 byte packet test - max 8172pps |
100% |
100% |
100% |
100% |
100Mbps |
�
|
Test 2.2 � HTTP �maximum stress� traffic with no transaction delays |
� 25Mbps |
� 50Mbps |
� 75Mbps |
� 100Mbps |
� Max |
|
Test 2.2.1 - Max 250 connections per second - ave packet size 1200 bytes - max 10,000 packets per second |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
|
Test 2.2.2 - Max 500 connections per second - ave packet size 540 bytes - max 23,000 packets per second |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
|
Test 2.2.3 - Max 1000 connections per second - ave packet size 440 bytes - max 28,000 packets per second |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
|
Test 2.2.4 - Max 2000 connections per second - ave packet size 350 bytes - max 36,000 packets per second |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
�
|
Test 2.3 � HTTP �maximum stress� traffic with transaction delays |
� 25Mbps |
� 50Mbps |
� 75Mbps |
� 100Mbps |
� Max |
|
Test 2.3.1 - Max 500 connections per second - ave packet size 540 bytes - max 23,000 packets per second - 10 sec delay - max 5,000 open connections |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
|
Test 2.3.2 - Max 1000 connections per second - ave packet size 440 bytes - max 10,000 packets per second - 10 sec delay - max 5,000 open connections |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
��
|
Test 2.4 � Protocol mix |
250Mbps |
500Mbps |
750Mbps |
1Gbps |
Max |
|
Test 2.4.1 - 72% HTTP (540 byte packets) + 20% FTP + 4% UDP (256 byte packets). Max 38 connections per second - ave packet size 555 bytes - max 2,200 packets per second - max 14 open connections |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
�
|
� Test 2.5 � Real World traffic |
250Mbps |
500Mbps |
750Mbps |
1Gbps |
Max |
|
Test 2.5.1 - Pure HTTP (simulated browsing session on NSS Web site). Max 10 connections per second - 3 new users per second - ave packet size 1000 bytes - max 11,000 packets per second |
� 100% |
� 100% |
� 100% |
� 100% |
� 100Mbps |
�
Section 3 - Network IDS Evasion
|
Test 3.1 � Evasion Baselines |
Detected? |
| Test 3.1.1 - NSS Back Orifice ping |
YES |
| Test 3.1.2 - Back Orifice connection |
YES |
| Test 3.1.3 - FTP CWD root |
YES |
| Test 3.1.4 - Fragroute baseline (test-cgi probe using HEAD) |
YES |
| Test 3.1.5 - ISAPI printer overflow |
YES |
| Test 3.1.6 - Showmount export lists |
YES |
| Test 3.1.7 - Test CGI probe (/cgi-bin/test-cgi) |
YES |
| Test 3.1.8 - PHF remote command execution |
YES |
| Test 3.1.9 - Whisker baseline (test-cgi probe using HEAD) |
YES |
|
Total |
9 / 9 |
�
|
Test 3.2 � Packet Fragmentation/Stream Segmentation |
Detected? |
Decoded? |
|
Test 3.2.1 - IP fragmentation - ordered 8 byte fragments � |
YES |
YES |
|
Test 3.2.2 - IP fragmentation - ordered 24 byte fragments � |
YES |
YES |
|
Test 3.2.3 - IP fragmentation - out of order 8 byte fragments � |
YES |
YES |
|
Test 3.2.4 - IP fragmentation - ordered 8 byte fragments, duplicate last packet � |
YES |
YES |
|
Test 3.2.5 - IP fragmentation - out of order 8 byte fragments, duplicate last packet |
YES |
YES |
|
Test 3.2.6 - IP fragmentation - ordered 8 byte fragments, reorder fragments in reverse |
YES |
YES |
|
Test 3.2.7 - IP fragmentation - ordered 16 byte fragments, fragment overlap (favour new) |
YES |
YES |
|
Test 3.2.8 - IP fragmentation - ordered 16 byte fragments, fragment overlap (favour old) |
YES |
YES |
|
Test 3.2.9 - TCP segmentation - ordered 1 byte segments, interleaved duplicate segments with invalid TCP checksums |
YES |
YES |
|
Test 3.2.10 - TCP segmentation - ordered 1 byte segments, interleaved duplicate segments with null TCP control flags |
YES |
YES |
|
Test 3.2.11 - TCP segmentation - ordered 1 byte segments, interleaved duplicate segments with requests to resync sequence numbers mid-stream |
YES |
YES |
|
Test 3.2.12 - TCP segmentation - ordered 1 byte segments, duplicate last packet |
YES |
YES |
|
Test 3.2.13 - TCP segmentation - ordered 2 byte segments, segment overlap (favour new) |
YES |
YES |
|
Test 3.2.14 - TCP segmentation - ordered 1 byte segments, interleaved duplicate segments with out-of-window sequence numbers |
YES |
YES |
|
Test 3.2.15 - TCP segmentation - out of order 1 byte segments � |
YES |
YES |
|
Test 3.2.16 - TCP segmentation - out of order 1 byte segments, interleaved duplicate segments with faked retransmits |
YES |
YES |
|
Test 3.2.17 - TCP segmentation - ordered 1 byte segments, segment overlap (favour new) |
YES |
YES |
|
Test 3.2.18 - TCP segmentation - out of order 1 byte segments, PAWS elimination (interleaved dup segments with older TCP timestamp options) |
YES |
YES |
|
Test 3.2.19 - IP fragmentation - out of order 8 byte fragments, interleaved duplicate packets scheduled for later delivery |
YES |
YES |
|
Total |
19 / 19 |
19 / 19 |
��
|
� Test 3.3 � URL Obfuscation |
Detected? |
Decoded? |
|
Test 3.3.1 - URL encoding |
YES |
YES |
|
Test 3.3.2 - /./ directory insertion |
YES |
YES |
|
Test 3.3.3 - Premature URL ending |
YES |
YES |
|
Test 3.3.4 - Long URL |
YES |
YES |
|
Test 3.3.5 - Fake parameter |
YES |
YES |
|
Test 3.3.6 - TAB separation |
YES |
YES |
|
Test 3.3.7 - Case sensitivity |
YES |
YES |
|
Test 3.3.8 - Windows \ delimiter |
YES |
YES |
|
Test 3.3.9 - Session splicing |
YES |
YES |
|
Total |
9 / 9 |
9 / 9 |
�
|
Test 3.4 � Miscellaneous Obfuscation Techniques |
Detected? |
Decoded? |
|
Test 3.4.1 - Altering default ports |
YES |
YES |
|
Test 3.4.2 - Inserting spaces in FTP command lines |
YES |
YES |
|
Test 3.4.3 - Inserting non-text Telnet opcodes in FTP data stream |
YES |
YES |
|
Test 3.4.4 - Altering protocol and RPC PROC numbers |
YES |
YES |
|
Test 3.4.5 - RPC record fragging |
YES |
YES |
|
Test 3.4.6 - Polymorphic mutation (ADMmutate) |
YES |
YES |
|
Total |
6 / 6 |
6 / 6 |
�
Section 4 - Stateful Operation Test
|
Test 4.1 � Attack Replay |
Alerts? |
DOS? |
Notes |
|
Test 4.1.1 - Snot traffic |
YES |
NO |
125 alerts raised (considered reasonable). |
|
Test 4.2.2 - Stick Traffic |
YES |
NO |
170 alerts raised (considered reasonable) |
�
|
Test 4.2 � Simultaneous Open Connections (default settings) |
|||||||
|
Number of open connections |
10,000 |
25,000 |
50,000 |
100,000 |
250,000 |
500,000 |
1,000,000 |
|
Test 4.2.1 - Attack Detection |
PASS |
PASS |
PASS |
PASS |
PASS |
FAIL1 |
FAIL1 |
|
Test 4.2.2 - State Preservation |
PASS |
PASS |
PASS |
PASS |
PASS |
FAIL1 |
FAIL1 |
�
|
Test 4.3 � Simultaneous Open Connections (after tuning) |
|||||||
|
Number of open connections |
10,000 |
25,000 |
50,000 |
100,000 |
250,000 |
500,000 |
1,000,000 |
|
Test 4.3.1 - Attack Detection |
PASS |
PASS |
PASS |
PASS |
PASS |
FAIL2 |
FAIL2 |
|
Test 4.3.2 - State Preservation |
PASS |
PASS |
PASS |
PASS |
PASS |
FAIL2 |
FAIL2 |
�
|
Notes:
We installed one Proventia A201 appliance reporting to a single SiteProtector server. We used the Evaluation policy, which has all attack signatures enabled (apart from port probes) and some key audit signatures.� Centralised management and alerting capabilities are excellent when using the SiteProtector Console. Both sensor management and policy deployment are well catered for - one of the best we have seen - and although the move to a three-tiered architecture has introduced occasional slight delays in the �real time� alerting, it has made the product much more robust and scalable in larger deployments.� Analysis and reporting has been improved by several orders of magnitude with this release, with an intuitive interface providing a quick and easy means of drilling down into the underlying data. Any number of custom views can be saved and printed or exported. The use of complete evidence logs provide the means for more detailed forensic analysis if required. When using the optional Fusion module, event escalation and correlation capabilities are introduced providing enormous assistance to the administrator who has to deal with large volumes of data.� Attack recognition was excellent using the Evaluation security policy, turning in one of the better recognition rates of the products tested out of the box. ISS were also one of the quickest to respond to the missing signatures, turning out a new signature pack which was made available to customers in less than 24 hours, and which then provided a 100 per cent recognition rate. Resistance to false positives also appeared to be very good, and we noted a significant reduction in �noise� in this release, with fewer test cases raising several alerts for a single exploit. All our �false negative� (modified exploit) cases were detected correctly, demonstrating that the RealSecure signatures are designed to detect the underlying vulnerability rather than a specific exploit. � Resistance to known evasion techniques was excellent, with Proventia being one of the few products to collect a clean sheet across the board in our IDS evasion tests. Fragroute, Whisker, ADMmutate and even RPC record fragging all failed to trick Proventia into ignoring valid attacks.� Stateful operation was also very good, with Proventia tracking and maintaining state on 250,000 connections with the default settings. Unfortunately, it was not possible to tune the appliance to go beyond this due to the limit of 1GB installed RAM. Note that Proventia is designed to ignore new connections once the limit is exceeded, but does not age out old ones. We actually prefer old connections to be aged out, since the dropping of new connections does provide an easier method of evasion for the attacker.� Resistance to Stick and Snot was acceptable, raising a fairly small number of alerts but never succumbing to a DoS condition and with no flooding at the Console. � In terms of detection rates, Proventia sensor turned in some excellent results, demonstrating that it could handle almost all of our test traffic loads up to 100Mbps. In our more extreme tests - our wire speed 64 byte packet test, for example - the performance did just begin to tail off at wire speed, but even at this extreme load the device managed an excellent 95Mbps. � The Proventia sensor performed impeccably in all our �real world� tests, however, and so for all �normal� background traffic loads we feel quite happy to rate it at a minimum of 100Mbps (note that ISS actually claims 200Mbps for this device). Click here to return to the ISS Proventia
A201 Review |
Send mail to webmaster
with questions or�
|