 |
|
|
Intoto delivers a software-based IPS product, IntruPro, licensed directly to OEMs. This allows them quickly to integrate IPS technology in their own hardware platform in order to build UTM devices, security appliances or security switches. The architecture is agnostic to hardware platforms, operating systems and hardware accelerators. Intoto submitted a reference hardware platform for the testing. Due to the fact that Intoto provides the IntruPro software platform to be integrated with the OEM’s hardware, OS and any other existing software components, it is very important for the OEM to tune and optimise their product appropriately. The efficacy of the product therefore depends to a certain extent on the skill of the integrator, and it should be understood that NSS Approval for IntruPro as a software product does not automatically confer NSS Approval on those devices using it. Potential customers should therefore ensure that any vendor offering an appliance based on the IntruPro product has obtained specific NSS Approval for that device before purchasing. The product is available in both layer 3 (routed) and layer 2 (transparent bridge) versions, making it suitable for both perimeter and core network deployments. On the reference hardware provided for this test (dual Xeon processor, 2GB RAM, and dual copper Gigabit ports) IntruPro provided itself capable of handling 100Mbps of traffic under normal network conditions. Blocking rates were good out of the box, improving to 95 per cent following an update, and resistance to common IDS/IPS evasion techniques was excellent. We did, however, notice a tendency for the device to raise multiple alerts for a single exploit, and false positives could be a concern on some networks. Throughput and latency were good under all network loads and across all packet sizes up to the rated 100Mbps. We also found IntruPro to be very stable and reliable under extended attack, with effective SYN flood mitigation capabilities. Management and configuration of single IntruPro devices is straightforward via the three-tier management architecture provided. A lack of any centralised policy management, however, means that the current solution will not scale well, forcing the administrator to attach to each Sensor individually in order to configure it. Alert handling and reporting are adequate, offering a range of filtering options on the log entries, and the ability to create custom alerts. Unfortunately, having created customised report filters and layouts, it is not possible to save these for re-use. In general, we would conclude that configuration and alert handling are acceptable for a single device or low number of devices, but not currently suitable for larger-scale deployments. Intoto also allows OEM vendors to integrate with their existing management software or a third party CMS tool. Intoto offers a three-tier architecture for sensor management - there is no direct Sensor management capability in the current version. Although this clearly provides reliability and scalability, it does mean that a separate, dedicated host is always required for the management server component, possibly making it a less attractive solution for only one or two sensors. Intoto also allows the OEM vendors to integrate their existing management software or web based interface using APIs provided by IntruPro sensor. The main components of the IntruPro system are:
The first three components are collectively referred to as the IntruPro Manager. IntruPro internally uses some components of its layer 3 firewall for layer 3 and 4 attack detection and prevention. The product submitted for testing was tested for routed gateway mode, making it most suitable for perimeter deployments.
A layer 2 transparent bridge version is also available along with the layer 2 Firewall offered by Intoto. Intoto has a history of application-layer firewall development, and therefore it is not surprising to see a wide array of protocol decoders (called Application Engines) in the IPS product This allows the IntruPro Sensor to make use of a range of detection techniques including:
The aim of this section is to verify that the sensor is capable of detecting and blocking exploits when subjected to increasing loads of background traffic up to the maximum bandwidth supported as claimed by the vendor. For each type of background traffic, we also determine the maximum load the IPS can sustain before it begins to drop packets/miss alerts. It is worth noting that devices which demonstrate 100 per cent blocking but less than 100 per cent detection in these tests will be prone to blocking legitimate traffic under similar loads. Note that Intoto IntruPro is a software product and was tested on reference hardware (dual Xeon processor, 2GB RAM, copper Gigabit ports) as a 100Mbps IPS device. Performance at almost all levels of our load tests was impeccable, with 100 per cent of all attacks being detected and blocked under all load conditions except at the 2000 connections per second level (maximum connection rate was around 1400cps - see Test Results section for full details) We would happily confirm Intoto’s 100Mbps rating for this device under normal network conditions, however. The basic latency figures of IntruPro on the reference hardware were good for a 100Mbps device across the board under all traffic loads. They ranged from 157�s with 25Mbps of 256 byte packets, to 222�s with 100Mbps of 1000 byte packets. Behaviour throughout the tests with no background traffic was reasonably consistent and predictable, with small decreases as additional network load was applied from 25Mbps to 100Mbps. Placing the device under a half load of 50Mbps of HTTP traffic increased latency somewhat, rising from 157�s to 220�s with 256 byte packets, from 214�s to 243�s with 550 byte packets, and from 254�s to 289�s with 1000 byte packets. HTTP response times were also very good, demonstrating an average of 203ms with 50Mbps of HTTP traffic. Latency when under 10Mbps (14800 packets per second) of SYN flood traffic was excessive due to packet loss. Despite this, the SYN flood mitigation technique (SYN proxy) used by Intoto was very effective, with very little of the DOS traffic making its way through to the target server. Although average HTTP response time for legitimate traffic increased to 623ms during the attack, there were very few failed transactions. IntruPro performed consistently and completely reliably throughout our tests. Under eight hours of extended attack (comprising millions of exploits mixed with genuine traffic) it continued to block 100 per cent of attack traffic, whilst passing 100 per cent of legitimate traffic. Exposing the sensor interface to ISIC-generated traffic had no adverse effect on the device at any time, and relevant protocol anomaly alerts were raised. All other malicious traffic continued to be blocked successfully during the attack, and there were no residual stability problems once the ISIC attack had been terminated. Please refer to the Testing Methodology section for full details of the methodology used and complete performance results. We installed one sensor with the latest signature library, and enabled all signatures, with traffic monitoring in both directions. Out of the box, blocking performance was good at 80 per cent, and was improved to 95 per cent following the application of a signature update after 48 hours. Detection/recognition rate was slightly lower (89 per cent following update) due to the fact that many DOS and ICMP attacks are blocked without alerting by the firewall module. We also noted that many of our test cases raised several alerts for a single exploit. This can be annoying, and we would prefer that the product performed simple correlation to reduce the number of superfluous alerts raised. It can, however, be mitigated by the fact that the source aggregation feature of the management interface results in a single entry on the GUI which can then be expanded to show the individual alerts if required. Performance in our “false negative” tests was good out of the box, and improved following the signature update, leaving just a single miss out of 14. A major concern in deploying an IPS is the blocking of legitimate traffic. The device initially failed five test cases in our false positive test suite, and although all of these were eliminated following the signature update, we also noted several false positives during our FTP testing, the result of some sloppy port-based backdoor signatures. Overall, false positives would be a concern for us with this device at present, and we would encourage potential purchasers to test in their own network before buying. Resistance to known evasion techniques was excellent, with IntruPro achieving a clean sweep across the board in almost all of our evasion tests. Fragroute and Whisker both failed to deceive the device into ignoring valid attacks, and all but one of the attempts were decoded accurately. Some minor configuration was required to enforce blocking for two of the test cases, but this should become the default configuration in future releases. Of the miscellaneous evasion techniques, only changing ports on backdoors caused problems (a not untypical situation, since it is expensive on resources to monitor for all backdoors on all ports), but the rest (including extensive RPC record fragging) were handled well. Out of the box, Intoto claims that the IntruPro on 100Mbps reference hardware can handle up to 50,000 open connections. This was not quite confirmed in testing, the device reaching its limits at around 45,000 open connections. We consider this to be just adequate for a 100Mbps device. During testing, we verified IntruPro’s ability to handle up to 45,000 simultaneously open connections whilst successfully maintaining state on our half-open exploits. The default operation of the device is to reject new connections when the state tables are full or resources are low, and this means that once the connection limit is exceeded, the device continues to maintain state on existing connections (thus detecting our half-open exploits), but inevitably, as a consequence, begins to block legitimate traffic. This behaviour is not configurable at run time. Stateless “exploits” are blocked by default, with no alerts. This behaviour is not configurable at run time, and there is no grace period following device start-up during which existing connections (which will appear as mid-flows initially) will be passed successfully. Please refer to the Testing Methodology section for full details of the methodology used and complete performance results. This part of the test procedure consists of a subjective evaluation of the features and capabilities of the product, and covers installation, configuration, policy editing, alert handling, and reporting and analysis. The IntruPro Sensor would normally be supplied as a turnkey appliance with a hardened and tuned operating system running the Sensor software. It is important that the vendor performs this task carefully, since it can affect both security and performance of the finished appliance. During testing, we noted several OS-specific and firewall-specific tuning parameters required amending in order to complete several of our tests. Intoto is now aware of these and will hopefully provide guidance for vendors offering their software in appliance form. All that is generally required when installing the Sensor is to configure the management interface and IP address, and specify the address of the management server. All of this is performed via the extensive text-based Command Line Interface (CLI) directly on the Sensor console. This is also the place to perform any additional performance tuning which may be required. Installation of the management server software is on a dedicated Windows (NT, 2000, XP, 2003) box, and requires that both the latest Java Run-time Environment (JRE) and database (MySQL or PostGreSQL) software is installed first. Once installed, a simple set-up Wizard guides the administrator through the initial configuration tasks, such as specifying database parameters, configuring the Log Listener daemon, creating Sensors, creating administrator accounts, and performing the first signature update. Documentation is good, with a Log Listener Administration Guide and IntruPro Manager Administration Guide provided in both electronic (PDF) and hard copy versions. All day-to-day configuration is performed via the IntruPro Console. The main screen is divided into four parts: the Main Toolbar, Top Toolbar, Side Toolbar, and Central Area (which acts as the main viewing and data-entry area). The operation of the Console follows a Web browser paradigm where there is a Back button which can be used to step backwards through screens previously accessed - this can make it quick to navigate when drilling down several levels.
The application options are grouped into tabbed entries to the main toolbar, and as each tab is selected, sub-options are displayed in the side toolbar. The main options available are:
All configuration tasks apply to the currently selected Sensor only, which is chosen from a drop-down menu of available Sensors at the top of the screen. In the current release, it is not possible to manage more than one Sensor at a time. Multiple Sensors can, however, be managed individually from a single Console, and new Sensors can be added via the Administrator tab simply by specifying the IP address of the management interface and a pre-shared secret if encryption is used. The Administrator tab also provides the means to configure database access, SMTP server parameters for e-mail alerts, Log Listener settings, and administrator user name and password. Note that there is no role-based access and only a single administrator account in the IntruPro Manager, and that user has access to all features of the GUI once logged in. Whilst acceptable for single-device management, this is not a system which is suitable for large-scale enterprise deployments without additional CMS integration. Once a configuration is completed, it can be saved to the database, and synchronised to the Sensor using the Sync button at the top of the screen. During the synchronisation process, the Sensor is unavailable for a short period of time (packets will be passed without inspection). Within IntruPro Manager there is no concept of policies which can be defined, saved and then applied to multiple Sensors. Instead, each Sensor is configure directly and independently - it is not even possible to upload the configuration from one Sensor and apply it to another, which could make large scale deployments cumbersome and error-prone. This is not currently a scalable management solution. For single-device management, however, it is fine, and configuring a Sensor is very straightforward via the Configuration tab. Selecting this tab (after selecting the Sensor to be configured from the drop-down menu) brings up a list of configuration sub-options in the side toolbar, the most important being the Signatures option.
The signature database is divided into both Families (General, Backdoor, DOS, Gain Access, Exploit, Traffic Info) and Categories (General, HTTP, Mail, RPC, FTP, Telnet, DNS, SNMP, ICMP, TCP, UDP, NNTP, IP) and entire Families or Categories can be enabled or disabled by selecting the check box against each in the hierarchical menu tree. Naturally, on selecting a specific Category/Family, the individual signatures within can also be enabled/disabled. Selecting an individual signature will display the description in the lower pane, and it is possible to enable/disable, clone or delete the signature using buttons along the bottom of the screen. All of the signature detail is visible when editing or cloning an existing signature, making it easy to tune any of the built-in signatures as required. It is also possible to create signatures from scratch, and all custom signatures (both those created from scratch and any built-in signatures which have been modified) can be examined via a separate User Rules menu tree (which will remain untouched during future signature update operations).
Full access is provided to all of the fields in all of the protocol decoders (as well as simple pattern matching fields) which can make signature creation somewhat complex for those new to the task, though obviously very powerful for those with more experience. Obviously the ability to clone existing signatures and then modify them should help tremendously in the creation of new ones, and the accompanying documentation is also very good. A very useful search facility allows the administrator to enter simple search criteria (for example, the text “IIS” or “Apache”) and have all the signatures matching that criteria extracted and displayed in a separate window. One or more of those signatures can then be selected via the check box next to each one, and opened for editing via a single click.
There are three more very important buttons on the Signature screen:
Actions taken when signatures are matched with malicious traffic are determined by the Action Settings tab:
We would prefer to see slightly more granularity here - for example, the ability to drop a session (drop packet and mark the rest of the session bad) but not send TCP Resets. Action Settings are applied to each of the Threat Levels (Critical, Severe, Warning, Informational) rather than on a per-signature basis. Thus it is possible to disconnect session for all Critical and Severe alerts, report only for Warning, and ignore all Informational alerts. If it is required to alter the action for a particular signature, all that is necessary is to amend its Threat Level. Changing the action for all Informational alerts from ignore to report, or switching all signatures en masse from disconnect session to report only, is then only a matter of selecting the appropriate action in the Action Settings tab. This is very straightforward and very powerful.
Of course, it does presume that the signatures have been classified accurately in the first place. For example, we noted some signatures which were in the Gain Access or Exploit Families but which were classified as Informational alerts. This was clearly incorrect, and required that we change the Threat Alert manually on those signatures - however, there is a good chance that such mis-categorisation would go unnoticed by the average user. This is a clear warning that even (or especially) where vendors attempt to make life easy for the administrator, it is still necessary to verify manually that the correct signatures are enabled in the active policy, and that the correct actions are being taken for the appropriate groups of signatures. Once again, granularity of change is also an issue. It is not possible, for example, to change the Threat Level for all IIS or Apache signatures in a single operation - it would require a search operation, followed by an extended session editing each signature individually. More work is needed in this area. Other options included in the Configuration tab include:
Once all changes have been made, the configuration is saved to the Database Store, following which it is synchronised with the Sensor (packet inspection is interrupted briefly during this operation). There is no way to save multiple versions of a configuration nor to roll back to a previous version, and there is no audit trail of configuration changes. Rather confusingly, the Alerts tab is not the best place to start monitoring alerts. Intoto Alerts are actually messages generated by filtering the log files based on user-defined rules, and an Alert is generated when any of its properties are satisfied by the log messages generated by the Sensor. Alert criteria can be defined based on the number of log entries generated within a specified time period, destination IP, source IP, Protocol, Intoto Rule ID, Rule Category (HTTP, SSH, FTP, etc.), and Threat Level (Critical, Informational, etc.). Any number of Alert Rules can be defined, and when log entries match one or more Alert Rules, Alerts are generated in the Alert Messages window. Selecting an Alert Message in the upper pane shows the contents in the lower pane, which may consist of a single alert or a number of similar or related alerts (depending on the time period and number of log entries parameters specified). Unfortunately, these are fixed lists containing limited alert information, and it is not possible to view more detail or drill down to the relevant log entries. Alerts are useful as a high-level view of what is being blocked, or for when an administrator wishes to define some very specific criteria for which he wishes to monitor and be informed immediately (i.e. HTTP attacks against a particular Web server). In this respect, they function as a basic correlation capability. More useful, in-depth alert handling is actually provided via the Reports tab, however. The upper pane of the Reports screen contains a number of useful selection criteria which are used to determine the log entries displayed in the lower pane:
Note that it is not possible to filter based on source or destination port, nor on attack description/Rule ID, both of which we consider significant omissions. We found the Aggregation feature to be particularly useful in reducing the “clutter” caused by single exploits raising multiple alerts, since aggregating based on Source IP ensured that only a single line was displayed on screen for each group of alerts from the same source. When aggregation is active, the summary line shows a count of alerts beneath it, and clicking on a small icon to the left of the summary will expand to show the individual alerts. Columns can be re-sized and re-ordered by dragging them around the screen, but it is not possible to sort the list based on a specific column, which we found very frustrating. Another frustration was the inability to save filter settings and column layouts for re-use - instead, it is necessary to define a report from scratch every time you wish to run it.
Having displayed the alerts based on the selection criteria, the administrator can select all of them or individual alerts and save to an HTML file or e-mail them to a specified address. Double clicking any line brings up a window showing the detailed alert information:
A tremendous amount of detail is provided for each alert, and all of this can be e-mailed or saved to an HTML file at the click of a button. Buttons are also available to allow the administrator immediately to modify, delete or disable the rule, making it very straightforward to tune policies quickly based on alerts raised.
The Monitor tab provides another real-time view on the current alerts. A graph will display the attacks in a specific time period (selected from a drop-down menu) categorised by threat response. The lower portion of this screen displays the list of related log messages (limited to the most recent 1000 logs). As with the Alerts screen, these are fixed lists - it is not possible to access further detail on log entries from this screen. In general, those administrators who are happy to simply know that threats are being blocked will probably be satisfied with the information provided by the Alerts and Monitor tabs. Those who want to know a little more detail about what is being blocked, or who want to tune their policy to weed out false positives, will get more use from the Reports tab. The most easily accessible monitoring screen is the “Home” screen which is displayed when the Console is first opened. The Home screen displays basic statistics and an overview of the logs collected:
The Graphs option under the Reports tab provides a simple interface to obtain an overall picture of the intrusions detected and protected by the IntruPro Sensor. A wizard-type interface guides the administrator through defining basic filters to create graphs, and has four steps:
The resulting graphs can be saved as image files by right clicking on the graph and selecting the option “Save as”, but they cannot be otherwise printed or exported directly from this interface. Selecting the Statistics option under the Monitor tab displays the statistics information of each signature Category (i.e. FTP, HTTP, etc.). Selecting a category from the list in the upper pane displays the total number of intrusions by Threat Category below. Naturally, the reports tab - which will probably be used to display only those alerts in the last hour or day when being used for alert analysis - can also be used to filter logs over a more lengthy time period for weekly or monthly reports (the same features apply as mentioned in the previous section). However, given that it is not possible to save report criteria or layout, nor to schedule reports for regular production, we would class reporting as basic in this product. Performance Note that Intoto IntruPro is a software product and was tested on reference hardware (dual Xeon processor, 2GB RAM, copper Gigabit ports) as a 100Mbps IPS device. Performance at almost all levels of our load tests was impeccable, with 100 per cent of all attacks being detected and blocked under all load conditions except at the 2000 connections per second level. We would happily confirm Intoto’s 100Mbps rating for this device under normal network conditions. The basic latency figures of IntruPro on the reference hardware were good for a 100Mbps device across the board under all traffic loads, and behaviour throughout the tests with no background traffic was reasonably consistent and predictable, with small decreases as additional network load was applied from 25Mbps to 100Mbps. HTTP response times were also very good. The device had no problems mitigating our DDOS traffic, although latency did increase significantly during the attack. IntruPro performed consistently and completely reliably throughout our tests, and exposing the sensor interface to ISIC-generated traffic had no adverse effect on the device at any time. Security Effectiveness Out of the box, blocking performance was good at 80 per cent, and was improved to 95 per cent following the application of a signature update after 48 hours. Detection/recognition rate was slightly lower (89 per cent following update) due to the fact that many DOS and ICMP attacks are blocked without alerting by the firewall module. We also noted that many of our test cases raised several alerts for a single exploit. This can be annoying, and we would prefer that the product performed simple correlation to reduce the number of superfluous alerts raised. It can, however, be mitigated by the fact that the source aggregation feature of the management interface results in a single entry on the GUI which can then be expanded to show the individual alerts if required. Performance in our “false negative” tests was good out of the box, and improved following the signature update, leaving just a single miss out of 14. The device initially failed five test cases in our false positive test suite, and although all of these were eliminated following the signature update, we also noted several false positives during our FTP testing, the result of some sloppy port-based backdoor signatures. Overall, false positives would be a concern for us with this device at present, and we would encourage potential purchasers to test in their own network before buying. Resistance to known evasion techniques was excellent, with IntruPro achieving a clean sweep across the board in almost all of our evasion tests. Fragroute and Whisker both failed to deceive the device into ignoring valid attacks, and all but one of the attempts were decoded accurately. Some minor configuration was required to enforce blocking for two of the test cases, but this should become the default configuration in future releases. Most of the miscellaneous evasion techniques (including RPC record fragging) were also handled well. During testing, we determined that the IntruPro device was able to handle up to 45,000 simultaneously open connections whilst successfully maintaining state on our half-open exploits. We consider this to be just adequate for a 100Mbps device. Stateless “exploits” are blocked by default, with no alerts. This behaviour is not configurable, and we would prefer to see a grace period following device start-up during which existing connections (which will appear as mid-flows initially) are passed successfully. Usability The IntruPro Manager provides a usable and straightforward environment for configuring and managing a single Sensor, or small number of Sensors. However, there is no role-based user access and no centralised policy management/distribution capabilities which limits the scope of this product to smaller deployments. Initial configuration is made extremely simple via the ability to click on a single button to select all recommended signatures, and then apply the required actions en masse via the Action Settings tab. This way, it is very simple to switch between a “monitor only” and “block traffic” mode of operation, even though there is no specific “pass through” capability. The search facility within the Signature tab makes it easy to extract a number of related signatures for editing, though it would be nice to have some form of bulk editing capability rather than have to then edit them individually. There are a number of different ways to view alerts, including a very useful custom Alerts tab whereby the administrator can define criteria to match against the logs. This equates to a basic form of high-level correlation, ensuring that only the most relevant log entries are brought to his attention, and can be extremely useful. Also of use are the summary graphs provided in the Monitor tab. Simple, high-level views in the Alerts and Monitor sections can thus be used to provide a rapid indication of what is being blocked, whilst those who wish to pursue a more detailed investigation will find the Reports tab offers most of what they require. The Reports section provides in-depth analysis via custom filters, though it is missing a couple of key filtering options (port and signature ID, for example). In addition, the administrator cannot sort on columns in Reports screen (a standard Windows feature) and it is not possible to save report layouts and criteria for re-use. This necessitates that every report is set up from scratch every time it is run. It is unfortunate that there is no direct drill-down capability from the Alerts or Monitor displays into the detailed log views, but double-clicking on any alert allows it to be edited or disabled directly from the Report screen - very useful for tuning Sensor configuration. An aggregation function provides multiple means to sort and group alerts to reduce on-screen clutter. In the present version, we would class both reporting and alert handling as basic, though adequate for the task in hand. We also feel that although the current system is adequate for single-Sensor management tasks, it is not scalable. Company
name: Intoto Inc.. Click
here
to return to the Intoto questionnaire |
Send mail to webmaster
with questions or
|