NSS Group logo

Westline Athena Aegis IPS 510L V2.1

Executive Summary

Westline offers a range of Athena Aegis appliances from dual 10/100Mbps to 5 x Gigabit interfaces. Although the IPS 510L tested is Gigabit-capable (only one network segment can be monitored in-line, despite sporting two fibre and two copper Gigabit interfaces), Westline rates it at only 200Mbps when all signatures are enabled. This could make the cost per megabit/cost per port higher than it should be for a device of this type. 

Overall, the performance of Athena Aegis IPS510L is very good providing the limit of 200Mbps is not exceeded. Blocking rates were adequate out of the box, and improved to over 90 per cent following an update, and resistance to the most common evasion techniques was excellent. Throughput and latency were excellent under all normal network loads (up to 200Mbps) and across all packet sizes.  

We also found the IPS510L to be very stable and reliable under extended attack, although resistance to high-levels of DOS/DDOS attacks (such as SYN floods) was poor. We would recommend deploying this device behind an additional layer of protection, such as an attack mitigator or firewall with DOS/DDOS mitigation capabilities. 

The device can be managed via the Java-base JConsole connecting to the Intrusion Management Centre (IMC). The IMC can be located either on a dedicated management server, or on the sensor, or both at the same time, and the management system can be deployed as a two-tier or three-tier or hybrid (mixture of the two) system. Alert handling is adequate and reporting is fairly basic. 

Architecture

There are three main components to the Athena Aegis IPS system: 

  • Athena Aegis IPS Appliance
  • Intrusion Management Centre (IMC)
  • JConsole management client 

Athena Aegis IPS Appliance

Athena Aegis IPS is offered in a range of appliances supporting various bandwidth requirements and physical port arrangements, each of which can be configured in transparent or gateway (routed) mode. The latter mode of operation makes use of the integrated stateful firewall, and allows the Aegis IPS to be installed as the main perimeter security device if required (though we would not recommend this with the current version due to lack of protection against high-volume DOS/DDOS attacks). 

The product submitted for testing was the Aegis IPS 510L, a 1U rack-mount appliance with two fibre and two copper Gigabit monitoring ports.  

Despite this physical port arrangement, only one port pair can be used to monitor a single in-line segment at a time - the remaining ports will remain idle. In addition, despite being Gigabit-capable, Westline rates the device at just 200Mbps with typical network traffic when all signatures are enabled.

Also on the front panel is a dedicated 10/100Mbps Ethernet management port, a serial console port, port indicator LEDs, and a two-line LCD display with four buttons. In normal use, the LCD panel will show the Aegis IPS general information such as IPS version, engine status and CPU utilisation.  

Via the four buttons, it can also be used to perform some basic initial configuration of the device - including setting the management IP address and netmask - as well as shutting down/restarting the sensor. 

Standard hardware is used throughout the Aegis IPS product range, based on Intel hardware with 512MB to 2GB memory, depending on the model. No network processors or other hardware acceleration techniques are employed in the current version. The appliance operating system is custom written, which has allowed Westline to take full advantage of multi-processor systems to provide maximum packet processing performance. 


Figure 1 - Aegis IPS: Architecture

In order to make use of SMP architectures, the IPS engine (originally based on open source software) is multithreaded, dedicating one CPU to capturing packets, and another for processing and sending/dropping them. Unlike typical Linux-based operating systems, this all happens at the kernel level to maximise performance and minimise latency when under load. When the engine is slightly overloaded, the cache can help minimise packet drops. 

There is no redundancy built in to the device (certain appliances feature dual power supplies), and nor is there a High Availability solution on offer at present. Westline does, however, offer an optional copper bypass device to allow traffic to pass unimpeded should the Aegis IPS appliance fail. 

Intrusion Management Centre (IMC)

The Intrusion Management Centre (IMC) is the server component which provides alerting, reporting and device management capabilities.  

A unique feature of Aegis IPS is that the IMC software can be built-in to the sensor or external. Each sensor comes with a built-in management server which is enabled by default, allowing one or more consoles to connect directly to the sensor out of the box, thus operating in a two-tier mode.

In addition, identical IMC software can be installed on an external server running Red Hat Linux with a MySQL database for centralised provisioning, consolidated reporting and multi-sensor management from a single console.  

This configuration is known as hybrid mode, since both two-tier and three-tier modes are supported simultaneously, allowing the console software to connect either to the IMC server, or to the sensor directly, or both as required. In this mode, alerts are stored locally on the sensor as well as centrally on the IMC server. 

The built-in IMC software is identical to the external version in all respects apart from the lack of any scheduling capability. Each sensor comes with this software on the installation CD and a free single-sensor management license is provided. Licenses to manage multiple sensors can be purchased as required. 

The Aegis IPS can also be configured with a pure 3-tier architecture. In this mode, the internal IMC software is disabled completely and the console can no longer connect to the sensor directly, but only via the central IMC server. Whilst this can provide a slight performance boost in heavily loaded networks, the downside with this mode is that alerts are no longer stored locally on the sensor. Thus if communication between IMC server and sensor is interrupted for any reason, alerts will be lost. 

Multiple read-only connections can be made to any IMC, meaning that multiple consoles can connect to an IMC server or to a sensor directly, and that multiple IMC servers can connect to each sensor. Read/write management control is achieved by an individual console acquiring a single management token per sensor. 

JConsole

This is a Java-based console which provides a graphical management interface for the Aegis IPS system. JConsole can connect to the IMC software over a secure channel, either directly to the Aegis IPS sensor, or to a central IMC server. 

Performance

The aim of this section is to verify that the sensor is capable of detecting and blocking exploits when subjected to increasing loads of background traffic up to the maximum bandwidth supported as claimed by the vendor.  

For each type of background traffic, we also determine the maximum load the IPS can sustain before it begins to drop packets/miss alerts. It is worth noting that devices which demonstrate 100 per cent blocking but less than 100 per cent detection in these tests will be prone to blocking legitimate traffic under similar loads. 

The Athena Aegis IPS 510L was tested up to 200Mbps, the rated speed of the device, and performance at all levels of our load tests was impeccable, with 100 per cent of all attacks being detected and blocked under all load conditions. We would happily confirm Westline’s 200Mbps rating for this device.

The basic latency figures of the Athena Aegis IPS 510L were acceptable across the board under all traffic loads. They ranged from 315�s with 50Mbps of 256 byte packets, to 416�s with 200Mbps of 1000 byte packets. Behaviour throughout the tests with no background traffic was consistent and predictable, with small increases as additional network load was applied from 50Mbps to 200Mbps.  

Placing the device under a half load of 100Mbps of HTTP traffic actually had the effect of reducing latency further, falling from 315�s to 156�s with 256 byte packets, 343�s to 187�s with 550 byte packets, and from 403�s to 244�s with 1000 byte packets. The effect of this is that latency under normal traffic loads is excellent for a device of this type - HTTP response times, for example, were also excellent. 

The device did have problems with SYN flood traffic, however, and would perform best when situated behind an attack mitigation device or a firewall with SYN flood mitigation capabilities. Latency when under 20Mbps of SYN flood traffic was excessive, as was packet loss. The overall effect was an increase in response times and a high number of failed transactions. 

This is an issue which Westline recognises and is working on at the time of writing. Future releases will include per-IP based rate limiting capabilities to help mitigate such attacks. For now, our recommendation would be to deploy the device as mentioned above, with additional protection in front. 

Athena Aegis performed consistently and completely reliably throughout our tests. Under eight hours of extended attack (comprising millions of exploits mixed with genuine traffic) it continued to block 100 per cent of attack traffic, whilst passing 99.9975 per cent of legitimate traffic. This is considered acceptable for a device of this type. 

Exposing the sensor interface to ISIC-generated traffic had no long-term adverse effect on the device, although communications between the management console and the management server/sensor were interrupted during the attack.  

Please refer to the Testing Methodology section for full details of the methodology used and performance results. 

Security Effectiveness

We installed one sensor with the latest updates, using a slightly modified version of the Detect_All policy, disabling Low severity informational signatures only. All exploit signatures are enabled in this policy. 

Out of the box, signature recognition was adequate at 75 per cent, and was improved to 88 per cent following the application of a signature update after 48 hours. Blocking performance was actually four per cent higher throughout - giving a much more creditable final result of 92 per cent - due to various types of malformed/invalid packets (such as those used in our DOS test suite) being blocked silently. 

Performance in our “false negative” tests was adequate out of the box, and improved slightly following the signature update. However, there were still four misses out of the 14 test cases following the signature update, indicating that many signatures are written for specific exploits rather than for the underlying vulnerability.

Having said that, we did notice that where a deliberate attempt was not made to evade the device, Athena Aegis IPS did extremely well at detecting most of the variants that we include alongside our basic test cases (i.e. if five different exploits are known for a vulnerability, the signature is generally written well enough to detect all of them).  

A major concern in deploying an IPS is the blocking of legitimate traffic. The device failed only one test case in our false positive test suite, and this was rectified following the signature update. No other false positives were noted during stress testing with either Avalanche traffic or real-world traffic mixes. 

Resistance to known evasion techniques was very good, with the Athena Aegis IPS achieving a clean sweep across the board in most of our evasion tests. Fragroute and Whisker both failed to deceive the device into ignoring valid attacks, and all of the attempts were decoded accurately.  

Of the miscellaneous evasion techniques, non-text Telnet opcodes in FTP data streams and RPC fragmentation both proved troublesome. Full RPC and FTP protocol parsers are under development for a future release.  

Out of the box, Westline claims that Athena Aegis IPS can handle just over 250,000 open connections, and this was verified in our tests. The default operation of the device is to reject new connections when the state tables are full or resources are low, and this meant that once we exceeded the connection limit the device continued to maintain state on existing connections (thus detecting our half-open exploit), but inevitably, as a consequence, began to block legitimate traffic. This behaviour is not configurable. 

Stateless “exploits” are not alerted upon (this is correct behaviour in order to be resistant to Stick and Snot tools) and mid-flows are blocked by default (a mid-flow violation alert is raised, if configured). It is not possible to configure the device to allow mid-flows.

Please refer to the Testing Methodology section for full details of the methodology used and performance results. 

Usability

This part of the test procedure consists of a subjective evaluation of the features and capabilities of the product, and covers installation, configuration, policy editing, alert handling, and reporting and analysis

Installation

Installation of the Aegis IPS appliance is very straightforward, even though it is not possible to perform the entire initial configuration via the front-panel LCD display. Instead, the serial console port is used, which brings up a menu offering the following options: 

  • Change IPS system mode (transparent or gateway/routed mode)
  • System reboot
  • System shutdown
  • Change IP address of administrative interface
  • Sensor information
  • Diagnostic information
  • Restore default configuration
  • Change time zone of the system
  • Set up default gateway
  • Change time of the day
  • Display current time
  • Change sensor to use external IMC
  • Change sensor to use built-in IMC
  • Configure third-party firewall
  • Set up DNS server
  • Set up DMZ and external interfaces (routed/gateway mode only)
  • Set up routes
  • Set up allowed IPs for management
  • Change monitoring port pair (copper of fibre) 

Note that there is no option to set a password or secret key for sensor-to-IMC/console communications. Instead, all sensors have a default password which is used to establish initial connection to the IMC or console. We consider this to be too insecure, even though it is necessary to specifically define which hosts are allowed to connect to the sensor for management (thus making it harder for a rogue console to gain access to a sensor). The user name and password can be changed via the GUI once initial connection has been established. 

If an external IMC is required, this can be installed on a Red Hat Linux platform which must be supplied by the end user (Westline sells the IMC as a software-only product). Although the procedure is documented in the manual, we felt it was not quite automated enough, requiring several commands to be issued at the command line (including separate and unassisted installation of the MySQL database) rather than wrapping everything in a single, automatic installation routine. 

Installation of the JConsole software is a very simple process, especially since there is no requirement for a separate Java Runtime Environment (JRE) to be installed in order for it to operate. 

Documentation is adequate, consisting of separate PDF-based user guides for the sensor and IMC. 

Configuration

The JConsole management GUI consists of three panes, any of which can be disabled to create more screen real estate.  

The pane on the left side of the screen is called the Function Window, and contains a hierarchical menu of “sites” (Westline’s term for devices - whether IMC servers or sensors) and management options within those sites. The right-hand pane is the Management Area, which is used to configure whichever option is currently selected in the Function Window.  

As entries are selected in the Function Window, a new tab is opened in the Management Area, and any number of these tabs can remain open allowing the administrator to switch quickly between them. Below these, running the full width of the screen, is the Message Window, which displays messages relating to sensors and IMC servers in real time.

A site is actually an instance of the IMC, and thus can either be a single sensor, or a central management server which controls multiple sensors. A site can be added to a console by anyone who knows the IP address, user name and password of that device, and whose own IP address has been defined as an allowed management console at the sensor. There is no other control, meaning that it is possible for two different administrators to define the same sensor within their own console and configure it independently of each other.

The only protection against this situation is that, although multiple read-only access is allowed in all cases using only the site password, write access is only granted to the person who has acquired the “management token” for that site. This is accomplished via a menu option within the console (it is released in the same way, or by exiting the console), and prevents two people from actually updating a sensor at the same time. However there is no protection from two separate administrators updating the same sensor with different policies at different times - a potentially dangerous situation from a security standpoint, made worse by the fact that there is no audit log available listing what changes were made to which sensors and policies. 


Figure 2 - Aegis IPS: Viewing alert details in the multi-pane JConsole

At the moment we feel that Westline’s definition of a site is flawed. A site should be just that - a central site containing multiple devices (sensors and/or management servers) which are defined within it. Devices created within a site should be assigned to that site only, and should then be unavailable for management stand-alone or as part of a site created on another management server.  

Each console is also protected by a simple password (not even a user name), which then provides unlimited access to the sites defined within that console. There is no role-based security model in place to ensure that certain administrators have access to certain sites, or are restricted to read-only access, for example. 

As it stands, we feel that the Westline management model is lightweight, and suited only to small-scale deployments where there is a single administrator managing a small number of sensors. This is unfortunate, since the ability to operate the system in either two-tier, three-tier or hybrid mode makes for a very flexible and scalable system otherwise. Hopefully this will be improved in future versions, which will make the management system very much more powerful and flexible overall. 

On first entering the console, the administrator is presented with a list of available sensors and/or IMC servers in the Function Window. Selecting any one of these allows him to connect to that device (more than one device can be connected from a single console) which brings up the hierarchical menu list beneath. Note that the menu list is duplicated under every connected device, and when in hybrid mode it would be possible to make changes to both the IMC server and the server(s) which the IMC controls.  

It is up to the administrator to remember which IMC controls which sensors and ensure that changes are made in one place only. Once again, we feel that it would be better if these relationships were enforced by the IMC and console to prevent potentially conflicting changes being made in more than one place.


Figure 3 - Aegis IPS: Monitoring sites within JConsole

Single sensor details can be viewed by double-clicking the sensor’s menu entry, and a list of sensors within a site can be viewed by double-clicking the site entry. We found the interface to be counterintuitive, on occasion - sometimes requiring a right-click, and other times requiring a double-click. Details of all available sensors in a site are listed in a table, including: 

  • Sensor IP
  • Serial number
  • Driver version
  • Status
  • Mode
  • Policy installed
  • Disk usage 

By selecting individual sensors, it is possible to view real-time statistics, including: 

  • Interface used for monitoring
  • Packets received per interval
  • Bandwidth bytes per second
  • Packets processed per second
  • Alerts generated per second
  • TCP SYN per second
  • TCP ACK per second
  • New TCP sessions per second
  • Closed TCP sessions per second
  • Number of TCP sessions per interval
  • CPU loading
  • Packets lost per interval 

We found these statistics to be very accurate, and very useful for troubleshooting. It would be nice if all vendors would expose such statistics via the management GUI in this way. 

Options are also available to start, stop and restart the engine, shutdown and restart the sensor, set external logging and alerting options (syslog and SNMP are supported currently), set alert options for the protocol parsers, packet decoders, TCP streaming engine, and mid-flow detection, and configure the options for SYN flood, UDP flood and ICMP flood defences. 

Finally, the alert database can be backed up, restored or purged, and software updates and signature updates can be applied from local files downloaded from the Westline Web site. Signature updates can also be applied direct from the Westline Web site or from local IMC servers, and updates can be scheduled for regular, automatic application.

Policy Management

Policies are defined at site level, which means they can apply to a single sensor, or to multiple sensors controlled by single IMC. 

Policies are made up of three elements: 

  • Network Objects - Entries describing individual hosts, networks (groups of hosts) and groups of networks. This allows the administrator to assign descriptive names to each of the above objects (i.e. DMZ instead of 10.10.16.0/24)
  • Service Objects - Entries describing Ports (i.e. HTTP=TCP/80) and Firewall Services (i.e. Finger=TCP/79 and UDP/79). Although similar, Ports are used in IPS rules, whilst Firewall Services are used in firewall rules.
  • Attack Objects - Entries describing attack signatures and groups of signatures.

When an Object window is opened in the Management Area within JConsole, it is divided into two parts: the Objects currently loaded in the IPS Engine, and the Objects currently in the configuration database (i.e. those being worked on currently in JConsole). All Objects in the Engine are read-only. As information is changed within JConsole it is changed in the configuration database only, and those Objects must then be explicitly applied to the Engine.  

Attack Objects, also known as Signature Objects are obviously the heart of the detection and prevention system, and Aegis IPS has over 2300 signatures built-in. All signatures are assigned to an appropriate Signature Group to make them easier to handle. The administrator can create custom Signatures and Signature Groups as required, and existing signature groups can be duplicated. It is not possible to amend the built-in Signatures, however. 


Figure 4 - Aegis IPS: Viewing/Editing Attack Objects

When creating or viewing Signatures, the Signature detail is divided into five tabs: 

  • Basic - Signature name, severity, source and destination port
  • General IP Header - Source and destination IP, time to live, type of service, option flags, and so on
  • Specific IP Header - Protocol-specific options, such as IP flags, ICMP type, and so on
  • Attack Pattern - Data size, offset, depth, pattern to match, and so on. This information is viewable on custom signatures only, and unfortunately is not visible for built-in signatures. Regular expressions are not supported.
  • Reference - CVE/Bugtraq IDs (built-in Signatures only) 

Whilst it is clearly not possible to create complex signatures, this would normally be enough for most administrators.

A detection policy is a collection of IPS, Firewall and NAT rules, with associated notification and action options. There are eight built-in policies provided by the system which cannot be changed or deleted: 

  • DETECT_ALL - Alert only policy that detects all attacks, scanning and normal activities.
  • RECOMMENDED_DETECT_AND_BLOCK - A policy that combines both alert and block actions, suitable for inspecting suspicious activities and blocking attacks. Chat programs and P2P are blocked.
  • RECOMMENDED_DETECT_ONLY - Another version of the policy above but where all block actions are changed to alert only. Suitable for initial deployment and monitoring.
  • RECOMMENDED_BLOCK_ONLY - A policy that focuses on blocking attacks, suitable for perimeter defence. Chat programs and P2P are blocked.
  • VERY_STRICT - A policy that blocks all exploits and potential scanning activities. To be used with caution.  


Figure 5 - Aegis IPS: Policy editor

New/custom policies can be created from scratch or based on existing policies and modified. Each policy contains multiple Rules, grouped by severity, each Rule containing the following components: 

  • Rule number within the policy - displayed on the alert screen when the rule is triggered for ease of identification
  • Status - Active or disabled
  • Source Group - Selected from the Network Objects. Can be a single host or an entire network, and it can be negated
  • Destination Group - Selected from the Network Objects. Can be a single host or an entire network, and it can be negated
  • Signature/Signature Group - Multiple Signatures and/or Signature Groups can be selected for each rule
  • Severity - The default severity value as defined in each attack object can be used when raising alerts, or it can be overridden on per-Rule basis. The system allows ten custom levels besides the standard “Critical”, “High”, “Medium”, “Low”, and “Info”.
  • Action - The Action column defines the response of the sensor when a Rule is matched. The following actions are available:
  • Default action - drop packets in IPS (alert only in IDS), raise alert to JConsole, log packet which triggered the Rule
  • Alert Only - raise alert to JConsole
  • Log TCP session traffic - log remainder of packets for same TCP session
  • Drop packet by IPS - trigger packet is dropped along with the rest of the connection
  • Close client connection - send TCP reset to source IP
  • Close server connection - send TCP reset to destination IP
  • Close connection on both sides - send TCP reset in both directions
  • Block by external firewall - configure rules on third party firewalls
  • Log packets - Record all packets from source IP or to destination IP for specified time period after alert is raised
  • Silent drop - Packet is dropped (and connection is marked bad) without raising an alert
  • Notification - The default behaviour of Aegis IPS is to log the attacking packets and send out an alert to JConsole. The following notifications are also supported:
  • No action
  • Syslog
  • SNMP Trap
  • Comment - description of rule 


Figure 6 - Aegis IPS: Editing Rules

Firewall and NAT rules are defined in a similar way. The Firewall rules allow the administrator to block or allow packets at a lower level than the IPS Engine, perhaps simply blocking all packets from a specific source address without even inspecting them. Naturally, NAT rules come into play only when the device is deployed in routed/gateway mode.

Note that any number of Signature Groups or individual Signatures can be added to each rule, and since each rule can be restricted to specific source or destination IPS addresses/networks this allows the administrator to define very flexible and powerful policies. For example, it would be possible to drop all Web-based exploits when directed at the internal network, but alert only when directed at the DMZ. 

A useful search capability is provided within the policy editor to allow the administrator to search for, say, all Signatures with “Apache” in the description when defining a Rule. Unfortunately, there is no bulk edit capability, so it is not possible to change a number of Rules - perhaps from Active to Disabled - in a single operation. This can make policy tuning quite tedious. 


Figure 7 - Aegis IPS: Installing Policies

Once the policy has been defined, it can be deployed from JConsole to a sensor. The top level policy screen provides a list of available policies, together with a description, number of active rules, and on which sensor(s) that policy is installed. What is missing, of course, is an indication of when the policy was last updated, if it has been updated since it was installed, who updated it, and what changes were made. There is also no facility for rolling back to previous versions of a policy. 

It would be nice to be able to select a policy from this list, and then choose one or more sensors to which is should be applied. Unfortunately, this is not possible. Instead, it is necessary to select a Site (device), and click on the Install Policy option, installing one policy to one device at a time.  

This could be tedious in an environment with a large number of sensors and a small number of policies. Changing one policy could result in quite a lot of work for the administrator in deciding first where that policy is to be deployed, and then actually deploying it. This needs improvement.

Alert Handling

There are three ways to view alerts in JConsole: Alert Monitor, Alert Summary and Alert Analysis

The Alert Monitor screen is a near real-time display of alerts as they are raised by the sensor, whilst the Alert Summary screen provides counts of the top 10 attack signatures, top 10 source IPs and top 10 destination IPs. The Alert Summary screen thus provides a rapid, high-level view of which attackers are sending which exploits to which victims, whilst the Alert Monitor provides the detail beneath that.  


Figure 8 - Aegis IPS: Alert Summary

It is possible to drill down one level in the Alert Summary screen. By double-clicking on one of the attack signature entries, a separate window will appear listing the source and destination IPs involved in those particular attacks. It is a pity that it is not then possible to drill down further to see the actual alerts. 

The Alert Monitor is divided into two windows, with a tabular list of the alerts sorted by date and time at the top, and a number of tabs containing details of the current alert selected at the bottom. The following information is presented in the tabular display: 

  • Timestamp
  • Severity
  • Sensor ID
  • Interface
  • Interface in/out
  • Signature name
  • Source IP address
  • Source MAC address
  • Source port
  • Destination IP address
  • Destination MAC address
  • Destination port
  • Protocol
  • Payload (indicates if payload data was recorded)
  • Event type (normal alert or packet recording)
  • Rule ID within policy
  • Action taken (alert, drop, etc.)
  • Notification (alert, syslog, SNMP) 

If alerts are arriving too quickly to view, the real-time update can be suspended at any time. It is a pity that once suspended it is not possible to re-sort on any column, add/remove columns, or right click on an entry and drill down on that data (i.e. right click on a particular source IP in order to display all alert from that IP). The Alert Analysis screen must be used for all drill-down operations. 

The auto-update interval can be adjusted to suit the administrator, as can the maximum number of alerts displayed. Note that this display should be considered as a “window” into the MySQL database which always shows the most recent alerts - once the maximum number of alerts is exceeded the earlier ones are discarded to make way for the new ones. However, the alert details remain in the database for further reporting and analysis. This is a reasonable compromise between availability and performance. 


Figure 9 - Aegis IPS: Alert Monitor

Selecting an alert within the table in the top window displays details of that alert in the bottom window.  

A total of seven tabs are available in this window, some of which may be greyed out depending on the protocol used by the exploit and whether or not packet data was recorded:

  • Summary - Replicates all of the data items listed previously from the table of alerts
  • IP - IP-specific data such as version, header length, flags, TTL, checksum, and so on
  • TCP - TCP-specific data such as source port, destination port, sequence number, flags, and so on
  • UDP - UDP-specific data such as source port, destination port, etc.
  • ICMP - ICMP-specific data such as ICMP type, ICMP code
  • Payload - Full protocol decode of the packet which triggered the alert, containing both hex and ASCII data
  • References - List of external references such as Bugtraq ID, CVE reference, and so on. 


Figure 10 - Aegis IPS: Alert Analysis

More detailed historical analysis is performed via the Alert Analysis option. This is a cross between the Alert Monitor and Reporting functions, in that it replicates the tabular display of the Alert Monitor, but instead of listing current alerts in real-time, it provides access to the historical alerts stored in the MySQL database via a set of search criteria: 

  • Time Range
  • Sensor - Single sensor, or consolidated across all sensors under the same IMC
  • Protocol
  • Source IP
  • Source Port
  • Destination IP
  • Destination Port
  • Signature(s) - One or more signatures can be selected 

Unfortunately, it is not possible to save the search criteria for re-use.

The display is identical to that provided by the Alert Monitor, except with this option the administrator can sort on any column. Double-clicking on a Signature brings up a window containing details of that Signature, but not the pattern matched for built-in Signatures, which can make it hard to determine why a false positive alert has triggered.  

It is not possible to go directly to the appropriate signature in the appropriate policy in order to edit the policy directly for tuning purposes, and nor is it possible to drill down on a particular field in order to, say, list all alerts for a particular target IP. Instead, it would be necessary to return to the selection criteria, specify the required target IP, and regenerate the query. This is cumbersome compared to some of the current competition. 

Reporting and Analysis

A number of standard reports are included in the IMC covering both IDS/IPS and Firewall: 

  • IDS/IPS Reports
  • Top Scan Sources (All/TCP/UDP/ICMP)
  • Top Scan Targets (All/TCP/UDP/ICMP)
  • Top Attacks by Severity (All/TCP/UDP/ICMP)
  • Top Attacks by Signature (All/TCP/UDP/ICMP)
  • Attacks Over Time (All/TCP/UDP/ICMP)
  • CPU loading
  • Custom Report
  • Firewall Reports
  • Top Connection by Sources (All/TCP/UDP/ICMP)
  • Top Connection by Targets (All/TCP/UDP/ICMP)
  • Custom Report 


Figure 11 - Aegis IPS: Specifying report criteria

Selecting a report brings up the report option windows in the Management Area. Different reports have different selection criteria on which the results can be filtered: 

  • Time Range - The time period covered by the report. The default is one month
  • Sensor - Reports can be for a single sensor, or consolidated across all sensors under the same IMC (the default)
  • Interface - The default is to include all interfaces under the same sensor
  • Source IP
  • Destination IP
  • Source Port
  • Destination Port
  • Top N - The default is top 20
  • Severity
  • Action
  • Signature(s) - One or more signatures can be selected (maximum is 500)
  • Interval - The default is one month 

The Custom Report option provides all the same selection criteria, but allows the data columns to be specified in addition. Once the selection criteria have been established, the report can be generated immediately or scheduled for hourly, daily or weekly automated runs (though it is not possible to specify a time for the report to be run - an unfortunate omission). The finished report is presented on-screen as both a graph and text-based table.  


Figure 12 - Aegis IPS: Typical report

As with other options selected from the Function Area menu, each report is opened in its own tabbed window in the Management Area.

This allows the administrator to generate several reports at once and switch quickly between them. He can also customise the report format (pie chart or bar chart), colours, fonts, plot styles, and so on, before exporting or printing the report.  

The report can be printed directly, saved as a PDF file, or exported to Excel, HTML, CSV or plain text formats. Scheduled reports are always saved as PDF files, and they must be manually retrieved from the IMC hard drive - it would be nice if they could be automatically e-mailed or uploaded to a Web or FTP server. 

We found report generation to be slow when there were a million or more alerts in the database, and, unfortunately, it is possible to save neither the selection criteria nor the custom formatting for re-use. 

Verdict

Performance

The Athena Aegis IPS 510L was tested up to 200Mbps, the rated speed of the device, and performance at all levels of our load tests was impeccable, with 100 per cent of all attacks being detected and blocked under all load conditions. We would happily confirm Westline’s 200Mbps rating for this device. 

The basic latency figures of the Athena Aegis IPS 510L were acceptable across the board under all traffic loads. Behaviour throughout the tests with no background traffic was consistent and predictable, with small increases as additional network load was applied from 50Mbps to 200Mbps.  

Placing the device under a half load of 100Mbps of HTTP traffic actually had the effect of reducing latency further. The effect of this is that latency under normal traffic loads is excellent for a device of this type - HTTP response times, for example, were excellent. 

The device did have problems with SYN flood traffic, however, and would perform best when situated behind an attack mitigation device or a firewall with SYN flood mitigation capabilities. Latency when under 20Mbps of SYN flood traffic was excessive, as was packet loss. The overall effect was an increase in response times and a high number of failed transactions. 

This is an issue which Westline recognises, and is working on at the time of writing. For now, our recommendation would be to deploy the device as mentioned above, with additional protection in front. 

Athena Aegis performed consistently and completely reliably throughout our tests. Under eight hours of extended attack (comprising millions of exploits mixed with genuine traffic) it continued to block 100 per cent of attack traffic, whilst passing 99.9975 per cent of legitimate traffic. This is considered acceptable for a device of this nature. 

Exposing the sensor interface to ISIC-generated traffic had no long-term adverse effect on the device, although communications between the management console and the management server/sensor were interrupted during the attack.

Security Effectiveness

We installed one sensor with the latest updates, and used a slightly modified version of the Detect_All policy, disabling Low severity audit/informational signatures only. All attack/exploit signatures are enabled in this policy. 

Out of the box, signature recognition was adequate at 75 per cent, and was improved to 88 per cent following the application of a signature update after 48 hours. Blocking performance was actually four per cent higher throughout - giving a much more creditable final result of 92 per cent - due to various types of malformed/invalid packets (such as those used in our DOS test suite) being blocked silently. 

Performance in our “false negative” tests was adequate out of the box, and improved slightly following the signature update. However, there were still four misses out of the 14 test cases following the signature update, indicating that many signatures are written for specific exploits rather than for the underlying vulnerability.  

Having said that, we did notice that where a deliberate attempt was not made to evade the device, Athena Aegis IPS did extremely well at detecting most of the variants that we include alongside our basic test cases (i.e. if five different exploits are known for a vulnerability, the signature is generally written well enough to detect all of them).

A major concern in deploying an IPS is the blocking of legitimate traffic. The device failed only one test case in our false positive test suite, and this was rectified following the signature update. No other false positives were noted during stress testing with either Avalanche traffic or real-world traffic mixes. 

Resistance to known evasion techniques was very good, with the Athena Aegis IPS achieving a clean sweep across the board in most of our evasion tests. Fragroute and Whisker both failed to deceive the device into ignoring valid attacks, and all of the attempts were decoded accurately.  

Out of the box, Westline claims that Athena Aegis IPS can handle just over 250,000 open connections, and this was verified in our tests.  

Usability

Westline offers a range of deployment options, from two tier (managing each sensor directly from the Console), to three-tier (using an IMC to manage multiple sensors) and even “hybrid mode” (a mix of two-tier and three-tier). This makes for a very flexible and scalable system.  

However, in the current release the implementation is flawed slightly, with no role-based access, not enough security during initial connection of sensors to management servers/consoles, and not enough protection to prevent the same sensor being updated with different policies via two different management servers/consoles. Once these issues have been rectified, the Westline management model will be very impressive. 

Policy management is flexible and powerful thanks to the rules-based approach which allows, in effect, different policies to be applied on a per-host or per-network basis. Some improvement is still required in the area of bulk edits and provision of audit trails for policy changes, but the current solution is very usable as it stands.

Many organisations will also find useful the ability to deploy this device in routed gateway mode complete with stateful firewall and NAT. 

Alert handling is adequate, and although we would like to see more drill-down capabilities from the Alert Monitor screen, all the necessary facilities are available, including real-time alert monitoring, high-level counts and summaries, and a more detailed log analysis capability.  

It would be useful to be able to go directly to the appropriate signature within a policy from an alert to disable the signature or fine tune its configuration parameters. 

Reporting is fairly basic, with several built-in graphical reports and a limited custom reporting capability. The biggest omission here is the inability to save customised report layouts and selection criteria for re-use. 

Contact Details

Company name: Westline Security Limited
E-Mail:  [email protected]
Internet: www.west-line.net

Address:
2/F Shui On Centre
8 Harbour Road
Wanchai
Hong Kong
Tel: +852 2824 8911
Fax: + 852 2824 8911

Click here to return to the Westline questionnaire
Click here to return to the IPS Index Section

top         Home

Certification Programs

Group Test Reports

White Papers

On-Line Store

Contact The NSS Group

Home

Send mail to webmaster with questions or 
comments about this web site.

Copyright � 1991-2006 The NSS Group Ltd.
All rights reserved.