 |
|
|
Snort is a freeware Network Intrusion Detection System which was written for Unix/Linux systems and later ported to the Windows platform. For this report, we have tested the latest stable release for non-Windows platforms (version 2.0.1). Snort is described as a cross-platform, lightweight network intrusion detection tool that can be deployed to monitor small TCP/IP networks and detect a wide variety of suspicious network traffic as well as outright attacks. There is nothing lightweight about its flexibility and performance, however. It has a wealth of development effort behind it, in terms of both source code and attack signature production, and more often than not has the very latest attack signatures available for it before they are available for commercial IDS products. Snort is available under the GNU General Public License, and is free for use in any environment, making the deployment of Snort as a network security system more of a network management and coordination issue than one of affordability. Because of the freeware nature of the product, our approach to testing Snort is slightly different to that for commercial offerings. For example, Snort itself is basically a packet sniffer and rules-based logging engine, with advanced pattern-matching capabilities providing IDS functionality. It does not, however, feature any user-friendly GUI management interface or reporting capability. Configuration is achieved by editing text files, and output is a simple dump of suspicious packet contents and alerts to disk or database. Being open source, there are, of course, numerous freeware solutions available that can provide front-end and back-end functionality, but the permutations are too numerous for us to test effectively. Most commercial offerings are tested �out of the box� and that is how we have chosen to test Snort - utilising only the basic distribution and rule sets provided on www.snort.org. Architectural information and other technical details have also been sourced extensively from the latest-available documents at www.snort.org, whilst technical support during testing and completion of the vendor questionnaire was provided by Sourcefire (www.sourcefire.com), creators of Snort and producers of a range of commercial IDS appliances based on the Snort engine. The overall architecture of a basic Snort deployment is simple, since it consists only of the sensor itself - there is no management server, no graphical console, and no three-tier architecture. The architecture of the Snort engine itself has changed radically for version 2.0 in order to improve stability and performance. The new Detection Engine is broken into three distinct technologies or phases: the Rule Optimiser, the Multi-Rule Inspection Engine (this includes the standard Snort validation), and the Event Selector. The Rule Optimiser utilises a set-based methodology for managing Snort rules and applying them to network traffic. Rule subsets are formed based on unique rule and packet parameters using a classification scheme based on set criteria. This allows the entire Snort rule set to be divided into smaller subsets of rules based on these unique parameters. The benefit of this set-based methodology is that the Snort rule subsets are predetermined during initialisation. Since these subsets are based on the unique rule parameters such as source port, destination port, and rule contents, each rule subset consists of the complete set of rules that are applicable to each packet. This guarantees that all applicable rules are tested against each packet, and ensures rules that cannot possibly match the packet are ignored. When Snort begins running, it reads and parses all the activated rules. The Snort rules are then passed to the Rule Classifier, which classifies them into subsets. This is done prior to any packet or stream processing. Once the Snort rules have been divided into subsets, each incoming packet is matched to a corresponding rule set based on the packet�s unique parameters. For example, if Snort is run with 1500 rules, these 1500 rules are divided into smaller subsets based on transport and application-layer protocols. So 500 of these rules may go into the HTTP client rule set, another 50 rules could go into HTTP server rule set, and so on. Once Snort proceeds to the rule processing stage for each packet, the packet parameters are passed to the Rule Manager to select the appropriate subset of rules to apply to a packet. Once the rule set is selected, the multi-rule search processing begins. Multi-rule inspection uses algorithms that make use of the optimised rule sets. Since all of the rules that need to be inspected are in one rule set, the inspection algorithms organise the rules so that the rules can be inspected simultaneously. This simultaneous inspection is the key to speed enhancements in multi-rule inspection. By using simultaneous inspection techniques combined with an optimal rule set, the overall inspection processing time is reduced by orders of magnitude. Snort 2.0 introduces a new high performance Multi-Rule Inspection Engine which is responsible for detecting rule matches during packet processing. Packets are first analysed by the Rule Optimiser to select the appropriate set of rules for inspection. Then the Multi-Rule Inspection Engine searches for rule matches, builds a queue of detected rule matches, and selects the best rule match to log based on a simple set of event priorities. The Multi-Rule Inspection Engine is broken into three distinct searches based on unique Snort rule properties:
The Inspection Engine uses a configurable, high performance, multi-pattern search engine to find all occurrences of protocol field and generic content patterns. The packet anomaly rules are processed using a search scheme based on the standard Snort rule processing. When a match is found in any of these three search types, the standard Snort processing fully validates the specific Snort rule. If the Snort rule is validated, an event is generated and added to the event queue. Once the Inspection Engine has completed processing the packet, the Event Selector processes the event queue. The event queue allows Snort to track every occurrence of every rule match event within a packet. The event selector then prioritises events from the event queue and selects events based on the assigned priority. Currently, the event with the longest single match is considered to have the highest priority and is selected. This event is then sent to the Snort output system. The Protocol Flow Analyser classifies network application protocols into client and server data flows. In-depth analysis of these protocol data flows allows the Detection Engine to make intelligent decisions about protocol inspection, greatly enhances performance and efficiency, and helps to reduce false positives. Protocol flow analysis is performed at a high level and is usually only concerned with a few important aspects of a particular protocol flow, such as a server response code or a client request type. Flow analysis does not replace other protocol inspection technologies - instead it complements them. It is a generic analysis that allows an application protocol to be classified as a client or server flow. Once an application protocol is classified into a client flow and a server flow, it gives Snort useful knowledge as to the type of inspection and the regions of the protocol flow to inspect. Protocol flow analysis gives Snort rudimentary knowledge of a particular application protocol. With this knowledge Snort can determine which, if any, part of a protocol to inspect. This significantly reduces processing time and reduces false positives by limiting the amount of inspection that is done. Currently, the Detection Engine has an HTTP flow analyser that is user-configurable and significantly reduces the engine�s HTTP processing time. Essential functions such as packet reassembly, port scan detection and HTTP traffic normalisation are performed by plug-in modules known as preprocessors. These can perform a variety of functions on packets - such as modifying packet contents, raising an alert, logging packet contents, or signalling the Detection Engine not to process the packet at all - before passing them to the Detection Engine. The stream4 module, for example, provides TCP stream reassembly and stateful analysis capabilities. Stateful stream reassembly capabilities allow Snort to ignore �stateless� TCP attacks such as those produced by tools like Stick and Snot. Stream4 also gives large scale users the ability to track large numbers of simultaneous TCP streams. One important point to note is that all of the preprocessors must examine each packet in turn - there is no mechanism for a preprocessor to prematurely cut short packet processing and bypass the remaining ones. This means that if a large number of CPU-intensive preprocessors are loaded then it could have an adverse effect on overall performance. However, during our tests we found little evidence that enabling or disabling specific preprocessors had any significant impact on performance. Once the preprocessors are done with the packet (and none of them turned off the detection flag), the packet is passed on to the detection system. Output processing is performed via plug-in modules known as output processors. These decouple output from the main Snort engine, and allow Snort to support a wide number of logging and alerting capabilities, including logging to binary tcpdump files, human-readable ASCII text files, SQL databases, syslog, Winpopup messages, and so on. These can be selected via entries in the configuration file or at run-time with command line switches. The latest Snort distribution can be downloaded from www.snort.org, and the distribution file contains full instructions on the install procedure. Whilst initially daunting for the Linux/Unix novice, installation of Snort is actually not that difficult, following the usual configure, make, make install process. You do, however, require libpcap too, though most are likely to find it already present on their systems. Those using Linux are well served by the RPM format, which removes the need for compilation and makes installation even more straightforward. Windows users can download the installer from the Silicon Defense Web site (www.silicondefense.com). Although there is a default rule set and configuration file included with the standard distribution, it is wise to download the latest rule set to be found at www.snort.org. Documentation can be described as adequate at best, though it is improving over time. Often, the best documentation can be found as comments in source code and configuration files, and this can be a huge deterrent to those not well versed in the arcane world of Unix. Once installed, the administrator needs to spend some time perusing the main configuration file, snort.conf, to determine the best configuration for the various preprocessors, output processors, and rule sets. Snort is actually capable of running in one of three modes:
The different modes are determined by the command line switches used to fire up Snort. The default is to log packets in a human-readable hierarchical directory structure based on source IP address in /var/log/snort, and to log alerts (also in ASCII format) to /var/log/snort/alert. Most of the configuration of Snort is done in the snort.conf file an ASCII file that contains a number of reasonably well-commented sections, including variable definitions, preprocessors, output modules, log and alert settings, and which rule sets to include. Many of the configuration options within snort.conf can be overridden via command-line switches, making it easy to test various configuration alternatives without having to edit snort.conf each time. Conversely, there now appears to be an equivalent snort.conf keyword for each one of the many command-line options. The Snort configuration file provides the means to define parameters for all the available preprocessors. Sensible defaults are provided for each in the standard snort.conf file supplied with the snort.org distribution, but these can be amended to suit individual circumstances. Preprocessors available include:
Preprocessors are the key to the power and flexibility of Snort. The product is occasionally - and undeservedly - labelled as �low end� or dismissed as merely a �stateless pattern matcher�, but some of the more recently written and updated preprocessors provide a feature set that some commercial IDS� struggle to match. For example, the Back Orifice preprocessor has the ability to brute force every encrypted BO packet, rpc_decode, telnet_decode and http_decode perform extensive traffic normalisation on a range of ports and protocols, and frag2 and stream4 perform full IP fragmentation reassembly and stateful inspection/TCP stream reassembly. The combination of these - all of which are configurable - makes Snort extremely difficult to evade using current techniques. The signature-matching rules themselves are contained in a number of separate text files, each dedicated to a specific type of rule (such as Web, backdoor, DDOS, FTP, Telnet, DNS, and so on).
Each Snort rule contained within those files consists of two logical parts: the rule header and options, as in Figure 1. The rule header determines the protocol, traffic direction, and source and destination IP addresses and ports that apply to the rule. The action field specifies what Snort should do when a rule is matched against packet contents:
The second half of the Snort rule contains one or more option fields, providing tremendous flexibility and complexity. The most obvious option is the msg keyword, which prints a message in alert and packet logs describing the attack that was spotted. Numerous other keywords (all documented in the User Guide) provide the means to trigger on such things as packet contents, IP options, fragmentation bits, payload size, traffic flow direction and TCP flags, amongst others. These can be combined in many ways to detect and classify packets that are of special interest, and all options must be true in order to generate a match and trigger the rule action. Other options are provided to enhance documentation and readability of alerts and packet logs, including unique signature ID, rule classification (i.e. Denial of Service, Information Leak, Privilege gain, and so on), attack severity/priority, and references to external attack identification systems such as Bugtraq, CVE or Arachnids. Some options have special functions, such as the tag option, which specifies that an additional number of packets should be captured following the one that triggered the alert. There is also the react option, which enables Snort to close connections dynamically and perhaps even send a message to a user�s browser warning them against visiting certain prohibited Web sites. The options available in the current release are too numerous to detail here, but are all documented in the User Guide, and are being increased all the time. Unfortunately, the documentation does not always keep up with the rapid pace of development. The range of options has been increased significantly in the 2.0 release of Snort, as has the way they can be combined, providing much more scope to write complex signatures that delve deep into the contents of each packet. The rule language has been extended to support the ability to describe the number of bytes between content matches within a packet, allowing the language to describe anomalies in application layer protocols as well as imparting the concept of pattern ordering. What this provides effectively is the partial ability to model application layer protocols with Snort�s content matching system, from RPC to HTTP to IMAP. This is an extremely useful capability that was lacking in previous versions - for example it allows Snort to have signatures that alert if a particular number of bytes have been seen following a command character without finding a line termination, allowing new buffer overflow exploits to be detected without requiring a specific signature for them. The use of the flow keywords provides a much more efficient means of specifying traffic direction than the old flags option, too. The new ability to isolate the server side and client side of a TCP conversation helps reduce false positives tremendously, since odd high port signatures that were previously triggered by talking to Web servers may now have a new constraint added. This allows traffic from port 80 to be checked to see if it is client side or server side traffic, for example. These improvements in the signature writing �language� of Snort have gone a long way towards improving the signature recognition and resistance to false positives, two features that showed up well in this round of testing. Snort rules are not particularly complex or difficult to read, though that does not always mean they are easy to write! Luckily, there is a huge user community investing time and effort in the development and maintenance of Snort and its rule set. This means that you can usually find signatures for the latest attacks within days - or even hours - of those attacks becoming known by monitoring the right user groups or Web sites. Interestingly, the Snort rules �language� has also received further validation via the introduction of Snort-compliant rules parsing engines in commercial IDS products - the most notable being the Trons engine contained within the ISS RealSecure/Proventia product, and the stateful signatures in Symantec�s ManHunt product. As with most open source software, some effort is required on the part of the end user to makes sure everything is maintained correctly - essential processes such as signature updates certainly don�t happen at the click of a button in a nice, neat GUI the way they might in a commercial IDS offering. Note also that in the base product, signature updates must be applied carefully - and manually - to each sensor deployed in the organisation. Clearly some degree of centralised management may well be available via another open source offerings, but the onus to source, install, configure and integrate this is on the end user. As with the configuration and management, Snort�s default event handling capabilities are somewhat basic. Not that it does not have sufficient output options, it�s just that the actual analysis and presentation of the oft-times voluminous amounts of data are left to the administrator to sort out using third party products. Snort decouples the output from the main Snort engine providing much more flexibility. Snort output is essentially split into two - packet logs and alerts - and the output modules are run whenever the alert or logging subsystems of Snort are called, after the preprocessors and detection engine. Alerting can either be �fast� (one line per alert) or �full� (multiple lines of data including full packet headers) - or turned off altogether - and alerts can be written to numerous destinations such as ASCII log files, binary files, syslog, SQL databases, CSV files, Unix sockets, or Windows pop-up messages. There is currently no built-in e-mail alerting capability, though this can be achieved using third party tools. Each alert also generates a log of the same packet, whilst, conversely, a log rule will not automatically generate an alert. Packet logs - which record complete or abbreviated packet contents depending on the detail level chosen - can be written to the same range of output locations and, like alerts, can be disabled altogether if required. As with preprocessors, many output processor options set in snort.conf can be overridden via command line switches. Careful use of the log and alert rules and output processors can provide a wide range of options for the administrator to provide effective alerting and forensic data logging capabilities. Multiple output processors can be specified for each type of output (alert or log) if required, thus allowing alerts to be written to both database and syslog, whilst full packet logs are written to a binary dump file, for example. For those who want to customise the alerting and packet logging process, it is possible to define new rule types (in addition to log and alert) and associate one or more output plugins specifically to that type. For example, the administrator could create a new rule type called redalert which would cause an alert to be written to syslog and a packet log to be written to a MySQL database. This rule type could then be used in place of the log or alert action directives in any Snort rule. The default settings in the standard Snort distribution force both alerts and packet logs to be written to text files in /var/log/snort. Whilst alerts are written to a single file, packet logs are written to individual ASCII files contained within an extended hierarchical directory structure based on source IP address and alert type. This can make analysis of large amounts of packet log data somewhat difficult, and is why most administrators would probably choose to log to a SQL database. As you can imagine, writing large numbers of ASCII files into a large number of subdirectories is likely to be very slow, and so Snort also provides a binary logging option. Here, packet logs are written to a tcpdump binary file for speed, which can be subsequently run through a separate Snort engine off-line in order to recreate the ASCII-based packet log directory structure for further analysis. Unified logging can also be used to completely decouple Snort from the onerous task of updating databases directly. The optimised logging utility Barnyard can be used to take the binary log file and write alert records to a database in close to real time, but without impacting on Snort�s detection performance. There are no reporting or analysis tools built in to the basic Snort distribution. For those for whom handling the standard Snort output proves too cumbersome, various third party tools � such as ACID or Snortsnarf � are available to provide a more extensive and user-friendly means of analysis. Some of these third party tools take considerable effort and knowledge to install, configure and maintain - a typical problem with open source software - and are beyond the scope of this report. Despite being pigeonholed as a �tool for small, lightly utilised networks�, we found that Snort�s performance and basic feature set is good enough to allow it to be deployed in large corporate networks. There are a large number of signatures available - many of which have been overhauled and improved in recent months after the release of version 2.0 - and it is not particularly difficult to develop your own. There has been considerable effort put into streamlining the signature set, weeding out duplicates and false positives, assigning fixed ranges of signature IDs (SIDs) for standard and custom signatures, and providing an on-line reference database of signature and exploit data to make forensic analysis that much easier. Snort demonstrated very good attack recognition capabilities in our tests with the standard rule set - a huge improvement since our last round of testing - and it would not take much customisation to improve this even further. We also noted good anti-IDS evasion capabilities (though TCP segment reassembly performance is not quite so good in this release) and improved resistance to false positives. On the down side, management of one or more Snort sensors is rarely going to be as straightforward as it is for a typical commercial IDS product, which will usually be far simpler to deploy and manage out of the box. Reporting and analysis, too, is a serious issue with the basic product. Snort does an excellent job of detecting and recording suspicious packets, but the subsequent means of transmitting the alerts to the administrator and analysing the data recorded is left entirely to the end user. True, there are various open source offerings available to provide the missing management and reporting capabilities, and it is nice to see that these are getting easier to obtain and install, with several sources offering almost �one click� install routines for many of these packages. Where that is not available, there are several good guides to be found on deploying the most common third party tools on the most popular operating systems. On its own Snort is a fairly raw tool, and requires quite a supporting cast in order to have everything managed, reporting and alerting the way you want. It is essentially a �roll your own� solution, which is likely to be beyond the capabilities of those organisations which do not have a reasonable level of in-house expertise in the black arts of Unix/Linux. But, for anyone who has that expertise - or who is happy to pay to acquire it from someone else - the overall Snort experience is likely to be a positive one. For those who like the idea of Snort but are wary of the DIY approach, there are now several commercial appliance offerings that bundle all the necessary software - including front-end management and back-end reporting capabilities - with the appropriate hardware. However you choose to implement it, the open nature of the product makes it one of the most flexible IDS offering we have looked at so far. With the latest release installed on an appropriately powerful hardware platform, Snort will handle any fully-loaded 100Mbps network with ease. Company
name:
Snort.org Click here
to return to the Snort questionnaire |
Security Testing |
Send mail to webmaster
with questions or
|
