 |
|
|
Gigabit Intrusion Detection
Systems (IDS) January 2004 Click here to view the latest Gigabit Intrusion Detection Systems (IDS) test report in full on-line
Introduction Whenever a company connects its network to the Internet, it opens up a whole can of worms regarding security. As the network grows, it will play host to numerous bugs and security loop holes of which you have never heard - but you can bet intruders have. Many organisations are recognising the value of a good security policy to define what is and is not allowed in terms of network and Internet access. Then they deploy a number of tools to enforce that security policy � usually in the form of a firewall or two. Firewalls may be billed as commodity items, but the �shrink wrap� element certainly doesn�t extend to their configuration. A detailed knowledge of what a hacker can do and what should and shouldn�t be allowed through the firewall is required before embarking on the configuration adventure, and a slip of the mouse is all it takes to open up a hole big enough for your average hacker to drive the proverbial bus through. The problem is, a badly configured firewall can be worse than no firewall at all, since it will engender a false sense of security. To protect an organisation completely, therefore, it is necessary to provide a second line of defence, and in order to achieve this, an entire category of software exists in the form of Intrusion Detection Systems (IDS). When it comes to computer and network security, there are a number of analogies that can be drawn with the �real world�. Such analogies are particularly useful for answering such questions as �I already have a firewall, why do I need Intrusion Detection Systems as well?�. Depending on how you approach the security of your home, for example, you may opt for high quality locks on your doors and windows. That will help to keep intruders out, and could be thought of as the equivalent of the firewall � perimeter defences. It�s nice to feel secure, but the determined burglar can often find ways around these measures. He can always throw a brick through your back window, for instance, and get in that way � or perhaps you simply forget to lock your door one day. Once he is inside your home he is free to wreak havoc, perhaps making it obvious he has been there by stealing or wrecking things, or perhaps simply taking copies of any keys he finds so he can come and go later at his leisure. Whatever happens, you don�t want your first knowledge of the break-in to be when you return home to the ransacked contents. That is why many people install a burglar alarm as well. Should the intruder gain access through the perimeter defences, the burglar alarm alerts you or your neighbours to the break in immediately, and provides an additional deterrent to the would-be thieves. IDS, therefore, are the equivalent of the burglar alarm. To be used alongside firewalls, they are a recognition of the fact that you can never have a 100 per cent secure system. However, should someone be clever enough to breach your perimeter defences, you want to know about it as soon as possible. It would also be nice to know what they have been up to while they were inside too. Intrusion Detection and Vulnerability Assessment are becoming increasingly important as the stakes become higher. In the 1980s and early 1990s, denial-of-service (DoS) attacks were infrequent and not considered serious. Today, successful DoS attacks can shut down e-commerce-based organisations like online stockbrokers and retail sites. Within the IDS market place there are four broad categories of product: This category, so often overlooked recently in the face of the more �interesting� technology of Network IDS, is seeing something of a resurgence. This could be due partly to the fact that high speed switched networks are providing a significant obstacle to effective Network IDS implementation, or it could also be that there is a growing realisation that there is more to IDS than detecting suspicious packets on the wire. This type of product can itself be split into two main categories, with some host-based systems providing elements of both: �Traditional� Host IDS (HIDS) Host IDS (HIDS) products employ an agent that resides on each host to be monitored. The agent scrutinises event/system logs, kernel logs, critical system files and other auditable resources looking for unauthorised changes or suspicious patterns of activity. Whenever anything out of the ordinary is noticed, alerts or SNMP traps are raised automatically. For instance, a HIDS will monitor the Registry for unauthorised access, kernel logs to detect when inappropriate processes are initiated, or logins to take note of when an attempt is made to access an account with an incorrect password. If a login attempt fails too many times within a short time span the system may conclude that someone is trying to gain access illegally and an alarm can be raised. Traditional HIDS are very good at detecting insider threats and usually provide extensive damage assessment and data forensics. Bear in mind that the term �insider� does not always refer purely to your own employees. It is possible, for example, for an attacker to gain access to internal systems via a legitimate user name and account combination without having to run any exploit that is detectable by a Network IDS product � perhaps gaining access via social engineering. At that point, the attacker would have all the rights and privileges associated with that user, and is much harder to detect. Disadvantages of the HIDS approach are the need for agent deployment on key systems, and the requirement for close attention to audit policy. They can often be the most difficult of all Intrusion Detection Systems to configure. File Integrity Assessment (FIA) File Integrity Assessment (FIA) products monitor the state of system and application files, or the Registry. They do this by making an initial pass of a clean system and storing a condensed �snapshot� of how that system should look, usually in the form of cryptographic �hashes� of the monitored objects. Once this has been done, it is impossible to tamper with either the original objects or the hash values without invalidating the checksum files. At regular intervals, the FIA product makes a fresh pass, recalculating the checksum values and comparing them against those stored previously. Thus if an intruder - or Trojan Horse - does manage to gain access to the system and make changes to key files, the FIA product will detect this and raise an alert. This makes FIA the perfect technology for assessing the true extent of the damage inflicted by a successful attack. The downside, of course, is that because the scans are periodic rather than real time, its strength is in forensic analysis after an attack has been perpetrated, and thus it is of little use where real-time alerting is required. The Network IDS (NIDS) monitors traffic on the wire in real time, examining packets in detail in order to detect patterns of misuse � perhaps spotting denial of service attacks or dangerous payload � before the packets reach their destination and do the damage. They do this by matching one or more packets against a database of known �attack signatures�, or performing protocol decodes to detect anomalies, or both. These signature databases are updated regularly by the vendors as new attacks are discovered. When suspicious activity is noticed, a network based IDS is capable of both raising alerts and terminating the offending connection immediately (as are some host-based IDS). Some will also integrate with your firewall, automatically defining new rules to shut out the attacker in future. Most of the network-based IDS available to date work in what is known as �promiscuous mode�. This means that they examine every packet on the local segment, whether or not those packets are destined for the IDS machine (much like a network monitor, such as Sniffer). Given that they have a lot of work to do in examining every single packet and tracking active sessions, they usually require a dedicated host on which to run due to their heavy use of system resources. For instance, most attacks are not based on the contents of a single packet, but are made up of several, sometimes sent over a lengthy period of time. This means that the IDS has to store a number of packets in an internal buffer in order to track established sessions and compare groups of packets with its attack signature database. This is known as �maintaining state�, and allows IDS to compare new packets against its signature database in the context of what has happened previously in a particular session, rather than examining every packet in isolation. You will also need one Network IDS sensor per segment, since they are unable to see across switches or routers, and some have problems keeping up with heavily loaded Fast Ethernet segments (never mind Gigabit). Clearly, they would also have problems with encrypted traffic. The Network Node IDS (NNIDS) is a type of �hybrid� IDS agent which overcomes some of the limitations of the network-based IDS. The NNIDS agent works in a similar manner to the network-based IDS in that it takes network packets and performs protocol analysis and/or compares them against signature database entries. However, this �micro agent� is only concerned with packets targeted at the network node on which it resides. Because it is installed within the protocol stack of the host, it is sometimes referred to as a Stack-based IDS. Rather confusingly, it is also occasionally referred to as �host-based�, but usually only by those who are looking at the industry purely from a Network IDS viewpoint. For the purposes of this report, Host IDS is concerned with monitoring of log files and behavioural analysis, whereas both Network and Network Node IDS are concerned with TCP analysis � the only difference is that one (NIDS) is running in promiscuous mode whilst the other (NNIDS) is not. The fact that the NNIDS system is no longer expected to examine every single packet on the wire, however, means that it can be much faster and take less system resources, and this allows it to be installed on existing servers without imposing too much overhead. It also makes it particularly suitable for heavily loaded segments, switched network environments, or VPN implementations with encrypted traffic on the wire � all areas where traditional network-based IDS� have problems. Obviously it is necessary to install a number of these NNIDS agents � one for every server to be protected � and they will all have to report back to a central console. Many organisations may opt for a combination of the two � NNIDS on individual servers in switched server farms, and traditional NIDS on less heavily used segments, where a single IDS can protect a large number of hosts. Intrusion Prevention Systems (IPS) Most IDS systems tend to be reactive rather than proactive � that is they often have to wait until something has actually happened before they can raise the alarm. The Intrusion Prevention System (IPS), however, attempts to be proactive, and is designed to stop intrusions dead, blocking the offending traffic before it does any damage rather than simply raising an alert as, or after, the malicious payload has been delivered. Of course, the downside with this approach is the potential for introducing a self-inflicted Denial of Service condition. The thorny issue of false positives is one that has plagued the IDS industry to date - sometimes it is very difficult to design an attack signature that will alert reliably on every variation of the exploit whilst ensuring that it will not be triggered accidentally by valid traffic. This is bad enough when you are just being pestered by false alarms at your IDS console, but when the IPS system cuts off a potential customer or your CEO from a vital computer system, the consequences can be far more serious. As with Host IDS systems, the Host IPS relies on agents installed directly on the system being protected. It binds closely with the operating system kernel and services, monitoring and intercepting system calls to the kernel or APIs in order to prevent attacks as well as log them. It may also monitor data streams and the environment specific to a particular application (file locations and Registry settings for a Web server, for example) in order to protect that application from generic attacks for which no �signature� yet exists. One potential disadvantage with this approach is that, given the necessarily tight integration with the host operating system, future OS upgrades could cause problems. The Network IPS combines features of a standard IDS, an IPS and a firewall, and is sometimes known as an In-line IDS or Gateway IDS (GIDS). As with a typical firewall, the NIPS has two network interfaces, one designated as internal and one as external. As packets appear at the internal interface they are passed to the detection engine, at which point the IDS device functions much as any IDS would in determining whether or not the packet being examined poses a threat. However, if it should detect a malicious packet, in addition to raising an alert, it will discard the packet and mark that flow as bad. As the remaining packets that make up that flow arrive at the IPS device, they are discarded immediately. Legitimate packets are passed through to the internal interface and on to their intended destination. A useful side effect of some NIPS products is that as a matter of course - in fact as part of the initial detection process - they will provide �packet scrubbing� functionality to remove protocol inconsistencies resulting from varying interpretations of the TCP/IP specification (or intentional packet manipulation). Thus any fragmented packets or packets with obfuscated URLs will be �cleaned up� before being passed to the destination host. Seems too good to be true, doesn�t it? Why bother with an IDS at all, since the NIPS device does everything an IDS can do as well as provide true prevention capabilities? Well, when something seems to good to be true, it usually is! Firstly, there is the aforementioned potential for an in-line device to introduce a Denial of Service condition by accidentally blocking a legitimate packet flow. If the NIPS is at the gateway to your DMZ containing your e-commerce servers, you had better be sure that none of your attack signatures are susceptible to false positives. For this reason, most adopters of this technology would probably run the NIPS with a reduced signature set that contained only those signatures that had a very low or zero propensity for triggering accidentally on benign traffic. They would then run a standard IDS product on the protected network behind the NIPS which utilised a more complete signature set in order to deal with the alerts that were raised by traffic not covered by the NIPS itself. The second potential problem is that given the amount of work this device has to do, it can act as a serious bottleneck when installed at the gateway to your network. Most NIPS vendors have recognised this and have sought to get around it by using custom hardware, populated with expensive FPGAs and ASICs. At the time of writing, therefore, adopters of this technology tend to require deeper pockets than those buying plain old NIDS products. One thing to watch out for - don�t let the �reactive� IDS vendors kid you into believing that they have intrusion prevention capabilities just because they can send TCP reset commands when they detect an attack (a worrying piece of FUD that we have noticed in some IDS marketing literature recently). That does not count as prevention because by the time the IDS has detected it, the attack payload may already been delivered. Certainly the initial packet that caused the alert will have reached its target, and if the payload is contained in a single packet�. game over! On high speed networks, it would also be possible to deliver an entire payload spread across many packets before the TCP reset command took effect. Only an in-line device (or Host IPS, of course) is capable of real prevention. Clearly, host-based IDS in their various forms are not (or should not be) affected by the speed of the network on which they are installed. Therefore whenever we talk about Gigabit IDS we are, by definition, focussing on Network IDS with a Gigabit capability. Where life gets difficult for those tasked with evaluating this technology is that different vendors have different ideas about what constitutes Gigabit IDS. Some products will be true Gigabit products, capable of pulling traffic off the wire for analysis at speeds of up to 1000Mbps (or beyond). Others are merely appliances that contain a Gigabit network card, whose main aim is to allow them to cope with 100Mbps or multiple 100Mbps segments easily. There is nothing inherently wrong with the latter approach providing the marketing message is honest and does not describe the product as a true Gigabit appliance. As long as all the customer needs is to be able to handle 100-200Mbps with confidence - and the price is right, of course - then this is a perfectly valid tactic. Even true wire-speed Gigabit appliances will have problems in certain areas if they are assembled from off-the-shelf components. At the time of writing, not even the best Gigabit network cards on the market are capable of pulling almost 1.5 million packets per second off the wire, never mind analysing that level of traffic. Thus a Gigabit network loaded with small packets (64 bytes) will cause problems for most Gigabit solutions, and the only way around that for the time being is to move towards custom hardware and ASICs. Administrators need to be aware of the overall performance limitations of any device when deploying on Gigabit networks. As with most Fast Ethernet networks, the average Gigabit subnet is unlikely to see much more than a fraction of its total available bandwidth in use at any given point in time, and so where only 200-400Mbps is being used, the performance of the Gigabit IDS used to monitor it is less of an issue. However, one tactic being employed in some organisations is to consolidate multiple 100Mbit segments using a Gigabit switch, and copy all the traffic from each segment to a single mirror, or SPAN (Switched Port ANalyser) port. The Gigabit IDS sensor is then attached to this port to monitor all of the traffic across multiple subnets, thus providing a cost-effective solution to monitoring a number of subnets using a single sensor. Of course, even if the average utilisation of each subnet is only 40-50Mbps, once you mirror 20 of these you are asking your IDS sensor to monitor getting on for a full Gigabit of network traffic (providing your switch is actually capable of mirroring that amount of traffic, of course - an entire topic in itself which is beyond the scope of this report). This is when the performance limitations of some so-called Gigabit devices will begin to manifest themselves. In some respects, detection performance is the least of the problems facing the administrator tasked with deploying these devices. The problem with any Gigabit IDS product is, by its very nature and capabilities, the amount of alert data it is likely to generate. With 1Gbps of traffic passing through the IDS, the number of alerts could reasonably be expected to be ten times that generated by the typical 100Mbps product. How many members of staff would be needed to process, investigate and resolve that number of alerts? How long before the IDS becomes just another device in the corner that is largely ignored thanks to its insistence on overloading the administrator on a daily basis? More than ever before in the IDS space, centralised management reporting and forensic analysis is key to the success of the Gigabit IDS appliance. A reduction in false positives (through more accurate protocol decodes and signatures) and global pre-filtering of �safe� alerts that can be ignored are both essential. After that comes the ability to consolidate alerts from multiple sensors to a single management console. Far from increasing the load for a single administrator, this consolidation should help in identifying where potential break-ins are disguised by multiple exploits run over multiple subnets and over a long period of time. Some management solutions will leave the administrator to determine the connection between exploits, providing the tools to �slice and dice� the data in a myriad of different ways in order to achieve this, whilst others will attempt to perform the correlation in an automated fashion (with varying degrees of success). Either way, the ability to create high-level reports of activity over a period of time, reshuffle and resort alerts in different ways, and then drill down to discover the exact trigger that caused the alert in each case in an efficient manner is now essential. Without effective management capabilities, alert handling, and forensic analysis tools, the Gigabit IDS is just another lump of iron sitting in the machine room equipment rack. Unfortunately for the potential purchaser of security products, there is no one answer to this question. The first myth to quash is the one which states that if you have an IDS you do not need a firewall - or vice versa. For those with a reasonable security budget, we would recommend purchasing both a firewall and at least one product from each of the aforementioned IDS categories (note that we still include IPS products in the more generic IDS group, since in order to protect, they first have to detect). The firewall guards your perimeter, whilst the IDS� monitor what is happening on your network, guarding against slip-ups by the firewall as well as internal mischief-makers. IDS devices can also be installed outside a firewall in order to detect the �doorknob rattlers�, attempted scans and other probes and attacks that are normally dealt with by your firewall and thus would not be detected by an internal IDS. This is purely a management issue, and depends on the human resources available to scan the resulting log files � those from an IDS installed outside a firewall are likely to be voluminous to say the least. Both host-based and network-based IDS are worth investing in, since they each have their own strengths. Network-based IDS will monitor the wire for suspect packets and are adept at spotting Denial of Service type attacks and unwelcome probes � usually from outside the network. Host-based systems, on the other hand, are watching the �crown jewels� � the actual data on the file servers, monitoring for inappropriate logins or changes to critical files from unauthorised sources. Although network-based products seem to get most of the publicity at the moment, the host-based system can sometimes be more valuable in determining the after-effects of an attack. We would still recommend deploying both technologies wherever possible. Intrusion Prevention products are also well worth considering, of course, whether host- or network-based. If the budget will stretch to their acquisition and the signature set can be carefully tuned to avoid false positives, they are a valuable addition to the armoury. So, you have been out and bought your shiny new IDS system and installed it somewhere on your network. You are now secure and immune from attack, right? Wrong! To begin with, deployment of IDS requires careful thought and planning if you are to get the most from it. What kind of resources are you protecting? If you have a DMZ containing only Unix-based Web servers, you could disable all IDS signatures to do with Windows NT or DNS servers. Some IDS products will go so far as to try and perform this step automatically for you depending on what operating systems and services are found during a network scan. To do the job properly, however, you should acquire some good Vulnerability Assessment tools and scan your own network (see the NSS Vulnerability Assessment Group Test report). This should provide a good starting point in determining which machines need protecting, and from what sort of attacks. Are you worried about intruders breaking through your firewall and launching DoS attacks against your machines? Then perhaps a Network or Network Node IDS would be a good buy. Or are you more concerned about Trojans on your desktops and file servers? File Integrity Assessment could be the best bet. Or perhaps you are worried about your own users making off with company secrets? In which case, Host IDS would be the right choice. If attacks against your Web servers are your chief concern, however, Intrusion Prevention Systems might be more appropriate. Most likely you will actually need a combination of all these technologies, and there is some considerable overlap in the host-based products, with the Host Intrusion Prevention Systems providing many of the capabilities of the more �traditional� Host IDS. Having selected your products, where are you going to install them? Host IDS, Network Node IDS, and Host IPS are simple - you put them on the hosts that need protecting. But Network IDS is another matter. Placing a network interface card into promiscuous mode generates an awful lot of material to be processed. All the traffic passing by on the subnet being monitored has to be collected and examined by the IDS sensor and compared against a database of known attack signatures, or analysed for protocol violations. Nor is it enough to simply look at this traffic on a packet by packet basis, since some attacks are spread over several packets, or may be fragmented deliberately to confuse the IDS. This necessitates a time- and memory-consuming process of buffering packets and maintaining state tables to keep track of individual sessions. To defeat the common IDS evasion techniques such as packet fragmentation, out-of-order fragments and so on, it is also necessary to perform some heavy-duty processing within the IDS itself to reassemble packets back into the form that will eventually be seen by the target host (or capture the packets at a higher level in the TCP stack of the sensor machine). Or perhaps a full protocol decode is performed. All of these options are likely to slow the detection process, but only then can the true packet contents be determined and compared against the signature database. And all this must be done whilst eliminating � or at least keeping to a minimum � the risk of the �false positive�. In an ambiguous situation, there is no point in raising an alert �just to be on the safe side�, otherwise the IDS runs the risk of crying wolf once too often and eventually being ignored altogether. In the case of the Network IPS, of course, the false positive has even more serious ramifications, potentially introducing a Denial of Service condition as it blocks a legitimate packet flow that has been misidentified. Quite a bit is involved behind the scenes then, and whilst most IDS products could quite happily keep up with traffic on a 10Mbit network, leased-line or DSL connection, it is much more difficult to do all this at wire speed on a 100Mbit network. Gigabit presents even more problems. Not only are we now looking at a significant increase in bandwidth � and thus a significant increase in the volume of traffic to be analysed � but we have also moved into the realms of the purely switched network. Don�t forget that the promiscuous mode sensor can only see traffic on its own segment, and in a switched environment, every connection to the switch is, effectively, a single segment. In the 10Mbps or 100Mbps world, this can be overcome by the use of network taps or mirroring all the switch traffic to a SPAN (Switched Port ANalyser) port, to which is attached an IDS sensor. But with Gigabit, the result would be a seriously overloaded sensor. Building IDS technology into the switch hardware itself, allowing the sensor to grab traffic directly from the back plane, may be one solution. Another is to move to a pure Network Node IDS implementation, where the agents are concerned only with the traffic directed at the host on which they are installed. A third option would be to invest in one of the more recent Gigabit IDS products that make use of improved high-performance hardware and software architectures to perform analysis and detection at something approaching wire speed on Gigabit networks. For those who need to go even further, solutions incorporating custom ASICs and FPGAs provide the highest levels of performance, making it feasible to detect - and even prevent - intrusions on multi-segment multi-Gigabit networks with a single device� at a price. One final point to consider is that of encrypted traffic. Companies that make extensive use of VPN�s will find problems with Network IDS, since the traffic picked from the wire will obviously be encrypted, thus rendering any pattern matching or protocol analysis completely useless. The obvious choice would be to install a VPN tunnel termination device at the network border, which would then forward the data to the destination device in plain text. If it is not acceptable to decrypt data at the network border, however, then once again the only solution is to install Network Node IDS, so that the IDS agent has access to the data stream as it is decrypted at the VPN end-point. At one time, most Network IDS products based their alerts purely on pattern matching packet contents against a database of known signatures. Then came a new breed of IDS offerings that approached the problem in a completely different way - by doing a full protocol analysis on the data stream. Others began to use heuristics or anomaly-based analysis to determine when an attempted attack had taken place. Today, most IDS employ a mixture of these detection methods in a single product, though some will be more biased towards one method than another. According to Cisco, there are five main methods of attack identification (source: Cisco Systems, The Science of Intrusion Detection System Attack Identification): Pattern matching in its most basic form is concerned with the identification of a fixed sequence of bytes in a single packet. In addition to the tell-tale byte sequence, most IDS will also match various combinations of the source and destination IP address or network, source and destination port or service, and the protocol. It is also often possible to tune the signature further by specifying a start and end point for inspection within the packet, or a particular combination of TCP flags. The more specific these parameters can be, the less inspection needs to be carried out against each packet on the wire. However, this approach can make it more difficult for systems to deal with protocols that do not live on well defined ports and, in particular, Trojans, and their associated traffic, which can usually be moved at will. Although it is often quite simple to define a signature for a particular exploit, basic pattern matching can often be too specific, sometimes requiring multiple signatures to be defined for minor variations in exploits. They are also prone to false positives, since legitimate traffic can often contain the relatively small set of criteria supposedly used to determine when an attack is taking place. This method is usually limited to inspection of a single packet and, therefore, does not apply well to the stream-based nature of network traffic such as HTTP sessions. This limitation gives rise to easily implemented evasion techniques. Stateful pattern matching offers a slightly more sophisticated approach, since it takes the context of the established session into account, rather than basing its analysis on a single packet. Stateful IDS products must consider arrival order of packets in a TCP stream and should handle matching patterns across packet boundaries. Thus, if the exploit string to be matched is foobar, and the exploit is split across two packets, with foo in one and bar in another, the simple packet matching IDS will miss the attack, since it will not be able to match the complete string. The stateful IDS, however, will maintain the session context and reassemble the traffic stream, once again making the complete string available to the detection engine. This requires more resources than simple pattern matching, since the IDS now has to allocate large amounts of memory and processing power to track a potentially large number of open sessions for as long as possible. This approach does make IDS evasion that much more difficult, though far from impossible. Protocol decode IDS take a radically different approach to simple pattern matching IDS products � though sometimes not quite as radically different as the marketing folks would have you believe. With this technique, the IDS detection engine performs a full protocol analysis, decoding and processing the packet contents in the same way that the target client or server application would. It also tends to be stateful. Although this may seem like using a sledgehammer to crack a nut, it does have the advantage of highlighting anomalies in packet contents much more quickly than doing an exhaustive search of a signature database. It also has the advantage of greater flexibility in capturing attacks that would be very difficult � if not impossible � to catch using pure pattern-matching techniques, as well as new variations of old attacks. These are attacks which � although changing only slightly from variant to variant � would normally require a new signature in the database for the �traditional� IDS architecture, but which would be detected automatically by a complete protocol analysis. One of the first things the protocol decode engine does is to apply rules defined by the appropriate RFCs to look for violations. This can help to detect certain anomalies such as binary data in an HTTP request, or a suspiciously long piece of data where it should not be � a sign of a possible buffer overflow attempt. One simple example of how this might work concerns searching Telnet login strings for one of the many well-known login names that rootkits tend to leave behind on the system. A pattern matching system might scan all Telnet traffic for all these patterns, in which case the more patterns you add, the slower it becomes (not always the case, but a reasonable assumption for the purposes of this example). In contrast, a protocol analysis system will decode the Telnet protocol and extract the login name. It can then perform an efficient search in a binary-search tree or a hash table for that login name, which should scale much better as new signatures are added. In theory, therefore, protocol decoding should offer more efficient processing of traffic and improved scalability as more signatures are added, compared to a pure pattern matching solution. In reality, pattern matching solutions rarely opt for a �brute force� approach, and so the differences are not always as marked as the marketing people would like us to believe. Note also, that pattern matching and protocol decoding are not mutually exclusive, as some would lead you to believe. A protocol analysis IDS can only go so far with its protocol decodes before it too will be forced to perform some kind of pattern matching, albeit against a theoretically smaller subset of �signatures�. One major downside, of course, is that if a completely new type of exploit does surface, it is likely that the IDS developer will have to create new code to handle it, whereas the pattern matching approach can allow the administrator to develop a custom signature much more quickly on site. Protocol decoding does offer a number of advantages, however. It minimises the chance for false positives if the protocol is well defined and enforced (although false positives can be higher if the RFC is ambiguous), and can also be more broad and general to allow the IDS to detect minor variations of an exploit without having to implement separate signatures. You may see this technique referred to in several different ways:
Each of these terms, if strictly applied, could use a slightly different approach to the problem. For example, we would expect a protocol decode engine to perform the sort of additional pattern matching and length checking mentioned above on the field contents in order to detect specific exploits or buffer overflows. Pure protocol validation or Protocol Anomaly Detection engines, however, might go no further than decoding just enough to be able to determine if the packet follows the RFC to the letter. If not, they will raise an alert - but in allowing a packet to pass, they cannot be sure that the contents will not contain a means of exploit that just happens to conform with the RFC. Beware the marketing hype in this particular area � no matter what architecture is used, the performance figures and detection rates in a live deployment will speak for themselves. Heuristic-based signatures use some kind of algorithmic logic on which to base their alarm decisions. These algorithms are often statistical evaluations of the type of traffic being presented. A good example of this type of signature is one that would be used to detect a port sweep. This signature looks for the presence of a threshold number of unique ports being touched on a particular machine. The signature may further restrict itself through the specification of the types of packets that it is interested in (that is, SYN packets). Additionally, there may be a requirement that all the probes must originate from a single source, and even that valid SYN ACK packets must be seen to be returned by the host being probed. Signatures of this type will react differently on different networks, and can be a significant source of false positives if not tuned correctly, requiring some threshold manipulations to make them conform to the utilisation patterns on the network they are monitoring. This type of signature may be used to look for very complex relationships as well as the simple statistical example given. The final approach is to forget about trying to identify the attacks directly, and concentrate instead on ignoring everything that is considered �normal�. This is known as �anomaly-based� IDS, and the basic principle is that, having identified what could be considered �normal� traffic on a network, then anything that falls outside those bounds could be considered an �intrusion� � or at the very least, something worthy of note. The primary strength of anomaly detection is its ability to recognise previously unseen attacks, since it is no longer concerned with knowing what an attack looks like � merely with knowing what does not constitute normal traffic. Its drawbacks, of course, include the necessity of training the system to separate noise from natural changes in normal network traffic (the installation of a new � perfectly legitimate - application somewhere on the network, for example). Changes in standard operations may cause false alarms while intrusive activities that appear to be normal may cause missed detections. It is also difficult for these systems to name types of attacks, and this technology has a long way to go before it could be considered ready for �prime time�. Which Detection Method Is The Best? Which detection method to choose is a difficult question, and in all honesty, it is not one with which most of those evaluating these products should concern themselves. Adequate performance to handle the traffic to which the sensor will be exposed, accuracy of alerts, low incidence of false positives, and centralised management and reporting/analysis tools are far more important than how the packets are processed. In some instances, the lines blur between methodologies to the point where they become almost indistinguishable. For example, most protocol decode analysis engines alert the user to the presence of protocol violations that are not directly related to any known attack but are �anomalous� (for example, length-based buffer overflow detection). Therefore, in this instance the engine has attributes of an anomaly-based system. As we have already mentioned, most protocol analysis systems are also reduced to performing some form of pattern-matching process following the protocol decode. Likewise, even the most basic pattern-matching systems perform some form of protocol analysis � even if it is only for a limited range of protocols. In truth, most Network IDS systems are already adopting a hybrid architecture in some way, and we foresee a time in the not too distant future when most NIDS will employ fully integrated pattern-matching, protocol decode, heuristic and anomaly detection engines in a single product. By and large, therefore, the pattern-matching vs. protocol decode debate is one of religion � something for the marketing departments to shout about until the aforementioned hybrid products become commonplace. Why should the average user care what happens under the hood as long as the product does what it claims to do � detect intrusions? Monitor-Evaluate-Modify: The Security Cycle Once you have your IDS deployed and working effectively, that is not the time to sit back and relax. In fact it is only the beginning of a cycle that must be constantly repeated if security is to be maintained. IDS is a valuable component of an organisation�s security plan, but it is just that � a component. The first point of defence may well be the firewall, and behind the Network and Network Node IDS system may well be additional port monitoring and File Integrity Assessment products to alert you as to when an intrusion attempt has been successful. All of these components must operate within the confines of a strict security policy, which should determine what is and is not allowed on the corporate network. This, in turn, will help specify how the individual components are to be deployed and configured, as well as offer guidelines as to how alerts are to be handled. There is no point in using the latest IDS technology only to have it log intrusion attempts to a file that is only examined once a month. There should be differing levels of importance assigned to different types of intrusion attempts, and the alert and response procedure should be scaled accordingly. There is little point in raising the roof should you discover your network is being port scanned by someone using nmap � that happens all the time, and it is enough to log that for periodic examination just to make sure it is not happening too often within a set time interval. On the other hand, a successful BackOrifice ping that elicits a reply from somewhere within your organisation indicates a serious compromise, and for that you may deem it appropriate to page the security administrator any time day or night. Between those two extremes are various other possible responses such as e-mail, SNMP alerts, session termination, firewall reconfiguration, and so on. Use them to the full, and make sure that your security staff take the time to examine the various log files at regular intervals to keep tabs on the more mundane intrusion attempts. And, of course, there is the constant battle against false positives. An IDS can only alert on what it sees compared with what it thinks is �bad�. A particular binary file on your network might, coincidentally, contain the exact pattern of bytes used in a particular shell code exploit. Or perhaps you actually have a file called cgi-bin/test-cgi and it is in common use. Both of these situations would cause numerous alerts to be logged on a daily basis by the IDS causing frustration for the administrator. One approach is to simply turn off the signatures that cause the alert, refining the security policy and reducing the number of false positives. Of course, this could have wider reaching consequences, in that should an attacker attempt to launch that shell code exploit or scan for your test-cgi file, the attempts will no longer be logged. It is far better to tune the signatures instead, if at all possible. This is dependent both on your chosen IDS product allowing the definition of custom signatures and your ability to research the exploit thoroughly enough to produce a sound signature. Is there, perhaps, a longer binary pattern to identify that shell code exploit that will not be triggered by your legitimate application? And could you simply rename that test-cgi file to something else? Intrusion Detection Systems are good at sounding alarms, but unless there is someone around who is prepared � and trained � to respond (even if it is only to determine that the alert is a false positive), it is no better than a car alarm that everyone ignores. An effective response is every bit as important as detecting the attack in the first place. Maintenance is equally important. Security is certainly not static, and new vulnerabilities are being discovered and exploited all the time. This should result in new signatures for your IDS and VA tools, patches for your operating systems and updates for your firewalls. Even so, it is a wise security administrator that keeps an eye on the various underground hacking sites and security alert mailing lists for himself. Between the point of discovery to the point where a patch is issued and applied, there is a window of opportunity for the hacker to exploit. It is up to the security administrator to minimise this window as much as possible. If an OS vendor is slow in bringing out a patch for a new vulnerability, for example, perhaps the administrator can reconfigure the corporate firewall to eliminate the sort of traffic that could exploit that vulnerability, or add a temporary custom signature to the IDS in order to detect it. Certainly as soon as new IDS signatures are made available from the vendor, they should be downloaded and deployed to every appropriate sensor on the network. Finally, the administrator should be using Vulnerability Assessment tools on a regular basis (or employing outside consultants to do the same) in order to continually test defences and update security policies accordingly. A VA scan may well highlight additions or changes to the network and its applications which might necessitate a rethink on how IDS sensors are deployed and which signatures are monitored by each sensor. Monitor � evaluate � modify. Then back to the beginning. It is a cycle that must be repeated over and over if you want to keep your network as secure as possible. Only by continual vigilance and refinement will you stay one step ahead of (or at least no more than one step behind) the hackers. The NSS Gigabit IDS Group Test The NSS Group has conducted the first comprehensive Gigabit IDS test of its kind. This exhaustive review will give readers a complete perspective of the capabilities, maturity and suitability of the products tested for their particular needs. As part of its extensive IDS test methodology The NSS Group subjects each product to a brutal battery of tests that verify the stability and performance of each IDS tested, and determine the accuracy of its security coverage. If a particular IDS has been designated as NSS Approved, customers can be confident that the device will detect a wide range of exploits at the maximum bandwidth claimed by the vendor without dropping packets and thus missing attacks. To assess the complex matrix of IDS performance and security requirements, the NSS Group has developed a specialised lab environment that is able to exercise every facet of an IDS product. The test suite contains over 500 individual tests that evaluate IDS products in three main areas: performance and reliability, security accuracy, and usability. This thorough review should give readers a complete perspective of the capabilities, maturity and suitability of the products tested for their particular needs. Click here to see the view the full report on line |
Send mail to webmaster
with questions or
|