The more an intrusion detection system (IDS) knows about the network it is trying to protect, the better it will be able to protect the network. This is the fundamental principle behind target-based intrusion detection, where an IDS knows about the hosts on the network.
This article explores how artificial intelligence (AI) is influencing IDS development, and what capabilities a popular IDS has with respect to intelligent intrusion detection. Snort is the IDS in question and this article describes some of its features that users might not be taking advantage of that would allow the IDS to adapt to networks and detect anomalies. AI alleviates some of the security professionals' work load by first learning about a network and gauging reactions from a security professional to reduce false positives, and second, by adapting to changes in the network to identify new attacks.
Such knowledge is important, for example, in identifying packet fragmentation attacks, where the hosts on a network have different policies for reassembling fragments. The packet fragmentation issue was discussed by Thomas Ptacek and Timothy Newsham roughly eight years ago in their paper Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection. Since then, Snort developers have implemented a preprocessor for Snort that attempts to address the fragmentation issue. Judy Novak of Sourcefire published a paper in 2005 titled Target-Based Fragmentation Reassembly (pdf). Readers interested in more detail in this topic should read both papers.
Target-based intrusion detection
The fragmentation attack problem can be summarized as follows: if the IDS reassembles packets differently then the host machine, it will miss attacks against the host machine. The fragmentation problem and subsequent attempts to deal with it demonstrate the evolution of intrusion detection and in particular the agility of Snort.
As an example of a fragmentation attack, consider the following Snort rule in the exploit.rules file that comes with a stock Snort install:
Alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg: 'EXPLOIT ssh CRC32 overflow /bin/sh'; flow:to_server,established; content:'/bin/sh'; reference: bugtraq,2347 reference: cve,2001-0144; reference: cve,2001-0572; classtype: shellcode-detect; sid:1324; rev: 6;)
One of the characteristics of this rule is that it looks for the
content '/bin/sh'. Now consider that an attacker fragments his packets
such that the packets sent have the following format. Assume for
simplicity sake that each packet is engineered to only send one byte of
The IDS manages to reassemble the packets and gets the following
However, it turns out that several of the packets are source routed.
The target host does not accept source routed packets and drops those
packets. As a result the host machine reassembles the content to:
Do you see the problem? The IDS and the target host have different packet fragment reassembly policies. The target host does not accept source routed packets while the IDS does, so they end up with different payloads. The payload at the IDS appears harmless; however, the payload at the target host is not. The IDS is unsure how the packets will be reassembled at the host and therefore misses an alarm. This is called an insertion attack, where the IDS accepts a packet that the target host rejects. A similar attack, called an evasion attack, occurs when the IDS rejects a packet that the target host accepts.