Showing posts with label ids. Show all posts
Showing posts with label ids. Show all posts

Friday, January 14, 2011

Python + divert sockets + scapy

I have prepared a small class for playing with divert sockets and python. This example shows how to register a packet handler for packets fetched from the divert, then it loads a Scapy packet from the IP layer, displaying the contents, and at last it forward the packet to be delivered. It's just a Proof of Concept, that doesn't block packets, but if you implement your logic at the packet handler, setting the veredict to false it should block it.

And you might think now, why not just use a crafted RST packet from scapy? And I would answer, because it is more reliable, and because you might prefer not to send anything to the source IP that you want to block. An rst packet is enough to know that something (some app) is alive at the other side. On the other hand, a combination of both would be the best fit here, because that will avoid long timeouts while waitting for an answer, and duplicated requests (some browsers make a lot of retries).


To test it (on MacOSX), you need to have scapy installed. Then create a divert socket with a rule similar to this (be careful, this will enqueue all the packets at ipfw and it might break your connections if you don't attach a program to veredict them on time):

one@macuto2$ sudo ipfw add divert 3282 tcp from any to any
Password:
00100 divert 3282 ip from any to any proto tcp

Now run the attached script as follows:
sudo python2.6 DivertSocket.py 3282
And it should start fetching packets reaching the function of the packet handler, where you should place your logic. You might want to try to block petitions by for example ip addresses (just for testing), or content payload, but remember that this doesn't reassemble tcp streams, there would be much more to do for content inspection. Anyway I hope someone will have some fun testing it.


DivertSocket.py
-- cut here --
import socket
import sys
import re

from scapy.all import *

if not socket.__dict__.has_key("IPPROTO_DIVERT"):
# Define if
socket.IPPROTO_DIVERT = 254

class DivertSocket:
def __init__(self, port, delegateFunc=None):

self.sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_DIVERT)
# Here the addr can be any. The important one is the port
self.sock.bind(("0.0.0.0", port))
# By default the max
self.bufsize = 65535
# Set blocking
self.sock.setblocking(True)
# Register callback
self.delegateFunc = delegateFunc
self.__loop = 1

def start(self, default=0):
self.fetchPackets(default)

def fetchPackets(self, default=0):
while self.__loop:
buf, addr = self.sock.recvfrom(self.bufsize)
# If we registered a delegate funcion, call it
if self.delegateFunc != None:
self.delegateFunc(buf, addr)
# Else send it if the default behavior matches
else:
print "Warning, no functions registered for inspection!"
if default:
self.sendPacket(buf, addr)
else:
print "You need to implement a callback function for inspection"
sys.exit(-1)

def setVeredict(self, buf, addr, veredict=False):
if veredict:
if self.__sendPacket(buf, addr) == False:
print "Pkt not sent. Weird.. Need to see which packet causes this error"

def __sendPacket(self, buf, addr = None):
try:
if addr:
return self.sock.sendto(buf, addr)
#else try send it raw anyway..
return self.sock.send(buf)
except KeyboardInterrupt, e:
print "Stopping Engine..."
sys.exit(0)
except:
print "Could not send packet..."
return False

def stop(self):
self.__loop = 0
self.sock.close()


def pktHandler(buf, addr):
p = IP(buf)
print p.display()
ds.setVeredict(buf,addr, True)

ds = DivertSocket(int(sys.argv[1]), pktHandler)

ds.start()
-- stop here --

BTW, I would be very pleased if someone can test it under linux using iptables. Sooner or later I'll try it anyway.

Happy hacking!
;-)

Wednesday, July 21, 2010

HowTo setup suricata 1.0.0 on Mac OS X on IDS and IPS mode with IPFW

I'm really proud to announce that Suricata 1.0 has been released. This is the first stable version, with just one year old, as a result of a great effort of the development team, covering compatibility with nearly all the emerging-threats rule feed. Thanks to all of you guys! Keep up the good work. The engine has some known issues, that you can check on the OISF redmine . After the SF meeting I'm sure that we will work out a lot of new features for incoming releases that will start accomplishing the phase two of the project.

In order to setup Suricata running on a Ubuntu Linux/GNU box , you might want to follow the howto's of Victor Julien, posted at his blog (Inliniac).

But if you are a Mac OS X user, you might want to follow this steps. First I will try to cover a basic setup on IDS mode, and later will add the steps needed for IPS mode with IPFW.


1. Get the compiler and libraries:

The fastest way is to install XCode. XCode is a toolkit for Mac OS X developers that includes the most common compilers for GUI and terminal development. It includes GNU gcc, and is able to compile C, C++, Objective-C, Objective-C++, Java and AppleScript.

Next you need the libraries. You can install them one by one, but I guess it's easier, and probably faster to install a port manager tool like MacPorts (that was my choice).

After setting up MacPorts, run the following command:
port install autoconf automake make libnet11 libpcap pcre \
libyaml libtool pkgconfig
(*probably the auto* tools and "make" are yet installed by xcode).

Ok, now we should have the libraries installed (by default the paths differs a bit from linux.. they are usually installed at /opt/local/include).


2. Get and build the source.

Now let's get the source. Go to the download section of OISF to fetch the latest stable release. Anyway, you can also do this (but you should get the latest up2date version):
wget \
"http://openinfosecfoundation.org/download/suricata-1.0.0.tar.gz"
tar xvzf suricata-1.0.0.tar.gz
cd suricata-1.0.0
# If you want to play with suricata code
# you might want to enable debug with --enable-debug.
# if not, unittests should be more than enough
./configure --enable-unittests
make
sudo make install

3. Prepare the environment

Now that we have the source code compiled, let's prepare the environment.

Let's follow some common steps (should be nearly the same as Linux).
Aadd a user account and group for running suricata. If you have Mac OS X 10.4 or less:
dscl / -create /Users/suricata UserShell /bin/false \
RealName "Suricata idps engine" UniqueID 500 PrimaryGroupID 500
If you have Mac OS X 10.5 or higher:
dscl . -create /Users/suricata UserShell /bin/false \
RealName "Suricata idps engine" UniqueID 500 PrimaryGroupID 500
As the shell is /bin/false, you should not be able to log in with it.
# Create a directory for logs:
sudo mkdir /var/log/suricata/
# A directory for the config files:
sudo mkdir /etc/suricata/

# Copy the config file and classification config to /etc/suricata/
sudo cp /path/to/suricata-1.0.0/suricata.yaml /etc/suricata/
sudo cp /path/to/suricata-1.0.0/classification.config\
/etc/suricata/

# Ensure we will have enough perms to write the logs
sudo chown suricata:suricata /var/log/suricata/
Now you can get rule feeds for the engine from two different providers: Emerging Threats, and Sourcefire VRT. Let's go with emerging threats:
wget http://www.emergingthreats.net/rules/emerging.rules.tar.gz
cd /etc/suricata/
sudo tar xzvf /path/to/emerging.rules.tar.gz

4. Start the engine

Now we have a basic environment prepared for running the engine, on IDS mode and emerging threats rule feed.

You can start the engine by executing:
suricata -c /etc/suricata/suricata.yaml -i en1 --user suricata\
--group suricata
(the interface is en1 on my box, but it might differ to yours).

Cool, now we have the engine working. You can now check the stats log files located at /var/log/suricata. By default all the output types are enabled. Of course this is not the most optimal configuration. You can disable the outputs that doesn't work for you by editing /etc/suricata/suricata.yaml. maybe you want to go further and install sguil, acidbase, snorby, or any other viewer compatible with unified output (but that's another article I should write). By now, lets just check /var/log/suricata/fast.log to view the generated alerts.

5. Setup suricata as IPS with IPFW.

If you want to use suricata as IPS you will need to recompile the source, adding an extra option to the configure script. Go to the path of the sources and run the following commands:

cd /path/to/suricata-1.0.0
./configure --enable-unittests --enable-ipfw
make
sudo make install
Now that we have the binary capable of talking with ipfw, let's say to ipfw what traffic we want to allow/reject with suricata. By default, ipfw has a "catch all" rule, allowing all ip traffic. This one:

ipfw list
65535 allow ip from any to any


We need a ipfw rule to forward the traffic to the engine. We also need the engine to be running and getting the packets from the divert port of ipfw.
***Otherwise, no program will say to ipfw to allow the traffic, and you'll break all your connections! :)
No worries. Just keep in mind that you can execute the following command to stop the ipfw rule:
ipfw flush
Are you sure? [yn] y

Flushed all rules.
And you will have connection again. So, what we are going to do is to add the following rule:
ipfw add 100 divert 8000 ip from any to any

# it should print something like this:
# 00100 divert 8000 ip from any to any

You can check the rules by running ipfw list. And then, launch suricata reading from the divert 8000 we have just loaded into ipfw:

suricata -c /etc/suricata/suricata.yaml -d 8000 --user suricata\
--group suricata


Please, notice that we do not specify an interface here. We are getting the traffic from ipfw (not the interface). Now suricata can tell ipfw which packets to allow and which ones to deny.

The engine will also need special rules. By default the rules start with the action "alert", but to use IPS, that word should be "drop".

So now, you can test someting like this:

drop tcp any any -> any 80 (msg:"testing drop"; content:"google"; http_header; sid:123321;)

Save this rule to a file named test.rules and start the engine with

suricata -c /etc/suricata/suricata.yaml -d 8000 --user suricata\
--group suricata -s test.rules

And then try to load any webpage different to google. It should work. And if you try to navigate to google, the engine should directly stop that packets. Getting no response on your web browser (probably a timeout).

Please, notice that to "drop" packets is different to "reject". A reject packet is not dropped, but a special packet is sent to the endpoints to force a connection close. You might want to combine rules with different actions on the same file. Something like:

alert tcp any any -> any 80 (msg:"testing drop"; content:"google"; http_header; sid:1;)

reject tcp any any -> any 80 (msg:"testing reject"; content:"yahoo"; http_header; sid:2;)

drop tcp any any -> any 80 (msg:"testing drop"; content:"bing"; http_header; sid:3;)


And the engine should log all of them, but should only drop requests to bing, reject requests to yahoo, and alert requests to google.

But there's another important action that we need to know. That is "pass". A "pass" rule allows as to ignore anyothers actions triggered from other rules. This means that you can use it to fix possible false positives and add certain exceptions, depending on your network, your ruleset, etc preventing that connections to be dropped or rejected. So we can add the following rule as an exception:

pass tcp any any -> any 80 (msg:"testing drop"; content:"mail.yahoo.com"; http_header; sid:4;)

6. Final notes

To build a good rule set for IPS mode is definitely a must, and a critical task. You will need to fine tune your rule set since you must prevent the engine to fall under false positives that might be the result of a bad rule design, that could drop packets where it should allow them. So you will need to keep an eye on this, and maybe write some scripts to make your own score of reliability for rules (for a huge number of rules), or check them.. one by one. Of course, for a production environment you should use a more adaptative approach, like setting up suricata as ids mode first for a certain testing period, check all the generated alerts for false positives, and avoid using them with the action drop/reject. After that period of time (that should depend on the number of hosts you're monitoring and the throughgput and type of traffic), you should have a more trustable list of rules to modify with the action of drop (or reject).

Reject doesn't depend on ipfw. Keep this in mind since you don't need to pass all the traffic through ipfw to block connections. You can do a midterm approach. For example, you can pass only certain traffic to ipfw by creating more custom rules like "ipfw add 100 divert 8000 ip from 192.168.10.0/24 to my.server.com", then write reject rules like "reject !192.168.10.0/24 any <> !my.server.com any (...)". The "drop" rules will only affect to the communication between 192.168.10.0/24 and my.server.com, but reject will affect to the rest of connections. IPFW has a lot of features you should check, in order to set up your firewall.

With all of this said, you can go further into more complex configurations.
And that's all for now.

Please, feel free to ask me any questions/problems/suggestions you might have following this guideline. I'll be happy to help.

Tuesday, April 20, 2010

New suricata release 0.8.2

New suricata release! Have a look at the new features and changelog!

The OISF development team is proud to introduce the 3rd beta release of
Suricata, the Open Source Intrusion Detection and Prevention engine. The
first release candidate is currently scheduled for early May, but check
https://redmine.openinfosecfoundation.org/projects/roadmap/suricata for
the up to date schedule!

Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-0.8.2.tar.gz

New features

- Support for the following keywords: detection_filter, http_client_body
- The HTTP parser can now set server personalities
- threshold.config support
- The experimental CUDA code now also works on x86_64
- IP address only rules for IPv6 are now supported as well
- Suricata can now write a pid file (pass --pidfile )
- A fuzzer script was added to the code base
- Policy lookup for defrag module

Improvements

- Much better average and worstcase performance in the detection engine
- Memory footprint was reduced
- More validation at signature loading stage
- Libnet 1.1 is now optional
- Negated uricontent and http_cookie matching is now supported
- Lots of fixes of issues found by Valgrind's DRD, CLANG and Parfait.
- Threads are named now in "top" (Linux only atm).
- Unified1 file handling is improved

Bugs fixed

Many :)
Several segmentation faults, upgrading is highly recommended.

See
https://redmine.openinfosecfoundation.org/projects/suricata/issues?fixed_version_id=6&set_filter=1&status_id=c

Known issues & missing features

We have made significant progress towards reaching our first full
(non-beta) release of Suricata. Your feedback is always important to us
and we appreciate your time and effort. As always, we are doing our
best to make you aware of continuing development and items within the
engine that are not yet complete. With this in mind, please notice the
list we have included of known items we are working on.

- Using the http_cookie keyword seems to cause a match on all packets.
- Currently we dont' support the dce option for byte_test and byte_jump.
- Stream reassembly is currently only performed for app-layer code.
- Inconsistent time stamps in http log file due to handling & updating
of the http state.
- DCE/RPC over udp is not currently supported.
- dce_stub_data does not respect relative modifiers.
- Engine does not work properly on big endian platforms.
- Time based stats are not calculated correctly.

See https://redmine.openinfosecfoundation.org/projects/suricata/issues
for an up to date list and to report new issues.

Saturday, September 19, 2009

Snort ( 2.8.* < 2.8.5stable) Unified1 output bug

I have published a bug of snort 2.8. There are two proof of concept scripts here in this tar.gz and a deep explanation of the problem in the .pdf

Here is the advisory :)


Advisory:
=========
Snort unified 1 IDS Logging Alert Evasion, Logfile Corruption/Alert Falsify


Log:
====
30/06/2009 Bug detected.
20/07/2009 First mail with snort team.
20/07/2009 Snort team answer they will fix it in the next release (2.8.5).
16/09/2009 Snort 2.8.5 released, bug fixed.


Affected Versions:
==================
snort-2.8.1
snort-2.8.2
snort-2.8.3
snort-2.8.4
snort-2.8.5.beta*


Discussion:
===========
snort-2.8.* is susceptible to a Denial Of Service Vulnerability with Snort unified 1 binary format.
It occurs when snort.conf use the classic unified 1 output configuration as follows:
output unified: filename snort.log, limit 128

and Stream5 preprocessor is enabled.

This issue is due to the application's failure to properly set the offset of a memory buffer write when logging packet rebuilt streams data, resulting in corrupted unified log header and data in the logfile, out-of-bounds offsets, making impossible to parse/view the generated alerts with a normal parser/alert frontend.

When an alert has packet data information (the raw packet) the function UnifiedLogStreamCallback() write the raw packet data overwriting the UnifiedLog header, that has the type and size of the alert, followed by the alert information.

--output-plugins/spo_unified.c line 803 at least in snort-2.8.4 function UnifiedLogStreamCallback()
-------->SafeMemcpy(write_pkt_buffer, packet_data,...
should be
-------->SafeMemcpy(write_pkt_buffer + offset, packet_data,...

With this bug, the alert type and size are overwritten with the MAC addresses of the raw packet, so with malformed packets (Eth/IP/TCP/Data with modified MAC addresses), the size and the type (and other information) can be set falsifing alerts for a later parsing process. If an attacker build malformed packets, so an alert is falsified the size is bigger than 128M (the unified log limit size by default), snort will continue inserting alerts in the file, but a parser when reading that alert will try to jump 128M skipping the alerts inserted after the falsified one.

An attacker can also insert a complete list of falsified alerts malforming packets, because the raw packet has TCP data that you can fill with falsified UnifiedLog alert structures (with the binary data), so would need to adjust the packet headers to set the "size of the alert" (overwritten with the MACs of the packet) making that the parser read the next alert in the offset that the TCP data will overwrite(the list of falsified alerts).


Impact:
=======
With this bug an attacker can break the alert log headers, making impossible for a parser to extract the alert information correctly. An attacker can also insert falsified alerts in the logfiles by injecting unified structures with false alerts, false pcaps (Ethernet/Ip/Tcp/Data) by malforming the packets of a TCP stream that match a normal alert, which wont be even correctly inserted.


Proof of concept:
=================

To reproduce the bug you must have a unified 1 parser accepting unified logs with the configuration in snort.conf as " output unified: filename snort.log, limit 128" and the Stream5 preprocessor enabled. Then you need to send a content payload that will generate an alert, but this payload must be divided in two parts, sending them in two consecutive (and different) packets so Stream5 will reassemble as a PKT_REBUILT_STREAM. The header of the unified Alert log will be overwritten with the raw packet information. There are two proof of concept scapy scripts, one generate a pcap that insert an alert overwriting the header so that a parser think that the alert is bigger than 128M, and another that insert a falsified alert.

The pcaps can be processed in snort with snort -r "the_file.pcap"...


Fix:
====
Install snort-2.8.5 or add the offset and recompile snort:
--output-plugins/spo_unified.c line 803 at least in snort-2.8.4 function UnifiedLogStreamCallback()

SafeMemcpy(write_pkt_buffer + offset, unifiedData->logheader,
sizeof(UnifiedLog), write_pkt_buffer,
write_pkt_buffer + sizeof(DataHeader) +
sizeof(UnifiedLog) + IP_MAXPACKET);

offset += sizeof(UnifiedLog);

unifiedData->data->current += sizeof(UnifiedLog);

if(packet_data)
{
-------->SafeMemcpy(write_pkt_buffer, packet_data,
offset + unifiedData->logheader->pkth.caplen,
write_pkt_buffer, write_pkt_buffer +
sizeof(DataHeader) + sizeof(UnifiedLog) + IP_MAXPACKET);

if(fwrite(write_pkt_buffer, offset + unifiedData->logheader->pkth.caplen,
1, unifiedData->data->stream) != 1)
FatalError("SpoUnified: write failed: %s\n", strerror(errno));

unifiedData->data->current += unifiedData->logheader->pkth.caplen;
}
else
--+ 825

Look at that closely and you'll see that the buffer is overwritten if packet_data is not 0, and then the buffer is written to the log file. The fix is really simple. Just write to write_pkt_buffer +offset, instead of write_pkt_buffer.

-------->SafeMemcpy(write_pkt_buffer, packet_data,...
-------->SafeMemcpy(write_pkt_buffer + offset, packet_data,...

or use unified 2.

Conclusions:
========
An attacker can:
1. Corrupt the log files.
2. Perform attacks after malformed packets in order prevent that they would get
logged/displayed.
3. Make a DOS for the parsers by inserting alerts with header size > than the filesize limit
(They would loose a lot of alerts...).
4. Insert a complete falsified attack session by encapsulating many alerts in the malformed
tcp packets.
5. We can patch it and be happy with our systems using Stream5 and the rest of the
preprocessors :)
6. It's extremely recommended to use unified2 if you're not using it yet.
7. If you still using unified1, use alert_unified or (log_unified), or just unified but patching snort.


Thanks to:
==========
Jaime Blasco and Juan Blanco working on the ossim-agent "arakiri".
Matt Jonkman and Victor Julien (a pcap generated with the splicer script was the starting point for the scapy scripts).
Carlos Terrón for his great unified1 parser.
The OSSIM project.


Credits:
========
Pablo Rincón Crespo 31/07/2009
pablo.rincon.crespo
at gmail

Wednesday, September 16, 2009

another birthday present

From: Snort Releases
To:
snort-devel@lists.sourceforge.net
Subject: [Snort-devel] Snort 2.8.5 Now Available

Snort 2.8.5 is now available on snort.org, at
http://www.snort.org/downloads/

[...]