We have a new release! Today, Victor Julien announced a new release of Suricata. See the details below:
The OISF development team is proud to announce Suricata 1.0.2, the
second maintenance release for Suricata 1.0, the Open Source Intrusion
Detection and Prevention engine.
Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-1.0.2.tar.gz
New features
- Added an SSH application layer module, improving performance and accuracy
- Added two new SSH rule keywords: "ssh.protoversion" and
"ssh.softwareversion"
- Added support for missing HTTP related PCRE modifiers /H, /M and /C
(bug #220)
Improvements
- Fixed several TCP stream engine evasion issues found by Judy Novak
from G2, Inc.
- Improved accuracy of the http_client_body keyword
- Improved dropping of packets in IPS mode when a signature matches in
the reassembled stream or the application layer
- Improved error reporting if the engine runs out of memory in the
initialization stage
- Fixed a reported segv in the HTTP method detection keyword (bug #231)
- Several smaller issues were fixed
Because of the TCP evasions that are fixed upgrading is highly recommended.
Known issues & missing features
As always, we are doing our best to make you aware of continuing
development and items within the engine that are not yet complete or
optimal. With this in mind, please notice the list we have included of
known items we are working on.
Wednesday, September 8, 2010
Wednesday, July 21, 2010
HowTo setup suricata 1.0.0 on Mac OS X on IDS and IPS mode with IPFW
I'm really proud to announce that Suricata 1.0 has been released. This is the first stable version, with just one year old, as a result of a great effort of the development team, covering compatibility with nearly all the emerging-threats rule feed. Thanks to all of you guys! Keep up the good work. The engine has some known issues, that you can check on the OISF redmine . After the SF meeting I'm sure that we will work out a lot of new features for incoming releases that will start accomplishing the phase two of the project.
In order to setup Suricata running on a Ubuntu Linux/GNU box , you might want to follow the howto's of Victor Julien, posted at his blog (Inliniac).
But if you are a Mac OS X user, you might want to follow this steps. First I will try to cover a basic setup on IDS mode, and later will add the steps needed for IPS mode with IPFW.
1. Get the compiler and libraries:
The fastest way is to install XCode. XCode is a toolkit for Mac OS X developers that includes the most common compilers for GUI and terminal development. It includes GNU gcc, and is able to compile C, C++, Objective-C, Objective-C++, Java and AppleScript.
Next you need the libraries. You can install them one by one, but I guess it's easier, and probably faster to install a port manager tool like MacPorts (that was my choice).
After setting up MacPorts, run the following command:
Ok, now we should have the libraries installed (by default the paths differs a bit from linux.. they are usually installed at /opt/local/include).
2. Get and build the source.
Now let's get the source. Go to the download section of OISF to fetch the latest stable release. Anyway, you can also do this (but you should get the latest up2date version):
3. Prepare the environment
Now that we have the source code compiled, let's prepare the environment.
Let's follow some common steps (should be nearly the same as Linux).
Aadd a user account and group for running suricata. If you have Mac OS X 10.4 or less:
4. Start the engine
Now we have a basic environment prepared for running the engine, on IDS mode and emerging threats rule feed.
You can start the engine by executing:
Cool, now we have the engine working. You can now check the stats log files located at /var/log/suricata. By default all the output types are enabled. Of course this is not the most optimal configuration. You can disable the outputs that doesn't work for you by editing /etc/suricata/suricata.yaml. maybe you want to go further and install sguil, acidbase, snorby, or any other viewer compatible with unified output (but that's another article I should write). By now, lets just check /var/log/suricata/fast.log to view the generated alerts.
5. Setup suricata as IPS with IPFW.
If you want to use suricata as IPS you will need to recompile the source, adding an extra option to the configure script. Go to the path of the sources and run the following commands:
We need a ipfw rule to forward the traffic to the engine. We also need the engine to be running and getting the packets from the divert port of ipfw.
***Otherwise, no program will say to ipfw to allow the traffic, and you'll break all your connections! :)
No worries. Just keep in mind that you can execute the following command to stop the ipfw rule:
Please, notice that we do not specify an interface here. We are getting the traffic from ipfw (not the interface). Now suricata can tell ipfw which packets to allow and which ones to deny.
The engine will also need special rules. By default the rules start with the action "alert", but to use IPS, that word should be "drop".
So now, you can test someting like this:
drop tcp any any -> any 80 (msg:"testing drop"; content:"google"; http_header; sid:123321;)
Save this rule to a file named test.rules and start the engine with
And then try to load any webpage different to google. It should work. And if you try to navigate to google, the engine should directly stop that packets. Getting no response on your web browser (probably a timeout).
Please, notice that to "drop" packets is different to "reject". A reject packet is not dropped, but a special packet is sent to the endpoints to force a connection close. You might want to combine rules with different actions on the same file. Something like:
alert tcp any any -> any 80 (msg:"testing drop"; content:"google"; http_header; sid:1;)
reject tcp any any -> any 80 (msg:"testing reject"; content:"yahoo"; http_header; sid:2;)
drop tcp any any -> any 80 (msg:"testing drop"; content:"bing"; http_header; sid:3;)
And the engine should log all of them, but should only drop requests to bing, reject requests to yahoo, and alert requests to google.
But there's another important action that we need to know. That is "pass". A "pass" rule allows as to ignore anyothers actions triggered from other rules. This means that you can use it to fix possible false positives and add certain exceptions, depending on your network, your ruleset, etc preventing that connections to be dropped or rejected. So we can add the following rule as an exception:
pass tcp any any -> any 80 (msg:"testing drop"; content:"mail.yahoo.com"; http_header; sid:4;)
6. Final notes
To build a good rule set for IPS mode is definitely a must, and a critical task. You will need to fine tune your rule set since you must prevent the engine to fall under false positives that might be the result of a bad rule design, that could drop packets where it should allow them. So you will need to keep an eye on this, and maybe write some scripts to make your own score of reliability for rules (for a huge number of rules), or check them.. one by one. Of course, for a production environment you should use a more adaptative approach, like setting up suricata as ids mode first for a certain testing period, check all the generated alerts for false positives, and avoid using them with the action drop/reject. After that period of time (that should depend on the number of hosts you're monitoring and the throughgput and type of traffic), you should have a more trustable list of rules to modify with the action of drop (or reject).
Reject doesn't depend on ipfw. Keep this in mind since you don't need to pass all the traffic through ipfw to block connections. You can do a midterm approach. For example, you can pass only certain traffic to ipfw by creating more custom rules like "ipfw add 100 divert 8000 ip from 192.168.10.0/24 to my.server.com", then write reject rules like "reject !192.168.10.0/24 any <> !my.server.com any (...)". The "drop" rules will only affect to the communication between 192.168.10.0/24 and my.server.com, but reject will affect to the rest of connections. IPFW has a lot of features you should check, in order to set up your firewall.
With all of this said, you can go further into more complex configurations.
And that's all for now.
Please, feel free to ask me any questions/problems/suggestions you might have following this guideline. I'll be happy to help.
In order to setup Suricata running on a Ubuntu Linux/GNU box , you might want to follow the howto's of Victor Julien, posted at his blog (Inliniac).
But if you are a Mac OS X user, you might want to follow this steps. First I will try to cover a basic setup on IDS mode, and later will add the steps needed for IPS mode with IPFW.
1. Get the compiler and libraries:
The fastest way is to install XCode. XCode is a toolkit for Mac OS X developers that includes the most common compilers for GUI and terminal development. It includes GNU gcc, and is able to compile C, C++, Objective-C, Objective-C++, Java and AppleScript.
Next you need the libraries. You can install them one by one, but I guess it's easier, and probably faster to install a port manager tool like MacPorts (that was my choice).
After setting up MacPorts, run the following command:
port install autoconf automake make libnet11 libpcap pcre \(*probably the auto* tools and "make" are yet installed by xcode).
libyaml libtool pkgconfig
Ok, now we should have the libraries installed (by default the paths differs a bit from linux.. they are usually installed at /opt/local/include).
2. Get and build the source.
Now let's get the source. Go to the download section of OISF to fetch the latest stable release. Anyway, you can also do this (but you should get the latest up2date version):
wget \
"http://openinfosecfoundation.org/download/suricata-1.0.0.tar.gz"
tar xvzf suricata-1.0.0.tar.gz
cd suricata-1.0.0
# If you want to play with suricata code
# you might want to enable debug with --enable-debug.
# if not, unittests should be more than enough
./configure --enable-unittests
make
sudo make install
3. Prepare the environment
Now that we have the source code compiled, let's prepare the environment.
Let's follow some common steps (should be nearly the same as Linux).
Aadd a user account and group for running suricata. If you have Mac OS X 10.4 or less:
dscl / -create /Users/suricata UserShell /bin/false \If you have Mac OS X 10.5 or higher:
RealName "Suricata idps engine" UniqueID 500 PrimaryGroupID 500
dscl . -create /Users/suricata UserShell /bin/false \As the shell is /bin/false, you should not be able to log in with it.
RealName "Suricata idps engine" UniqueID 500 PrimaryGroupID 500
# Create a directory for logs:Now you can get rule feeds for the engine from two different providers: Emerging Threats, and Sourcefire VRT. Let's go with emerging threats:
sudo mkdir /var/log/suricata/
# A directory for the config files:
sudo mkdir /etc/suricata/
# Copy the config file and classification config to /etc/suricata/
sudo cp /path/to/suricata-1.0.0/suricata.yaml /etc/suricata/
sudo cp /path/to/suricata-1.0.0/classification.config\
/etc/suricata/
# Ensure we will have enough perms to write the logs
sudo chown suricata:suricata /var/log/suricata/
wget http://www.emergingthreats.net/rules/emerging.rules.tar.gz
cd /etc/suricata/
sudo tar xzvf /path/to/emerging.rules.tar.gz
4. Start the engine
Now we have a basic environment prepared for running the engine, on IDS mode and emerging threats rule feed.
You can start the engine by executing:
suricata -c /etc/suricata/suricata.yaml -i en1 --user suricata\(the interface is en1 on my box, but it might differ to yours).
--group suricata
Cool, now we have the engine working. You can now check the stats log files located at /var/log/suricata. By default all the output types are enabled. Of course this is not the most optimal configuration. You can disable the outputs that doesn't work for you by editing /etc/suricata/suricata.yaml. maybe you want to go further and install sguil, acidbase, snorby, or any other viewer compatible with unified output (but that's another article I should write). By now, lets just check /var/log/suricata/fast.log to view the generated alerts.
5. Setup suricata as IPS with IPFW.
If you want to use suricata as IPS you will need to recompile the source, adding an extra option to the configure script. Go to the path of the sources and run the following commands:
Now that we have the binary capable of talking with ipfw, let's say to ipfw what traffic we want to allow/reject with suricata. By default, ipfw has a "catch all" rule, allowing all ip traffic. This one:
cd /path/to/suricata-1.0.0
./configure --enable-unittests --enable-ipfw
make
sudo make install
ipfw list
65535 allow ip from any to any
We need a ipfw rule to forward the traffic to the engine. We also need the engine to be running and getting the packets from the divert port of ipfw.
***Otherwise, no program will say to ipfw to allow the traffic, and you'll break all your connections! :)
No worries. Just keep in mind that you can execute the following command to stop the ipfw rule:
ipfw flushAnd you will have connection again. So, what we are going to do is to add the following rule:
Are you sure? [yn] y
Flushed all rules.
ipfw add 100 divert 8000 ip from any to anyYou can check the rules by running ipfw list. And then, launch suricata reading from the divert 8000 we have just loaded into ipfw:
# it should print something like this:
# 00100 divert 8000 ip from any to any
suricata -c /etc/suricata/suricata.yaml -d 8000 --user suricata\
--group suricata
Please, notice that we do not specify an interface here. We are getting the traffic from ipfw (not the interface). Now suricata can tell ipfw which packets to allow and which ones to deny.
The engine will also need special rules. By default the rules start with the action "alert", but to use IPS, that word should be "drop".
So now, you can test someting like this:
drop tcp any any -> any 80 (msg:"testing drop"; content:"google"; http_header; sid:123321;)
Save this rule to a file named test.rules and start the engine with
suricata -c /etc/suricata/suricata.yaml -d 8000 --user suricata\
--group suricata -s test.rules
And then try to load any webpage different to google. It should work. And if you try to navigate to google, the engine should directly stop that packets. Getting no response on your web browser (probably a timeout).
Please, notice that to "drop" packets is different to "reject". A reject packet is not dropped, but a special packet is sent to the endpoints to force a connection close. You might want to combine rules with different actions on the same file. Something like:
alert tcp any any -> any 80 (msg:"testing drop"; content:"google"; http_header; sid:1;)
reject tcp any any -> any 80 (msg:"testing reject"; content:"yahoo"; http_header; sid:2;)
drop tcp any any -> any 80 (msg:"testing drop"; content:"bing"; http_header; sid:3;)
And the engine should log all of them, but should only drop requests to bing, reject requests to yahoo, and alert requests to google.
But there's another important action that we need to know. That is "pass". A "pass" rule allows as to ignore anyothers actions triggered from other rules. This means that you can use it to fix possible false positives and add certain exceptions, depending on your network, your ruleset, etc preventing that connections to be dropped or rejected. So we can add the following rule as an exception:
pass tcp any any -> any 80 (msg:"testing drop"; content:"mail.yahoo.com"; http_header; sid:4;)
6. Final notes
To build a good rule set for IPS mode is definitely a must, and a critical task. You will need to fine tune your rule set since you must prevent the engine to fall under false positives that might be the result of a bad rule design, that could drop packets where it should allow them. So you will need to keep an eye on this, and maybe write some scripts to make your own score of reliability for rules (for a huge number of rules), or check them.. one by one. Of course, for a production environment you should use a more adaptative approach, like setting up suricata as ids mode first for a certain testing period, check all the generated alerts for false positives, and avoid using them with the action drop/reject. After that period of time (that should depend on the number of hosts you're monitoring and the throughgput and type of traffic), you should have a more trustable list of rules to modify with the action of drop (or reject).
Reject doesn't depend on ipfw. Keep this in mind since you don't need to pass all the traffic through ipfw to block connections. You can do a midterm approach. For example, you can pass only certain traffic to ipfw by creating more custom rules like "ipfw add 100 divert 8000 ip from 192.168.10.0/24 to my.server.com", then write reject rules like "reject !192.168.10.0/24 any <> !my.server.com any (...)". The "drop" rules will only affect to the communication between 192.168.10.0/24 and my.server.com, but reject will affect to the rest of connections. IPFW has a lot of features you should check, in order to set up your firewall.
With all of this said, you can go further into more complex configurations.
And that's all for now.
Please, feel free to ask me any questions/problems/suggestions you might have following this guideline. I'll be happy to help.
Tuesday, April 20, 2010
New suricata release 0.8.2
New suricata release! Have a look at the new features and changelog!
The OISF development team is proud to introduce the 3rd beta release of
Suricata, the Open Source Intrusion Detection and Prevention engine. The
first release candidate is currently scheduled for early May, but check
https://redmine.openinfosecfoundation.org/projects/roadmap/suricata for
the up to date schedule!
Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-0.8.2.tar.gz
New features
- Support for the following keywords: detection_filter, http_client_body
- The HTTP parser can now set server personalities
- threshold.config support
- The experimental CUDA code now also works on x86_64
- IP address only rules for IPv6 are now supported as well
- Suricata can now write a pid file (pass --pidfile)
- A fuzzer script was added to the code base
- Policy lookup for defrag module
Improvements
- Much better average and worstcase performance in the detection engine
- Memory footprint was reduced
- More validation at signature loading stage
- Libnet 1.1 is now optional
- Negated uricontent and http_cookie matching is now supported
- Lots of fixes of issues found by Valgrind's DRD, CLANG and Parfait.
- Threads are named now in "top" (Linux only atm).
- Unified1 file handling is improved
Bugs fixed
Many :)
Several segmentation faults, upgrading is highly recommended.
See
https://redmine.openinfosecfoundation.org/projects/suricata/issues?fixed_version_id=6&set_filter=1&status_id=c
Known issues & missing features
We have made significant progress towards reaching our first full
(non-beta) release of Suricata. Your feedback is always important to us
and we appreciate your time and effort. As always, we are doing our
best to make you aware of continuing development and items within the
engine that are not yet complete. With this in mind, please notice the
list we have included of known items we are working on.
- Using the http_cookie keyword seems to cause a match on all packets.
- Currently we dont' support the dce option for byte_test and byte_jump.
- Stream reassembly is currently only performed for app-layer code.
- Inconsistent time stamps in http log file due to handling & updating
of the http state.
- DCE/RPC over udp is not currently supported.
- dce_stub_data does not respect relative modifiers.
- Engine does not work properly on big endian platforms.
- Time based stats are not calculated correctly.
See https://redmine.openinfosecfoundation.org/projects/suricata/issues
for an up to date list and to report new issues.
The OISF development team is proud to introduce the 3rd beta release of
Suricata, the Open Source Intrusion Detection and Prevention engine. The
first release candidate is currently scheduled for early May, but check
https://redmine.openinfosecfoundation.org/projects/roadmap/suricata for
the up to date schedule!
Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-0.8.2.tar.gz
New features
- Support for the following keywords: detection_filter, http_client_body
- The HTTP parser can now set server personalities
- threshold.config support
- The experimental CUDA code now also works on x86_64
- IP address only rules for IPv6 are now supported as well
- Suricata can now write a pid file (pass --pidfile
- A fuzzer script was added to the code base
- Policy lookup for defrag module
Improvements
- Much better average and worstcase performance in the detection engine
- Memory footprint was reduced
- More validation at signature loading stage
- Libnet 1.1 is now optional
- Negated uricontent and http_cookie matching is now supported
- Lots of fixes of issues found by Valgrind's DRD, CLANG and Parfait.
- Threads are named now in "top" (Linux only atm).
- Unified1 file handling is improved
Bugs fixed
Many :)
Several segmentation faults, upgrading is highly recommended.
See
https://redmine.openinfosecfoundation.org/projects/suricata/issues?fixed_version_id=6&set_filter=1&status_id=c
Known issues & missing features
We have made significant progress towards reaching our first full
(non-beta) release of Suricata. Your feedback is always important to us
and we appreciate your time and effort. As always, we are doing our
best to make you aware of continuing development and items within the
engine that are not yet complete. With this in mind, please notice the
list we have included of known items we are working on.
- Using the http_cookie keyword seems to cause a match on all packets.
- Currently we dont' support the dce option for byte_test and byte_jump.
- Stream reassembly is currently only performed for app-layer code.
- Inconsistent time stamps in http log file due to handling & updating
of the http state.
- DCE/RPC over udp is not currently supported.
- dce_stub_data does not respect relative modifiers.
- Engine does not work properly on big endian platforms.
- Time based stats are not calculated correctly.
See https://redmine.openinfosecfoundation.org/projects/suricata/issues
for an up to date list and to report new issues.
Subscribe to:
Posts (Atom)