min-width: mobile
min-width: 400px
min-width: 550px
min-width: 750px
min-width: 1000px
min-width: 1200px

Blog & News

Using Sagan with Bro Intelligence feeds

Posted by Champ Clark on February 20, 2015

At Quadrant Information Security, we have been big fans of “Bro” (https://www.bro.org/) for a while and it is one of the tools we use as part of our managed security offerings. If you are not already familiar with Bro and you work in the security space, stop what you’re doing and research/play with it now!

Seriously.  

Bro is more than a traditional Intrusion Detection System (IDS); it is a network security framework you can bend and adapt to identify different types of threats.  We think that it is a great tool.

At Quadrant, we like to take the logs from Bro and forward them to our Sagan software. Sagan can then do further analytics based on that data.   For example, we might use Sagan “flowbits” to associate data from Bro with other security software. Sagan “flowbits” act as the “glue” between log events which allows you to create conditions across multiple logs events.  An example of a “flowbit” in action with Sagan and Bro might be as follows:

“If Sagan receives a Bro log of a malicious EXE being download and Sagan does not receive a corresponding log from an anti-virus stating that it detected the malicious EXE,  then trigger an alarm.” 

On twitter, I saw an announcement from @taosecurity (Richard Bejtlich) about a new service called “Critical Stack” (http://intel.criticalstack.com) / @criticalstack. When I arrived at work, I found that I was not the only one who had heard of “Critical Stack”; several of my colleagues had heard about it as well. “Critical Stack” is “a free threat intelligence tool that is aggregated, parsed, and delivered by Critical Stack, Inc. for use on the Bro network security monitoring platform”.    Essentially, “Critical Stack” gives you access to multiple security intelligence feeds. At the time of this writing, “Critical Stack” aggregates a total of 58 feeds.  These feeds can include known blacklisted IP addresses,  hashes of known malware,  pre-generated malware DGAs,  hostile URLs,  hostile domains, and more.  The idea of “Critical Stack” is to create a “sensor” and tie the “feeds” to that sensor. You can then use the Critical Stack API / software to “download” the feeds to your Bro system.

If you have ever dealt with these types of feeds in the past, one issue you have likely noticed is that each feed’s data is formatted slightly differently from data of other feeds. While one feed might be a traditional CSV file that you can download and import, another might be a ZIP file that is tab delimited rather than comma delimited. 

What you end up with is a bunch of files with useful data that you have to “hack” together so your software can read and utilize them.  

This is an area where ‘Critical Stack” comes into play; the feeds come to you “optimized and ready” for the Bro intelligence framework. In other words, you don't have to kludge the data together; it is already ready to be loaded into Bro! 

In a nut shell, here is what I like about “Critical Stack”:

  1. Gives you a really nice Web portal to “subscribe” to multiple feeds. This interface is nice, simple, and slick.
  2. Really good assortment of different types of feeds.
  3. All feeds are downloaded to the Bro sensor via one program / API (Critical Stack).
  4. Feeds that are downloaded are automatically formatted to be fed to Bro.
  5. Critical Stack uses the Bro Intelligence framework to format intelligence data.

Why should Bro get to have all the fun??!?!

Sagan has been able to use “blacklist” data for years and years. From a commercial stand point, Sagan can do real time log analysis and Websense Threatseeker look ups.  You can also create custom “blacklist” addresses or keywords you would like to search for in your logs in real time.

Since we like all these cool features about “Critical Stack”, why not allow Sagan to read Bro Intelligence Feeds?   So… that's what we did. 

I quickly saw that implementation would be fairly easy;  Bro Intelligence feeds are pretty easy to parse and read.   If we can read Bro Intelligence feeds,   we can pull in Critical Stack data ((typically master-public.bro.dat).   The Bro Intelligence framework has several different types of “indicator_types”.   Current,   Sagan supports the following Bro Intellegence framework indicator types:

Intel::ADDR
Intel::FILE_HASH
Intel::URL
Intel::SOFTWARE
Intel::EMAIL
Intel::USER_NAME
Intel::FILE_NAME
Intel::CERT_HASH

If the Sagan “bro-intel” processor is enabled, it will load each of these “indicator types” into an array for later use. Speaking of the Sagan “bro-intel” processor, this is how it is loaded from the “sagan.conf” configuration file:

processor bro-intel: /opt/critical-stack/frameworks/intel/master-public.bro.dat

The “processor bro-intel:” portion of the configuration informs Sagan that you would like to load Bro Intelligence data to use with log analysis.    The “/opt/critical-stack/frameworks/intel/master-public.bro.dat” tells Sagan what Bro Intel file(s) to load.  Note that I said “file(s)”.  Multiple Bro Intelligence files can be loaded.   The file names simple need to be comma delimited.   In our case,  we are loading the default file location Critical Stack Bro Intel data for log analysis.

When you load Sagan, you should see some information pertaining to “Bro Intel” data being loaded.  It will look something like this:

[*] Bro Intel::ADDR Loaded: 17412
[*] Bro Intel::DOMAIN Loaded: 27247
[*] Bro Intel::FILE_HASH Loaded: 441
[*] Bro Intel::URL Loaded: 27832
[*] Bro Intel::SOFTWARE Loaded: 0
[*] Bro Intel::EMAIL Loaded: 0
[*] Bro Intel::USER_NAME Loaded: 0
[*] Bro Intel::FILE_NAME Loaded: 0
[*] Bro Intel::CERT_HASH Loaded: 0
[*] Bro Intel Duplicates Detected: 2


This is to inform you what types of Bro Intellegence data has been loaded (in our case,  Critical Stack data!).

Yeah.  Let me see what I can do.  Now you need rules to use the Bro Intelligence / Critical Stack data feeds!  

Let's create a really simple rule as an example just to get started. In our simple example, let's say our router creates a syslog message of “TCP connection from {src ip address} to {dst ip address}”. The syslog message is create by the program 'router'. What we want to alert on is any time the {dst ip address} shows up in our Intel::ADDR list.  We might create a rule like this:

alert tcp any any -> any any (msg: “[Sagan/Bro Test Rule] Found SRC IP in Intel ADDR!”; program: router; content: “TCP connection from”; parse_src_ip: 1; parse_dst_ip: 2; bro-intel: src_ipaddr; classtype: suspicious-traffic; sid: 900000; rev: 1;)

Sagan will only trigger this rule if the program 'router' sends a syslog message that contains “TCP connection from” in it and the source ip address is found in the Bro Intel::ADDR feed!

If we would like to check both the source and destination address,  we could change the “bro-intel” option to:

bro-intel: src_ipaddr,  dst_ipaddr;

Or better yet:

bro-intel: both_ipaddr;

Another example might be examining proxy logs for possible indicators. In this case, you might want to check multiple indicator types.  For example:

bro-intel: domain, url,  dst_ipaddr;

This would cause Sagan to lookup Intel::DOMAIN, Intel::URL and Intel::ADDR (on the destination address) when a Sagan rule is triggered. 

Want to check all logs for possible indicators? All you need to do is create a Sagan rule that lacks pcre or content. When Sagan lacks “content” or “pcre”, it will “match” on all log lines. We can then use the “all_ipaddr” with our “bro-intel” rule option. For example:

alert udp any any -> any any (msg: “[Sagan/Bro Test Rule] IP Address found in Intel ADDR!”; bro-intel: all_ipaddr;  parse_src_ip: 1;  parse_dst_ip: 2;  parse_proto; classtype: suspicious-traffic; sid 9000001; rev: 1;) ;

Any time Sagan encounters a non-RFC1918 IP address, it will check against your “Bro Intelligence” feeds. The “parse_src_ip” and “parse_dst_ip” simply tell Sagan what to define as the source and destination; you could easily use liblognorm to do that job more efficiently.  If the log message contains 2 or 20 IP addresses, Sagan will query the Bro Intellegence “Intel::ADDR” data for all of them. The “parse_proto” option tells Sagan to determine the protocol used (UDP, ICMP, TCP, etc) if possible.  

Of course, doing this will use a good bit of CPU!   It's a good thing Sagan is multi-threaded!

Both of these examples are very simple. When “Bro Intelligence” feeds are tied with other rule options like flowbit, GeoIP, after/thresholding, time based alerting, Snortsam (automatic firewalling),  meta_content, etc., you expand your options on detecting threats via log analysis.   A more “serious rule” might look something like this:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg: "[CISCO-BRO] Suspicious connection found in Intel ADDR"; program: %ASA-6-302014;  classtype: suspicious-traffic; normalize: cisco; threshold: type limit, track by_src, count 5, seconds 300; flowbits: set, bro_suspicious_connection, 300; bro-intel: both_ipaddr; fwsam: src, 1 day;  reference: url, wiki.quadrantsec.com/bin/view/Main/9999995; sid: 9999995; rev: 1;)

This example will alert if Sagan receives a log from a Cisco ASA of ASA-6-302014 (TCP session "teardown") and the source or destination is to a place listed in the Bro Intel::ADDR (IP address) database.   If the event happens more than 5 times within a 5 minute period,  we'll threshold the alert.  This rule will let log normalization (via liblognorm) handle the extraction of source/destination IP addresses,  protocols,  source/destination ports, etc. If this event triggers,   Sagan can,  if configured,  automatically block the traffic via Snortsam.  Sagan will also set a flowbit named “bro_suspicious_connections” which can be used by other rules to “tie” events together.  

For more information about The Bro Intellegence Framework,  Critical Stack and Sagan rule option,  go to:

Sagan "bro-intel" option:

https://wiki.quadrantsec.com/twiki/bin/view/Main/SaganRuleReference#bro_intel_src_ipaddr_dst_ipaddr

To see all Sagan rule options, go to:

https://wiki.quadrantsec.com/twiki/bin/view/Main/SaganRuleReference

Bro Intelligence Framework: 

https://www.bro.org/sphinx-git/frameworks/intel.html

For more information and news about Critical Stack:

https://www.criticalstack.com

https://intel.criticalstack.com

We also have pre-built bro-ids.rules available in Sagan.

“Bro Intelligence” support in Sagan is very new and currently (02/20/2015) only available in the source code hosted on our Github Sagan Repo. Over the next few weeks, we will be adding “Bro Intelligence” options into the Sagan rule sets.  

Keep an eye out for them!

Posted in Sagan Blog Post