Archive for the ‘SiLK’ tag

Threat intelligence: OTX, Bro, SiLK, BIND RPZ, OSSEC  

Posted at 8:15 am in Uncategorized

Building a toolbox around threat intelligence can be done with freely available tools. Shared information about malicious behaviour allows you to detect and sometimes prevent activity from – and to – Internet resources that could compromise your systems’ security.

I’ve already described how to use lists of malicious domain names in a BIND RPZ (Response Policy Zone). Adding an information feed like AlienVault OTX (Open Threat Exchange) to the mix further extends the awareness and detection capabilities.

AlienVault is probably most known for their SIEM (Security Information and Event Management) named Unified Security Management™, with a scaled-down open source version named Open Source Security Information and Event Management (OSSIM). They also provide a platform for sharing threat intelligence, namely Open Threat Exchange (OTX). OTX is based on registered users sharing security information, for instance domains and hostnames involved in phishing scams, IP addresses performing brute force SSH login attempts, etc. The information is divided into so-called pulses, each pulse a set of information items considered part of the same malicious activity. For example, a pulse can contain URLs to a site spreading drive-by malware, the IP addresses of their C&C, along with hashes of the files. By selecting which pulses and/or users to subscribe to, the registered information in each pulse will be available through a feed from their API.

Carefully reviewing which users/pulses to subscribe to – there’s always a risk of false positives – I’m now regularly receiving an updated feed. This feed is parsed and currently split into two files: One RPZ file containing hostnames and domains for use with BIND, and one file containing IP addresses for use with SiLK.

As explained in an earlier post, OSSEC will let me know if someone (or something) makes DNS requests for a domain or hostname registered as malicious. Extending this to include the DNS records obtained from OTX was simply a matter of defining a new RPZ in BIND. Depending on how this is used (block? redirect? alert?), a whitelist should be in place to prevent accidental blocking of known good domains. One pulse describes all the Internet resources a client infected by a certain exploit will contact, including some certificate authorities which are not necessarily considered evil.

The file with IP addresses can be used directly with a firewall, by logging or even blocking or throttling traffic to/from the IP addresses in question. For rear-view mirror analysis it can be used with SiLK, to find out if there has been any network traffic to or from any of these addresses. To do this, you will first have to create an IP set with the command rwsetbuild:

# rwsetbuild /some/path/ip-otx.txt /some/path/ip-otx.set

 

Now we can use this set file in our queries. For this query I’ve manually selected just a few inbound matches:

# rwfilter --proto=0-255 --start-date=2016/01/01 \
  --sipset=/some/path/ip-otx.set --type=all \
  --pass=stdout | rwcut -f 1-5
            sIP|            dIP|sPort|dPort|pro|
   185.94.111.1|  my.ip.network|60264|   53| 17|
   216.243.31.2|  my.ip.network|33091|   80|  6|
   71.6.135.131|  my.ip.network|63604|  993|  6|
   71.6.135.131|  my.ip.network|60633|  993|  6|
   71.6.135.131|  my.ip.network|60888|  993|  6|
   71.6.135.131|  my.ip.network|32985|  993|  6|
   71.6.135.131|  my.ip.network|33060|  993|  6|
   71.6.135.131|  my.ip.network|33089|  993|  6|
   71.6.135.131|  my.ip.network|33103|  993|  6|
   71.6.135.131|  my.ip.network|33165|  993|  6|
   71.6.135.131|  my.ip.network|33185|  993|  6|
   71.6.135.131|  my.ip.network|33614|  993|  6|
   71.6.135.131|  my.ip.network|33750|  993|  6|
   71.6.135.131|  my.ip.network|60330|  993|  6|
  185.130.5.224|  my.ip.network|60000|   80|  6|
  185.130.5.224|  my.ip.network|60000|   80|  6|
  198.20.99.130|  my.ip.network|    0|    0|  1|
  185.130.5.201|  my.ip.network|43176|   53| 17|
  129.82.138.44|  my.ip.network|    0|    0|  1|
  185.130.5.224|  my.ip.network|60000|   80|  6|
  185.130.5.224|  my.ip.network|60000|   80|  6|

 

When you need more details about the listed address or other indicators, OTX provides a search form to find the pulse(s) in which the indicator was registered.

OTX can be used with Bro as well, and there are at least two Bro scripts for updating the feeds from the OTX API. The one that works for me is https://github.com/hosom/bro-otx. The script will make Bro register activity that matches indicators from an OTX pulse.

Sample log entries, modified for readability:

my.ip.network 59541 some.dns.ip    53 - - - union83939k.wordpress.com
                                            Intel::DOMAIN DNS::IN_REQUEST
my.ip.network 40453 54.183.130.144 80 - - - ow.ly
                                            Intel::DOMAIN HTTP::IN_HOST_HEADER
74.82.47.54   47235 my.ip.network  80 - - - 74.82.47.54
                                            Intel::ADDRConn::IN_ORIG

 

This article mentions just a few components that can be combined. Obviously there’s a lot of possibilities for integrating and interfacing between different systems. There are several companies that provide threat intelligence feeds, some for free and some for paying customers. Depending on the product(s), a SIEM would be able to combine and correlate the different kinds of threat intelligence to detected events.

Written by bjorn on March 9th, 2016

Tagged with , , , , , , , , , ,

Geomapping network traffic  

Posted at 11:28 pm in Uncategorized

Did you ever wonder where your network traffic goes (and originates from)? With the SiLK suite and optionally some JavaScript map classes it’s quite easy to find out.

SiLK is a tool quite equal to Cisco‘s NetFlow, and SiLK does indeed accept NetFlow output from a router. Just like NetFlow tools, SiLK stores network traffic metadata (like “when” and “where”, but not “what”), so as opposed to capturing the complete network traffic SiLK can store a lot of information over a long time without eating too much disk space. In my setup I’ve configured my Mikrotik router to transmit traffic flow data to a Linux server running SiLK.

With GeoIP mapping, SiLK can identify the country of source and destination IP addresses. Combined with a “top 20” construct, it turned out easier than expected to create a world map like this:

Network traffic world map

For the map I’ve been using the very useful JavaScript interactive Highchart maps (Highmaps) from the Norwegian company Highsoft. To feed the map with data I wrote a small piece of code that converts the output from SiLK’s rwfilter/rwstats output to JSON, which makes the map dynamically update itself upon refresh.

World map percentage from SwitzerlandAs shown on the screenshot to the left, when hovering the mouse over each bubble the JavaScript map code will show the percentage value of the traffic associated with the different countries. When identifying traffic from unexpected sources you can use the command line based SiLK tools to drill down in order to find out what’s really going on, like in this case when I was wondering what was being transferred from Switzerland (it turned out to be some Flash movies the kids were watching). In addition to the command line tools there are also GUI and web based interfaces for querying SiLK data. The command used to find the source IP and port for the traffic originating from Switzerland (.ch) is shown below:

# rwfilter --start-date=2015/10/09 --end-date=2015/10/09 \
 --proto=0-255 --type=all --pass=stdout --scc='ch' | rwstats --top \
 --count=5 --fields=sip,sport --value=bytes
INPUT: 327 Records for 19 Bins and 39783096 Total Bytes
OUTPUT: Top 5 Bins by Bytes
          sIP|sPort|               Bytes|    %Bytes|   cumul_%|
80.239.148.37| 1935|            38839614| 97.628435| 97.628435|
  62.48.3.112|   80|              476083|  1.196697| 98.825132|
80.239.148.47| 1935|              402414|  1.011520| 99.836652|
80.239.148.44|   80|               15276|  0.038398| 99.875050|
80.239.148.45|  443|                9328|  0.023447| 99.898497|

 

There’s a few snags with my setup, but with some tweaking it gives a general idea and I think I’ve handled the corner cases. These are the knows issues – so far:

  • This setup is a NATed IPv4 network environment. In order to detect inbound traffic I have to enter my router’s outside address as the destination. This address might change now and again, breaking historical data.
  • I’m also running an IPv6 network provided by HE.net (tunnelbroker.net). My allocated IPv6 ranges are consequently registered as physically located in the US, so SiLK will register any internal-only IPv6 network traffic as based in the US, increasing that percentage.

Written by admin on October 9th, 2015

Tagged with , , , , , ,