Time-based alerts for snort

Snort is a very powerful intrusion detection system, and I've learned that when you combine it with a little kung fu you can do amazing things with it.  You can write incredibly granular alerts that will catch almost anything while minimizing false positives.  However, there is one thing missing in the alerting functionality and that is the ability to specify that some alerts only fire at certain time.

For example, let's say that you maintain a customer portal for your supply customers to order new shipments of some product.  You can expect that you will have customers hitting this page and trying to log during normal business hours so you probably wouldn't want to alert on that.  However, the same kind of traffic happening at 11:30 at night might seem suspicious.  At that time, events that might not be terribly interesting (like five failed logon attempts) suddenly become much hotter.

So how can we get time based alerts in snort?  Well there are two ways that I can think of and hopefully someone can think of a few more.  The first is the less-than-elegant solution.  You can write a script that adds a few rule lines to your local.rules file at certain times of the day and sends a kill -1 to your snort process so that it will reprocess it's rules files.  The advantage of using this process is that it is the simplest way of doing it. One way of doing it would be to have two different versions of your local.rules file. Then write a cron job that switches the two files and sends the kill -1 to the snort process. The disadvantage to this process is that it isn't very flexible. Let's say you want to ignore web activity between 8am and 5pm. Well that is pretty easy using this approach. But let's also say that on a different system you want to ignore activity from a block of IP addresses during a different time period. Now you would need to have three and possibly four versions of your local.rules file and your cron job would start to get pretty complicated. You would also need to make sure that you keep your script updated with the process ID that snort is running under. If you restart the service for some reason you're going to need to update your scripts. So I didn't like this approach.

Here is what I ended up doing to get these time-based alerts. I'll describe it in a nutshell, and then go into the detailed process. First, I set up a named pipe in the log directory that snort writes to. Then I created another type of rule called OddHours and told snort to write the alert for these rules into the named pipe I created. Then I wrote a python script that monitors the named pipe and takes an action whenever a new alert is found. It checks the day of the week and the time of day and then makes a decision to either send me an email or do nothing. Last step was to write the snort rules that I was looking for.

OK, so first is the named pipe. I have a lot of difficulty explaining what a named pipe is, but basically it is a way of sending the output from one program into another. For you command line linux types you probably do this all the time, like when you pipe the output of ps into grep to find your snort process ID.

ps aux | grep snort

That | symbol is doing the same thing as our named pipe, but it doesn't have a name and it isn't persistent on the file system the way our named pipe is. So let's create it:

mkfifo oddhours.csv

Now the custom alerts. In the snort.conf file that comes with snort, you can do a search for redalert and find an example of doing this. The custom alert is just a way of telling snort that for some alerts you want to react differently that you usually do. So I added this in underneath the redalert example:
ruletype oddhours

{
type alert
output alert_csv: oddhours.csv timestamp,sig_id,msg,proto,src,dst,dstport
}

So now if snort sees traffic that matches a rule that is written to use this custom ruletype it will send the time of thealert, the signature ID, the message, protocol, source and destination information into our named pipe.

This would be a bad time to restart snort if you're following along with me. The thing about named pipes is that you must have both ends of the pipe wired up for them to work. In the next step we'll create a script that "listens" to the pipe but if you don't have that script running when you start snort, then snort will sit there and wait for the script to start.

OK, let's dig into that python script. This is pretty easy. Just read a line of text from the named pipe and check what time it is. Right now the script is just writing to a log file, but I've put in the logic to send the email message that I ultimately want to receive.

import time
import smtplib
from email.mime.text import MIMEText
import logging

#######################
# Here is the logging setup stuff
#######################
LOG_FILENAME = '/home/kevin/oddoutput.txt'
logging.basicConfig(filename=LOG_FILENAME, level=logging.INFO)
#######################
# Here is the SMTP setup stuff
#######################
smtpServer = 'smtp.domain.edu'
smtpFrom = 'email.address@domain.edu'
smtpTo = 'email.address@domain.edu'

#######################
# Here i a function for sending alert emails
#######################
def SendAlertEmail(inMessage):
    logging.debug('\tSending Email alert')
    msg = 'Snort has detected activity at an unusual hour.\n'
    msg += inMessage
    msg = MIMEText(msg)
    msg['Subject'] = 'Activity at an unusual time'
    msg['From'] = smtpFrom
    msg['To'] = smtpTo

    s = smtplib.SMTP(smtpServer)
    s.sendmail(smtpFrom, smtpTo, msg.as_string())
    s.quit
    logging.debug('\tMessage sent.')


infile = open('/home/kevin/snort/logs/oddhours.csv','r')
while True:
    data = infile.readline()
    if data:
        logging.info('Line of data received.')
        logging.debug('\tChecking if it is Saturday.')
        if time.strftime( '%a' ) == 'Sat':
            logging.critical('\t\tIt\'s Saturday!  ALERT ALERT')
            logging.critical('\t\t'+data)
            #SendAlertEmail(data)
            continue
        else:
            logging.debug('\t\tNope. It\'s '+time.strftime('%a'))
        logging.debug('\tChecking if it is Sunday.')
        if time.strftime( '%a' ) == 'Sun':
            logging.critical('\t\tIt\'s Sunday!  ALERT ALERT')
            logging.critical('\t\t'+data)
            #SendAlertEmail(data)
            continue
        else:
            logging.debug('\t\tNope.It\'s '+time.strftime('%a'))
        logging.debug('\tChecking if it is early in the morning.')
        if int(time.strftime( '%H' )) <= 7:
            logging.critical('\t\tIt\'s Early.  ALERT ALERT')
            logging.critical('\t\t'+data)
            #SendAlertEmail(data)
            continue
        logging.debug('\tChecking if it is late in the day.')
        if int(time.strftime( '%H' )) >= 17:
            logging.critical('\t\tIt\'s Late.  ALERT ALERT')
            logging.critical('\t\t'+data)
            #SendAlertEmail(data)
            continue
        logging.debug('\tPassed all tests.')
        logging.debug('\t'+data)

OK. Last thing we need to do is write some rules that make use of our custom ruletype. In this example, I want to see web surfing that happens on Saturday or Sunday or outside of the normal work day. So my rule is going to look for anything on my home_net that is going to port 80 or 443 with the Syn flag set. Keep in mind that I wouldn't do this on a production system unless you want a lot of email or you're sure that this doesn't happen. I'm using this example because it fires a lot of alerts and lets you check right away if your rule is working.

oddhours tcp $HOME_NET any -> any 80 (msg:"Surfing at an odd hour."; flags:S; sid:2009122301; rev:1;)
oddhours tcp $HOME_NET any -> any 443 (msg:"Surfing at an odd hour."; flags:S; sid:2009122302; rev:1;)

OK. Now we're ready to fire the whole thing off. First, start up the python script and set it to run in the background: python oddhours.py &

Then fire up snort and start watching the output file. You can change the logging level so that you can get more or less information from the script if you want to do some troubleshooting.

Later on down the road, you can create functions for each sid that you want to monitor. When a line of data comes in you can pass control over to the function that you've written for that sid. That way you can have different time-based rules for each sid. It would also be pretty easy to expand the script so that you could send email alerts to different people based on the sid.

And hey, if you know a better way to do this then please hook me up with the answer. I'll gladly post it on my blog and give you all the credit.