How to Use OSSIM / USM Active Lists with Python Scripts

Hello, dear Alien Nation and all the community!

What I want to explore today is the so-called “active lists” functionality and how it can be implemented in USM/OSSIM. Let me explain what I call an “active list” with the following example.

Let’s say we have some application (A) to which users log in do their work and then log off. The application (A) writes a log message about the person's login and logout events.

Now we need to track which users are logged in to A at the moment so that we can run other correlation rules against them. For example, we may want to know if the person is logging in to some server B while already logged in to A. How do we accomplish this?

Active lists will help us. When a certain event is triggered (in our example it is the user login to A), a value from a field of the event is put into an active list (in our example it is username). Another event (in our example it is user logoff from A) removes a value from the list. And of course, when some rule is triggered (in our case it is user’s login to server B), a check of the username is performed against the active list. Please see the figure below that explains the idea of “active lists” in graphics.

Fig. 1 – Active lists concept

In the following few pages I will explain my idea of implementing the functionality of active lists with the help of a python script, three correlation directives and policies and a custom plugin.

In the following tutorial I am using active lists for tracking user logons and logoffs. But active lists can also be used for tracking IP addresses showing suspicious activity or infected with malware or any other values (filenames, port numbers, anything.)

1. Workflow

Here is how the active lists in my lab environment work (lab environment shown on figure 3):

1) User “root” logs into the server 192.168.2.30 using SSH;

2) The policy “Add to logged users list on BCKP” fires using the directive “User logon on BCKP” and launches the script active_list_manager.py in the following manner:

active_list_manager_py add logins_list $USERNAME

as a result, a file is created with the name “logins_list” and the username is added to this file;

3) A user logs in via SSH to the server 192.168.10.2;

4) The policy “Check logged users list on MAIL” fires using the directive “User logon on MAIL” and launches the script active_list_manager.py in the following manner:

active_list_manager_py check logins_list $USERNAME

If a user is found in the logins_list the script generates a syslog message like this:

Sep 6 15:40:25 siem active_list_log: Match |List:logins_list |Value:root

5) This log message is picked up by the custom plugin “active_list_monitor” and an event shows up in the OSSIM/USM interface like this:

Fig. 2 – Active list match event

So we can create an alert to fire on this type of event;

6) The user, who was logged in to 192.168.2.30 (on step 1) logs out 192.168.2.30;

7) The policy “Remove user from logged users list on BCKP” fires using the directive “User logout from BCKP” and launches the script active_list_manager.py in the following manner:

active_list_manager_py del logins_list $USERNAME

as a result, the line with the $USERNAME is removed from the file “logins_list”.

Fig. 3 – Lab environment

2. Resources

I created the following resources to accomplish the task:

Directives:

• “ User logon on BCKP “;
• “ User logon on MAIL ”;
• “ User logout from BCKP ”;

Policies:

• “ Add to logged users list on BCKP ”;
• “ Check logged users list on MAIL ”;
• “ Remove user from logged users list on BCKP ”;

a python script “active_list_manager.py”;
a custom parser for OSSIM named “active_list_monitor”.

3. Directives

The created directives are shown in the figure below.

The main parameters for the directive “User logon on BCKP” are:

• DATA SOURCE – AlienVault HIDS-authentication_success
• EVENT TYPE – 5501 (which is successful login)

I put the asset “backup” in the field “TO”. This asset has IP 192.168.2.30.

So this directive fires when we get an event of successful user login to the server 192.168.2.30.

The main parameters for the directive “User logout from BCKP” are:

• DATA SOURCE – AlienVault HIDS-syslog
• EVENT TYPE – 5502 (which is logout)

In this case I put the asset “backup” in the field “FROM”. This asset has IP 192.168.2.30.

So this directive fires when we get an event of user logout from the server 192.168.2.30.

The main parameters for the directive “User logon on MAIL” are:

• DATA SOURCE – AlienVault HIDS-authentication_success
• EVENT TYPE – 5501 (which is successful login)

I put the asset “mail” in the field “TO”. This asset has IP 192.168.10.2.

So this directive fires when we get an event of successful user login to the server 192.168.10.2.

Fig. 4 - Directives

4. Policies

The Policies are meant to execute the script “active_list_manager.py” with different parameters when the described directives fire.

When the directive, which is included in the DS Group used in the policy, fires the policy then performs the “policy action”. And the policy action is executing the script with parameters.

The settings for each of the three policies include:

• new DS group based on the directive appropriately (this one triggers the policy);
• appropriate policy action.

The policy “Add to logged users list on BCKP” utilizes the directive “User logon on BCKP“ via the DS group with the same name and “Add user to active list” action. This means that when the directive is fired (a user successfully logs in to BCKP) the action “Add user to active list” takes place.

The policy “Remove user from logged users list on BCKP” utilizes the directive “User logout from BCKP “ via the DS group with the same name and “Remove user from active list” action. This means that when the directive is fired (a user logs out from BCKP) the action “Remove user from active list” takes place.

The policy “Check logged users list on MAIL” utilizes the directive “User logon on MAIL“via the DS group with the same name and “Check user in active list” action. This means that when the directive is fired (a user successfully logs in to MAIL) the action “Check user in active list” takes place.

The policy settings are shown in the figure below. That is the setup of the “Add to logged users list on BCKP”. All the other policies are created in the similar way changing the DS groups and policy actions.

Fig. 5 – Sample policy

4.1 DS Groups

To create a DS Group navigate to “Configuration” – “Threat intelligence” – “Policy” – “Data source”. Press the button “Add new group” in the left upper corner. Input the group name in the corresponding field (in my case the name matches the name of the directive). In the new window press “ADD BY EVENT TYPE” button and input some key word of the directive name you have created before (in my case it is “BCKP”). Press “Search”. In the output table put a tick next to the data source you need and hit “Add Selected” and after that “Update”. See the figure below.

Fig. 6 –DS Group

4.2 Policy actions

To create a policy action go to “Configuration” – “Threat intelligence” – “Policy” – “Actions”. Press the button “New” and fill in the fields as shown on the figure below. This is the example for “Add user to active list” action. For other actions use different parameters listed below.

For all the actions: “Add user to active list”, “Remove user from active list” and “Check user in active list” use the same TYPE – “execute action”. The NAME and COMMAND fields should be different. In COMMENT field you can write whatever you like.

For “Add user to active list” the field COMMAND is “python /usr/share/ossim/scripts/active_list_manager.py add logins_list USERNAME”

The goal of this command is executing the script with parameters that add the USERNAME from the event to the list named “login_list”.

For “Remove user from active list” the field COMMAND is “python /usr/share/ossim/scripts/active_list_manager.py del logins_list USERNAME”

The goal of this command is executing the script with parameters that remove the USERNAME from the list named “login_list”.

For “Check user in active list” the field COMMAND is “python /usr/share/ossim/scripts/active_list_manager.py check logins_list USERNAME”

The goal of this command is executing the script with parameters that check if the USERNAME from the event is in the list named “login_list” and if it is, send a syslog message.

Fig. 7 – Policy action

5. Script “active_list_manager.py”

This script is meant to add, delete or check if a user is in the active list. It can check not only users, but any info, that is in the list (that’s in case you populate your list with IP addresses or something else).

The syntax is the following:

# active_list_manager.py   

So if we use it like this:

# active_list_manager.py add logins_list mario

We are adding the value “mario” (whatever it is) to the list “logins_list”.

If we use the script with “check” action we check the list for the value supplied. If a match is found the script issues a syslog message in LOCAL5 facility. The message has the following format:

  active_list_log: Match |List: |Value:

Example:

Sep 7 15:57:00 siem active_list_log: Match |List:user_list |Value:alex

This message is parsed by the custom plugin described in the next section of the document.

Following is the listing of the script “active_list_manager.py”:

#!/usr/bin/python
import sys
import os
import syslog
listfile="/usr/share/ossim/scripts/"+sys.argv[2]
if len(sys.argv) <> 4:
   print "Quit due to incorrect syntax.
Please check syntax:
active_list_manager.py (add|del|check)  "
   sys.exit()
if sys.argv[1] == "add":
   open(listfile, 'a').close()
   file = open(listfile, 'r+')
   lines = file.readlines()
   file.seek(0)
   file.truncate()
   for line in lines:
      if not line.startswith(sys.argv[3].split(",")[0]):
         file.write((line))
   for key_value in sys.argv[3].split(","):
      file.write((key_value+' '))
   file.write(('
'))
   file.close()
elif sys.argv[1] == "del":
   if ',' in sys.argv[3]:
      print "Quit due to incorrect syntax.
Only one variable allowed for delete action.
Please check syntax:
active_list_manager.py (add|del|check)  "
      sys.exit()
   file = open(listfile, 'r+')
   lines = file.readlines()
   file.seek(0)
   for line in lines:
      if not sys.argv[3] in line.split():
         file.write(line)
   file.truncate()
   file.close()
elif sys.argv[1] == "check":
   if ',' in sys.argv[3]:
      print "Quit due to incorrect syntax.
Only one variable allowed for delete action.
Please check syntax:
active_list_manager.py (add|del|check)  "
      sys.exit()
   file = open(listfile, 'r')
   lines = file.readlines()
   file.close()
   for line in lines:
      if sys.argv[3] in line.split():
         print "found a match in:", line
         syslog.openlog( 'active_list_log', 0, syslog.LOG_LOCAL5 )
         syslog.syslog( ('Match |List:' + sys.argv[2] + ' |Value:' + sys.argv[3]) )
else:
   print "Quit due to incorrect syntax.
Please check syntax:
active_list_manager.py (add|del|check)  "
   sys.exit()

The script should be placed in /usr/share/ossim/scripts. However, it’s no problem to put it in a different directory with a bit of editing. You can run a script from the console to test it like this:

# active_list_manager.py add logins_list mario

This will create a file “logins_list” in the working directory of the script (/usr/share/ossim/scripts) and add “mario” to this file.

6. Custom parser “active_list_monitor”

This custom parser is designed to read the log file which is populated by the script. When a match on an active list is found the script writes a message to syslog LOCAL5 facility. The rsyslog daemon is configured to pass the messages with LOCAL5 facility to a certain file. In this tutorial it is /var/log/active_list_alerts.log

To configure this we need to create a new file (let’s name it active_list_alerts.conf) in /etc/rsyslog.d/ with the following lines in it:

local5.* -/var/log/active_list_alerts.log
& ~

And restart the syslog daemon with /etc/init.d/rsyslog restart.

Here is the listing of cfg file of the custom parser:

# active_list_monitor.cfg
[DEFAULT]
plugin_id=9005
[config]
type=detector
enable=yes
source=log
location=/var/log/active_list_alerts.log
create_file=false
process=rsyslogd
start=no
stop=no
startup=/etc/init.d/rsyslog start
shutdown=/etc/init.d/rsyslog stop
[translation]
Match=1
[active_list - 01]
event_type=event
regexp=(?Pw+s+d+s+d+:d+:d+)s+(?PS+)s+S+s+(?PS+)s+S+:(?PS+)s+S+:(?PS+)
date={normalize_date($date)}
device={resolv($sensor)}
plugin_sid={translate($sid)}
username={$username}
userdata1={$list_name}

You should place this file under /etc/ossim/agent/plugins and name it something like active_list_monitor.cfg. Don’t miss the “cfg” extension – it is important!

Following is the listing of the SQL file which is used to populate the OSSIM / USM DB with the info about the new plugin:

# active_list_monitor.sql
DELETE FROM plugin WHERE id = "9005";
DELETE FROM plugin_sid where plugin_id = "9005";
INSERT IGNORE INTO plugin (id, type, name, description) VALUES (9005, 1, 'Active list monitor', 'Monitoring plugin for active lists functonality');
INSERT IGNORE INTO plugin_sid (plugin_id, sid, category_id, class_id, name) VALUES (9005, 1, NULL, NULL, 'Active list Match detected');

You can create this file in any place of the filesystem. I advise storing this file under /usr/share/doc/ossim-mysql/contrib/plugins/

Populate the OSSIM / USM DB:

# ossim-db < active_list_monitor.sql

Then we need to enable the plugin via webUI or console. Using the console go to “Configure sensor” – “Configure Data Source Plugins”. Choose the new plugin from the list and put a tick next to it. Press “OK” – “Back” – “Apply all changes”.

After the server restarts you are ready to go.

7. Resume

What I achieved through this tutorial is getting a message like the one on the figure below every time when a user currently logged in to server BCKP tries to log into server MAIL.

Fig. 8 – Notification message example

And of course, you can create an alarm to fire every time a message like this appears.