Tutorial: Create a Plugin for Microsoft Exchange

Applies to Product: USM Appliance™ LevelBlue OSSIM®

In this tutorial, we use Microsoft Exchange to show how to develop a log plugin. The preferred method of collecting logs from the Exchange Server is through NXLog. See Microsoft Exchange Server through NXLog for details.

Plugin Development Steps

  1. Examine the log file from the data source from which you want to create a plugin. Identify all the types of log messages, as well as messages sharing a common structure, but using different values.
  2. Create the <filename>.cfg file, either by writing a new file or by copying an existing and similar file, then rewriting it.
  3. Give the plugin a numeric ID. (See Creating a Plugin Configuration File for available values.)
  4. Specify the location of the file from which the plugin should read.
  5. Write regular expressions to parse individual messages from the log file.
  6. Test your regular expressions to see if they perform as they should, using a testing tool such as the one available at regex101.com.
  7. Create the .sql file by copying an existing and similar .sql file. Change the fields to describe events included in the custom plugin.

  8. Write the .sql file to the SIEM database.

  9. Enable the plugin through either the LevelBlue Setup menu, the USM Appliance web UI, or a USM Appliance asset. (See Enable Plugins.)
  10. Test the plugin by sending logs from the data source to USM Appliance. (See Verify that an Enabled Plugin Is Working Properly.)

This task creates a plugin configuration file for a data source called "exchangews," and which uses SNMP for data transfer.

To create a plugin configuration file

  1. (Optional) Use an existing plugin as a template for the new one by copying an existing plugin file like SSH.cfg and renaming it exchangews.cfg.
  2. Write the new plugin configuration settings:

    1. Change the plugin_id field, using any value in the range from 9001 to 2147483647 except for the following:

      90003, 90005, 90007, 90008, 10002, 12001, 19004, 19005, 19006, 20505

      Note: Because you have copied SSH.cfg, you do not need to create a header. If you created a file from scratch, you would need to create one at this juncture. See The Plugin File Header.

    2. Change location to point to the log file /var/log/exchangews.log.
    3. Delete the startup and shutdown fields. These fields are not going to be used. There is no application associated with this plugin.
    4. (Optional) Create a new translation table.

      5. A translation table translates a string to a number so that it can be used as a plugin_sid.

      HELLO=1

      MAIL=2

      RCPT=3

      DATA=4

      QUIT=5

      xxxx=6

      DEFAULT_=9999

    5. Create new rules, filling up the fields below.
    6. Create two regular expressions to parse the data, because there are two different formats in the log file.

      [exchangews - Generic rule]

      #2011-10-09 05:00:15 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 HELO - +36A42160 250 0 48 13 0 SMTP - - - -

      #2011-10-09 05:00:16 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 MAIL - +FROM:+<test@sample1.com> 250 0 57 45 0 SMTP - - - -

      event_type=event

      regexp="(?P<date>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})\s(?P<src_ip>\IPV4)\s(?P<userdata2>\S+)\s(?P<hostname>\S+)\s(?P<userdata3>\S+)\s(?P<dst_IP>\IPV4)\s\d\s(?P<type>\w+)"

      date={normalize_date($date)}

      plugin_sid={translate($type)}

      dst_ip={resolv($dst_ip)}

      src_ip={resolv($src_ip)}

      hostname={$hostname}

      userdata2={$userdata2}

      userdata3={$userdata3}

      [exchangews = Generic rule 2 NCSA Format]

      #1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250 46

      #1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:41 -0600] "MAIL -?+FROM:+<Keith@testdomain.com> SMTP" 250 46

      event_type=event

      regexp="(?P<src_ip>\IPV4)\s-\s(?P<dst_ip>\S+)\s\[(?P<date>\d\d\/\w{3}\/\d{4}:\d\d:\d\d:\d\d)\s-\d{4}\]\s\"(?P<type>\w+)"

      date={normalize_date($date)}

      plugin_sid={translate($type)}

      dst_ip={resolv($dst_ip)}

      src_ip={resolv($src_ip)}

    7. Check regular expressions with logs inside the file /var/log/exchangews.log.

      8. There are several utilities on the Internet to test regular expressions written in Python. It is recommended to use one of these utilities to check that the created regular expressions match the logs.

      Note: The location parameter is limited to 100 files.

The following example shows the plugin .sql file corresponding to the plugin configuration file example.

INSERT INTO plugin (id, type, name, description) VALUES (9001, 1, 'exchangews', 'Exchange E-mail Web server');

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 1, NULL, NULL, 'exchangews: HELO' ,3, 2);

INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 9999, NULL, NULL, 'exchangews: Generic exchange event' ,3, 2);

USM Appliance must store all the plugin IDs and event types in its database before it can store any events. For this reason, if you develop a new plugin and you don't first update the database with that data, the USM Appliance Server drops those events, even though the plugin is working correctly.

To update the SIEM database

  1. Write the changes to the SIEM database:

    cat exchangews.sql | ossim-db

  2. Apply changes in the SIEM:

    ossim-server restart