Customize Existing Plugins Yourself

After enabling a plugin, the USM Appliance Sensor processes the plugin's log data and sends events it collects to the USM Appliance Server, where they are stored in the SIEM database. You can then view the events in the USM Appliance web UI by selecting the Analysis > Security Events (SIEM) option.

After using the plugin for a while, you may find a number of events that you don't find useful to display or track, and just create noise and take up space in the SIEM database. Rather than creating USM Appliance policies to filter out these events, which incurs some processing overhead, you can include the exclude_sids parameter in a .local copy of the plugin's configuration file to achieve the same result.

To exclude collection of specific event type IDs from a plugin

1. Connect to USM Appliance through SSH.
2. On the LevelBlue Setup menu select Jailbreak System.
3. If the file does not already exist, create the file /etc/ossim/agent/plugins/<plugin_name>.cfg.local.

4. In the <plugin_name>.cfg.local file, add the exclude_sids parameter in the [CONFIG] section and specify one or more event type IDs (separated by commas) that you don't want the sensor to process. For example:

   [CONFIG]

   exclude_sids=200,302,404,403

5. Save the file and restart ossim-agent:

   /etc/init.d/ossim-agent restart

The SIDs you specify with the exclude_sids parameter are the Event Type IDs that USM Appliance assigns to each event that matches a specific rule in the plugin configuration file. The quickest way to locate the SID number of a specific event type is to go to the Analytics > Security Events (SIEM) page, and click on the row of a particular event you want to exclude. From the Event Detail display that is shown, you can locate the Event Type ID associated with the event. Use that value for the exclude_sids parameter in the plugin's configuration file.

Another way, in which you can view all the event type IDs associated with every plugin, is to go to the Configuration > Threat Intelligence page and select the Data Source tab. From there, each row of the display lists information about a plugin, and you can double-click on a specific row to view all the associated event type IDs for the selected plugin.

If you have a plugin that processes logs from an asset located in a different timezone than your sensor, sometimes due to the source device using an incorrect timezone or a timezone that cannot be changed, you can modify the timezone in the plugin file instead. The following task shows how to do this using the SSH plugin as an example.

To change the timezone used by a plugin

1. Connect to USM Appliance through SSH.
2. On the LevelBlue Setup menu select Jailbreak System.

3. Check the default timezone that ossim-agent uses.

   ~# grep tzone /etc/ossim/agent/config.cfg

   tzone=Europe/Dublin

   This should match the timezone of where your USM Appliance Sensor resides.

4. Create the file /etc/ossim/agent/plugins/ssh.cfg.local.

5. In ssh.cfg.local, add the following parameter:

   [DEFAULT]

   tzone=Australia/Melbourne

   where

   Australia/Melbourne represents the timezone of the asset you are collecting logs.

   Note: USM Appliance validates timezones against this list.

6. Save the file and then restart ossim-agent.

   /etc/init.d/ossim-agent restart

7. Check the agent log to make sure the new timezone has been applied:

   ~# grep -A1 "Starting detector" /var/log/alienvault/agent/agent.log

   2016-01-18 10:39:38,948 AlienVault-Agent [INFO]: Starting detector ssh (4003)..Plugin tzone: Australia/Melbourne

   2016-01-18 10:39:38,950 AlienVault-Agent [WARNING]: Using custom plugin tzone data: Australia/Melbourne

Different devices often use different date and time formats in their logs. USM Appliance solves this issue by providing a built-in function, normalize_date(), which converts different date formats to ISO 8601, the format accepted by the USM Appliance Server. For a list of formats that the function recognizes, see Supported Formats by the normalize_date() Function.

If the date and time format is not supported by the normalize_date() function, for example, DD/MM/YYYY HH:MM:SS, you can create a custom function to handle the specific date and time format that you need.

Note: Existing date formats are defined in the date_formats.json file. Modifying this file is NOT recommended, since USM Appliance updates will overwrite any changes you make to the file.

The following example provides a template for a custom function named normalize_date_not_american(), in which you can define the patterns of date and time you want to format.

#
# Description:
#   This function should only be called when a date 
#   is in this format:
#   12/08/2016 21:44:47
#   dd/mm/yyyy hh:mm:ss
#
# Usage:
#   date={:normalize_date_not_american( $date_log )}
#
Start Function normalize_date_not_american
from re import compile
from datetime import datetime
def normalize_date_not_american( self, string = "" ):
  pattern = compile( r'(?P<day>\d{1,2})/(?P<month>\d+)/
    (?P<year>\d{4})\s(?P<hour>\d+):(?P<minute>\d+)
    :(?P<second>\d+)' )
  result = pattern.search(string)
  groups = result.groupdict()
  date = datetime(year=int(groups['year']), 
    month=int(groups['month']), day=int(groups['day']), 
    hour=int(groups['hour']), minute=int(groups['minute']), 
    second=int(groups['second'])).isoformat(' ')
  return date
End Function

After creating and saving the custom function, you can use it in the plugin configuration file. For more information on using custom functions in plugins, see Define and Use Custom Functions.

In addition to the built-in functions that USM Appliance supports, you may also create your own custom functions to convert extracted data to the format required by specific event field types.

To create a new custom function

1. In the Config section, specify the location of a new file containing one or more new function definitions.

   [config]

   custom_functions_file=/etc/ossim/agent/plugin/custom.cfg

2. Create a text file that matches the custom function's file name. In the new file, define the operation of each new custom function you want to create. Each new function must start with Start Function <function_name> and end with End function.

   The following example shows the definition of two new functions , log_hello and log_hello_data.

   Start Function log_hello
      def log_hello(self):
      return "Hello log!"
   End Function
    
   Start Function og_hello_data
     def log_hello_data(self,data):
     return "Hello log: %s" % data
   End Function

   In this example, the log_hello() function returns the string "Hello log!" and the value assigned to a normalized event field. The second function returns the "Hello log!" string concatenated with the extracted user name value.

3. Save and close the custom function file.

   Note: Be careful if you add a custom function to a plugin or if you access a proprietary database. This can degrade performance if not well designed. Custom plugins might take up to five minutes to appear in the USM Appliance web UI after you add them.

4. To use the new custom functions, you can simply include as part of any event field assignment corresponding to extracted data returned for a specific rule or log event. For example

   [ssh - Failed password]

   # Feb 8 10:09:06 server1 sshd[24472]: Failed password for dgil from

   192.168.6.69 port 33992 ssh2

   event_type=event

   regexp="(\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<sensor>\S*).*ssh.*Failed

   password for

   (?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*port\s+(?P<sport>\d{1,5})"

   plugin_sid=1

   sensor={resolv($sensor)}

   date={normalize_date($1)}

   src_ip={$src}

   dst_ip={resolv($sensor)}

   src_port={$sport}

   username={$user}

   userdata1={:log_hello()}

   userdata2={:log_hello_data($user)}

Important: You are not allowed to use a custom function in a built-in function, for example, translate(:log_hello()), as custom functions are the last functions to be executed in a rule.

The following task shows how to change an existing rule and add a new rule to a plugin, in this case, for Cisco ASA.

To change or add a rule to an existing plugin

1. Connect to USM Appliance through SSH.
2. On the LevelBlue Setup menu select Jailbreak System.

3. Create the file /etc/ossim/agent/plugins/cisco-asa.cfg.local.

4. (As needed) Change any existing regex mappings or rules needed for the re-purposed plugin.

   The following example shows the changed rule, in bold, as the value of the regexp parameter. The commented section shows the mapping target of the regexp parameter.

   #Sep 17 06:25:31 5.5.5.5 : Sep 17 06:26:10 EDT: %ASA-ids-4-401004: #Shunned packet: 1.1.1.1 ==> 1.1.1.2 on interface inside

   [0001 - cisco-asa - Shunned packet]

   regexp="(?P<date>\w{3}\s+\d{1,2}\s(\d{4}\s)?\d\d:\d\d:\d\d)\s*:?((?P<device>\S+)\s*:?)?\s+[\w\:\s]*%?(?P<asa_short_msg>(asa|ASA)-[\w\-]*-?(\d+)-(?P<sid>\d+)):\s+(?P<userdata>[^:]+:\s+(?P<src_ip>\S+)\s+==>\s+(?P<dst_ip>\S+).*)"

5. (As needed) Add any new rule needed for the revised plugin, as shown in the following example. The commented section shows the mapping target of the regexp parameter.

   #Mar 28 12:03:04 testbox %ASA-7-609002: Teardown local-host inside:1.1.1.1 duration 0:02:12

   [9000 - cisco-asa - Teardown local-host]

   precheck="Teardown"

   regexp="(?P<date>\w{3}\s+\d{1,2}\s(\d{4}\s)?\d\d:\d\d:\d\d)\s*:?((?P<device>\S+)\s*:?)?\s+[\w\:\s]*%?(?P<asa_short_msg>(asa|ASA)-[\w\-]*-?(\d+)-(?P<sid>\d+)):\s+(?P<userdata>(?:Teardown\slocal-host\s)(?P<iface>[^:]*):(?P<src_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\S+\s(?P<duration>\S+).*)"

   event_type=event

   date={normalize_date($date)}

   device={resolv($device)}

   plugin_sid={$sid}

   src_ip={$src_ip}

   userdata1={$asa_short_msg}

   userdata2={$iface}

   userdata3={$duration}

   userdata4={$userdata}

6. Save the file and then restart ossim-agent:

   /etc/init.d/ossim-agent restart

7. Check regular expressions and also field assignments within the file /var/log/cisco-asa.log.

   We recommend that you use any of the utilities available on the Internet to test that the Python regular expressions you added match the logs.

After creating a new rule, you will also need to define a new event type ID to assign to events matching the new rule.

However, if you have many new event types to add, you should use the plugin .sql file instead, because it is much faster. The web UI is better when you have only one or two event types to add.

Note: If you do not perform this task, these events will not appear in the Security Events view in the USM Appliance web UI.

To add new event types

This procedure adds a custom event type to an existing or newly developed plugin, using the USM Appliance web UI.

1. Go to Configuration > Threat Intelligence > Data Source.
2. Double-click a data source to open it.

3. Click Insert New Event Type.

   The Add New Event Type dialog box appears.

4. Fill in the fields, make value selections from the lists, and then click Update.

Values with an asterisk (*) are required.