Configure Log Plugins

Applies to Product: USM Appliance™ LevelBlue OSSIM®

Log plugins extract events from log files by matching each line in a log file using a regular expression. The plugin then normalizes the information to create events containing the data fields from the text.

 

# Plugin ssh id:4003 version: 0.0.2

# Last modification: 2015-05-13 16:11

#

# Plugin Selection Info:

# OpenBSD:OpenSSH:-

#

# END-HEADER

# Accepted products:

# openbsd - openssh 5.4

# openbsd - openssh 5.5

# openbsd - openssh 5.6

# openbsd - openssh 5.7

# openbsd - openssh 5.8

# openbsd - openssh 5.8p2

# openbsd - openssh 5.9

# Description:

 

[DEFAULT]

plugin_id=4003

dst_ip=\_CFG(plugin-defaults,sensor)

dst_port=22

 

[config]

type=detector

enable=true

source=log

location=/var/log/auth.log

create_file=true

process=sshd

start=no

stop=no

startup=/etc/init.d/ssh start

shutdown=/etc/init.d/ssh stop

 

[translation]

none=1

opened=25

publickey=2

version=22

throughput=23

closed=26

password=1

 

[0000 - Failed password]

event_type=event

regexp=(?P<date>\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<dst>\S+) sshd\[\d+\]: Failed password for\s(?P<info>invalid user\s)?(?P<user>\S+)\sfrom\s(?P<src>\S+)\sport\s(?P<sport>\d{1,5})

date={normalize_date($date)}

plugin_sid=1

src_ip={resolv($src)}

dst_ip={resolv($dst)}

src_port={$sport}

username={$user}

userdata1={$info}

userdata2={$dst}

device={resolv($dst)}

 

[0001 - Invalid user]

event_type=event

regexp=(?P<date>\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s(?P<dst>\S+)\ssshd\[\d+\]: Invalid user (?P<user>\S+) from\s+(?P<src>\S+)

date={normalize_date($date)}

plugin_sid=3

src_ip={resolv($src)}

dst_ip={resolv($dst)}

username={$user}

device={resolv($dst)}

.

.

.

<Additional rule matching Regex expressions added, as needed>

 

Understanding the Plugin File

Every plugin monitors a different log file for new syslog messages. If the plugin is enabled at the sensor level, this log file is defined in the location parameter under the [config] section. For example

[config]

...

location=/var/log/auth.log

Log plugins extract events from logs by matching each line in the log according to a regular expression. The plugin then normalizes the data fields from the text. For example, when a log message arrives, as shown

Feb 8 10:09:06 server1 sshd[24472]: Failed password for dgil from 192.168.6.69 port 33992 ssh2

The SSH plugin matches it with a regular expression (regex) in the rule of

regexp=(?P<date>\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<dst>\S+) sshd\[\d+\]: Failed password for\s(?P<info>invalid user\s)?(?P<user>\S+)\sfrom\s(?P<src>\S+)\sport\s(?P<sport>\d{1,5})

As soon as a rule matches a log line, matching stops, no matter how many remaining rules may match. The regular expression also extracts the relevant information from the matched log line. The regex fields, shown in boldface in the above example, identify the text to be mapped to the Security Event fields.

As a second step, the plugin normalizes that information for presentation within the USM Appliance Security Event view.

Date = Feb 8 10:09:06

src_ip =192.168.6.69

Username = dgil

The data source log format dictates the level of detail needed to generate events. The data source could require either just a few rules or one particular rule for each event.

The field plugin_sid identifies each individual event. This field is assigned either to every rule or it can be based on a field captured from a log line.

[0000 - Failed password]

event_type=event

regexp=(?P<date>\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<dst>\S+) sshd\[\d+\]: Failed password for\s(?P<info>invalid user\s)?(?P<user>\S+)\sfrom\s(?P<src>\S+)\sport\s(?P<sport>\d{1,5})

date={normalize_date($date)}

plugin_sid=1

src_ip={resolv($src)}

dst_ip={resolv($dst)}

src_port={$sport}

username={$user}

userdata1={$info}

userdata2={$dst}

device={resolv($dst)}

Configure the USM Appliance Sensor to Receive Logs Through Syslog

Important: This task is only required if you enable the Log plugin through Enable Plugins from the Sensor Configuration. LevelBlue strongly recommends that you enable Log plugins through assets for ease of use and maintenance, unless you want to use the same plugin for a large number of devices.

For text logs received through the rsyslog service running on USM Appliance, you need to define the syslog routing rules in the rsyslog configuration file, located in /etc/rsyslog.d/. You also need to add a configuration file for logrotate, located in /etc/logrotate.d/, to rotate the logs.

To add rules for rsyslog and logrotate

  1. Connect to the LevelBlue Console through SSH and use your credentials to log in.

    The LevelBlue Setup menu displays.

  2. On the LevelBlue Setup main menu, select Jailbreak System to gain command line access.

    Select Yes when prompted. You will be in the root directory.

  3. Create a new configuration file to filter incoming logs. For example,

    nano –w /etc/rsyslog.d/01_<dataSource_name>.conf

    Where <dataSource_name> is the name of the plugin. The prefix of 01_ ensures that the file is processed before the default USM Appliance configurations.

  4. Add the following line to the configuration file to identify the devices from which you should receive logs.

    if ($fromhost-ip == ‘<IP_Address_1>’) or ($fromhost-ip == ‘<IP_Address_2>’)

    then <path>/<dataSource_name>.log
    & stop

    Where

    • <path>/<dataSource_name>.log matches the file listed in the location parameter of the plugin file
    • <IP_Address_1> is the IP address of the first device and <IP_Address_2> is the IP address of the second device.
    • If you want to receive logs from more devices in different subnets, add more 'or' clauses using the same syntax, ($fromhost-ip == ‘<IP_Address>’).
    • If you want to filter for a subnet or a range of IP addresses, you can use the ($fromhost-ip startswith ‘<partial_IP>’) syntax. For example, ($fromhost-ip startswith ‘192.0.1.’).
    • You can also use($fromhost == ‘<hostname>’) if DNS resolution is enabled in your network.
  5. Save the file by pressing Crtl+W and exit the editor by pressing Crtl+X.

  6. Restart the Syslog Collector.

    /etc/init.d/rsyslog restart

    The USM Appliance Sensor should now process the incoming logs as soon as you enable the plugin.

  7. Create a new logrotate configuration file.

    nano –w /etc/logrotate.d/<dataSource_name>

  8. Add the following lines of code to the file

    <path>/<dataSource_name>.log

    {

    # save 4 days of logs

    rotate 4

    # rotate files daily

    daily

    missingok

    notifempty

    compress

    delaycompress

    sharedscripts

    # run a script after log rotation

    postrotate

    invoke-rc.d rsyslog rotate > /dev/null

    endscript

    }

You do not need to keep the source log files on USM Appliance for more than a few days. Rotating these files regularly maintains enough free disk space on USM Appliance for standard operations.