Tutorial 4: Correlation engine primer

December 18, 2007  |  Dominique Karg

Introduction

In order to answer to a recent forum post https://sourceforge.net/forum/message.php?msg_id=4666889 [no longer available] I had to do a quick research since it had been some time since I last tested this.

The exact question was:

Hello,



Is there a document talking about how the directives are processed?  One question

that I have is if you have multiple directives created and an event comes in

that matches the initial states of more than a single directive will both actually

process the event, or only the first match (which I think is the case)?



Thanks for any clarification you can provide.



Stephen

This post gives a bit of insight to how the correlation engine works and features some simple, custom made directives that help me answer that question.

The test environment features two events belonging to the ssh plugin (plugin_id 4003):

  • SSH password failed (plugin_sid 1)
  • SSH password accepted (plugin_sid 7)

In order to test this I’ve created three directives (plugin_id 1505)

  • Test directive 21, grouping one login failure and one success
  • Test directive 22, grouping one login failure and one success
  • Test directive 23, used in the second case, grouping those two

So, with all of this in place it was easy to simulate this failing a login and succeeding afterwards.

Case 1: Two concurrent correlation rules

Directives

Put these into generic.xml for example

<directive id="21" name="Testing concurrency 1" priority="5">

  <rule type="detector" name="Level 1:1" reliability="7"

    from="ANY" to="ANY" port_from="ANY" port_to="ANY"

    time_out="160" plugin_id="4003" plugin_sid="1" >

        <rules>

                <rule type="detector" name="Level 1:2" reliability="10"

                occurrence="1" from="ANY" to="ANY" port_from="ANY" port_to="ANY"

                plugin_id="4003" plugin_sid="7" />

        </rules>

  </rule>

</directive>



<directive id="22" name="Testing concurrency 2" priority="5">

  <rule type="detector" name="Level 2:1" reliability="7"

    from="ANY" to="ANY" port_from="ANY" port_to="ANY"

    time_out="160" plugin_id="4003" plugin_sid="1" >

        <rules>

                <rule type="detector" name="Level 2:2" reliability="10"

                occurrence="1" from="ANY" to="ANY" port_from="ANY" port_to="ANY"

                plugin_id="4003" plugin_sid="7" />

        </rules>

  </rule>

</directive>

Logs

2007-12-10 04:19:56 OSSIM-Message: Event received: event id="0" alarm="0" type="detector"

date="2007-12-10 04:19:53" plugin_id="4003" plugin_sid="1" src_ip="10.0.1.3" src_port="64270"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" asset_src="1" asset_dst="1"

log="Dec 10 04:19:53 ossim sshd[11366]: Failed password for root from 10.0.1.3 port 64270 ssh2"

username="root" 



2007-12-10 04:20:06 OSSIM-Message: Event received: event id="0" alarm="0" type="detector"

date="2007-12-10 04:20:05" plugin_id="4003" plugin_sid="7" src_ip="10.0.1.3" src_port="64270"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" asset_src="1" asset_dst="1"

log="Dec 10 04:20:05 ossim sshd[11366]: Accepted password for root from 10.0.1.3 port 64270 ssh2"

username="root" 



2007-12-10 04:20:06 OSSIM-Message: Event received: event id="3" alarm="0" type="detector"

date="2007-12-10 04:20:06" plugin_id="1505" plugin_sid="21" src_ip="10.0.1.3" src_port="64270"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" priority="5" reliability="10"

asset_src="1" asset_dst="1" data="directive_event: Testing concurrency 1, Priority: 5

Rule 1 [2007-12-10 04:20:00] [4003:1] [Rel: 7] 10.0.1.3:64270 -> 0.0.0.0:0

Rule 2 [2007-12-10 04:20:06] [4003:7] [Rel: 10] 10.0.1.3:64270 -> 0.0.0.0:0

" 



2007-12-10 04:20:07 OSSIM-Message: Event received: event id="4" alarm="0" type="detector"

date="2007-12-10 04:20:06" plugin_id="1505" plugin_sid="22" src_ip="10.0.1.3" src_port="64270"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" priority="5" reliability="10"

asset_src="1" asset_dst="1" data="directive_event: Testing concurrency 2, Priority: 5

Rule 1 [2007-12-10 04:20:01] [4003:1] [Rel: 7] 10.0.1.3:64270 -> 0.0.0.0:0

Rule 2 [2007-12-10 04:20:06] [4003:7] [Rel: 10] 10.0.1.3:64270 -> 0.0.0.0:0

Explanation

Here we wrote two directives with exactly the same events matching, naming it differently. That way we’ll be able to answer the concurrency question since I honestly don’t remember how that part worked.

As you can see in the logs, two events arrived, and two directives got matched.

Result: two alarms

(Image removed, broken link, I’m very sorry. DK.)

Case 2: Two concurrent correlation rules, a third grouping rule

Directives

<directive id="21" name="Testing concurrency 1" priority="5">

  <rule type="detector" name="Level 1:1" reliability="7"

    from="ANY" to="ANY" port_from="ANY" port_to="ANY"

    time_out="160" plugin_id="4003" plugin_sid="1" >

        <rules>

                <rule type="detector" name="Level 1:2" reliability="10"

                occurrence="1" from="ANY" to="ANY" port_from="ANY" port_to="ANY"

                plugin_id="4003" plugin_sid="7" />

        </rules>

  </rule>

</directive>



<directive id="22" name="Testing concurrency 2" priority="5">

  <rule type="detector" name="Level 2:1" reliability="7"

    from="ANY" to="ANY" port_from="ANY" port_to="ANY"

    time_out="160" plugin_id="4003" plugin_sid="1" >

        <rules>

                <rule type="detector" name="Level 2:2" reliability="10"

                occurrence="1" from="ANY" to="ANY" port_from="ANY" port_to="ANY"

                plugin_id="4003" plugin_sid="7" />

        </rules>

  </rule>

</directive>



<directive id="23" name="Aggregated test directive" priority="5">

  <rule type="detector" name="Directive 1" reliability="7"

    from="ANY" to="ANY" port_from="ANY" port_to="ANY"

    time_out="160" plugin_id="1505" plugin_sid="21,22" >

        <rules>

                <rule type="detector" name="Directive 2" reliability="10"

                occurrence="1" from="ANY" time_out="300" to="ANY" port_from="ANY" port_to="ANY"

                plugin_id="1505" plugin_sid="21,22" sticky="true" sticky_different="SRC_IP" />

        </rules>

  </rule>

</directive>

Logs

2007-12-10 06:24:56 OSSIM-Message: Event received: event id="0" alarm="0" type="detector"

date="2007-12-10 06:24:55" plugin_id="4003" plugin_sid="7" src_ip="10.0.1.3" src_port="64620"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" asset_src="1" asset_dst="1"

log="Dec 10 06:24:55 ossim sshd[19967]: Accepted password for root from 10.0.1.3 port 64620 ssh2"

username="root" 



2007-12-10 06:24:56 OSSIM-Message: Event received: event id="35" alarm="0" type="detector"

date="2007-12-10 06:24:56" plugin_id="1505" plugin_sid="21" src_ip="10.0.1.3" src_port="64620"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" priority="5" reliability="10"

asset_src="1" asset_dst="1" data="directive_event: Testing concurrency 1, Priority: 5

Rule 1 [2007-12-10 06:24:51] [4003:1] [Rel: 7] 10.0.1.3:64620 -> 0.0.0.0:0

Rule 2 [2007-12-10 06:24:56] [4003:7] [Rel: 10] 10.0.1.3:64620 -> 0.0.0.0:0

" 



2007-12-10 06:24:56 OSSIM-Message: Event received: event id="36" alarm="0" type="detector"

date="2007-12-10 06:24:56" plugin_id="1505" plugin_sid="22" src_ip="10.0.1.3" src_port="64620"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" priority="5" reliability="10"

asset_src="1" asset_dst="1" data="directive_event: Testing concurrency 2, Priority: 5

Rule 1 [2007-12-10 06:24:51] [4003:1] [Rel: 7] 10.0.1.3:64620 -> 0.0.0.0:0

Rule 2 [2007-12-10 06:24:56] [4003:7] [Rel: 10] 10.0.1.3:64620 -> 0.0.0.0:0

" 



2007-12-10 06:24:56 OSSIM-Message: Event received: event id="37" alarm="0" type="detector"

date="2007-12-10 06:24:56" plugin_id="1505" plugin_sid="23" src_ip="10.0.1.3" src_port="64620"

dst_ip="0.0.0.0" sensor="10.0.1.50" interface="eth2" protocol="OTHER" priority="5" reliability="10"

asset_src="1" asset_dst="1" data="directive_event: Aggregated test directive, Priority: 5

Rule 1 [2007-12-10 06:24:56] [1505:21] [Rel: 3] 10.0.1.3:64620 -> 0.0.0.0:0

Rule 2 [2007-12-10 06:24:56] [1505:22] [Rel: 10] 10.0.1.3:64620 -> 0.0.0.0:0

Explanation

This is exactly the same event as before with two added features: a grouping directive with our two recently created directives and a couple of keywords that prevent our grouping directive to fire twice, since the same event is used for level 1 and level 2. Using sticky=“true” and sticky_different=“SRC_IP” we are telling the engine to not spawn a second instance of the directive if it comes from the same source IP.

I think everything is quite self-explanatory.

Result: one alarm, grouping two others

Foot note

I’ve detected some glitches during these tests, mainly in the alarm panel grouping code. As you can see on the screenshots event count isn’t correctly interpreted and separate alarms are being spawned, besides the grouped one. I’ll check that asap.

Share this with others

Get price Free trial