What is Event Log Correlation?
In simple terms, event correlation provides the ability to discover and apply logical associations among disparate individual raw log events in order to:
- Make informed security decisions
- Identify and respond to security threats
- Validate effectiveness of security controls
- Measure and report on compliance with PCI, HIPAA, SOX, and other standards
- Detect policy violations
Log correlation is one of the most powerful tools in the security analyst’s toolkit, but it can get pretty complicated pretty quickly. So we wanted to spend a few minutes describing how to use log correlation, and how it works in this short video. As a troubleshooting tool, event logs are your friend. Logs contain the essential breadcrumbs of network and device intelligence. What are users doing? What data is being accessed? What are the blips on our radar of system performance or network activity? Could these blips signal a security threat or an attack in progress? In fact, according to a recent Verizon data breach investigations report, 84% of organizations that had a security breach had evidence of that breach in their log files. But none of those log files contained entries that said “You’re being attacked!” Instead, the log entries are more along the lines of “Successful login from an authenticated user”. That’s why log correlation is so critical, and yet so complicated.
First, logs vary greatly from system to system, and even from version to version for the same system. Second, some logs are written in plain language that a human can understand, and other are quite cryptic, with only esoteric system codes. Third, logs have siloed lenses – each system sees their world through its own imperfect and incomplete filter. An example here is that a network IDS sees packets and streams, while an application log sees sessions, users and requests. So while these systems will log similar activities, the way they articulate these activities is quite different. Fourth, logs record static, fixed points in time, without the full context of sequence of related events. Logical analysis, either through event correlation rules, or through human intervention is therefore necessary in order to bring in that full context.
Log correlation, or event log correlation, provides the answer to these challenges, so that security analysts and incident responders can make the right decision on what to do next to respond and investigate. The secret sauce on converting raw log data into actionable alarms, alerts and reports is – well I mentioned it a few minutes ago – the use of event correlation rules. Event correlation rules merely tell people what to think about the raw log events by connecting the dots on related, yet disparate data. The logic in the event correlation rules essentially translates these raw log snippets into alarms, so that the appropriate action can take place. And that’s log correlation in a nutshell.