BlueApps use parsers to extract and normalize data received from different data sources. A parser in USM Anywhere is a JavaScript Object Notation (JSON) file that defines the method of dividing input into different pieces, and then mapping those pieces to the specific fields of a normalized event. The generic parser looks like this:
{ "name": "", "type": "", "version": "", "enrichmentScript": "", "device": "", "vendor": "", "deviceType": "", "family": "", "parentName": "", "parentVersion": "", "app": "", "hints" : [ ], "highlight_fields": "properties" : { "separator.pair" : "a", "separator.groupings" : "b" }, "dictionaries": { "main": { "load": "main-dictionary-0.1.json" }, "additional": { "contents": { "val1": [ 'a', 'b', 'c' ], "val2": [ 'a', 'b', 'c' ], "val3": [ 'a', 'b', 'c' ], "val4": [ 'a', 'b', 'c' ] } } }, "tags": { "field1": [ "map('key1') == '' ? map('key2') : map('key1')" ], "field2": [ "dict('main', map('howdy'), 1)", "regexp(/(d+)(.*)/, dict('additional', map('howdy'), 1), 1)" ], "field3": [ "split(dict('additional', tag('field2')), ',', 2)" ] } }
The following table includes each field and its description that a parser uses:
Field | Description |
---|---|
name | Name of the parser. |
type | Log type. The value depends on the log format for the specific data source. Some valid values are these: regex, CEF, CLF, CSV, GELF, JSON, keyvalue, LEEF, split, w3c, XML. |
version | Version of the parser. |
enrichmentScript | Specify the Lua script used to process a log line. |
device | Data source that is sending the logs. |
vendor | Data source vendor. |
deviceType | Data source type (for example, firewall, router). |
parentName | If a parentName is declared for the parser, a copy of the parent parser will be made and the child parser will overwrite that copy. |
parentVersion | Version of the parent parser. |
app | Name displayed under Data Sources > BlueApps. |
hints | References to unique information within a syslog message that identify the data source sending the logs. BlueApps that contain hints will process the message when the information in the log message matches the criteria set within the parser. See BlueApps and Data Sources for more information. |
highlight_fields | The most important fields shown in the principal event view. |
properties | This field describes the different properties of the parser, depending on the type. |
dictionaries | For each declared dictionary, you can either call out to an external file by name (with the assumption that the path is relative to the parser file) or you can declare the contents of the dictionary inline. Every entry in the dictionary is defined as a key and a series of values. |
tags | Tags define how different pieces in a log line map to the fields of a normalized event. For each tag that is defined, the USM Anywhere Sensor begins by evaluating the first code line. If the first code line returns a value, the field in event will be populated with that value. Otherwise, it evaluates the next code line until one returns a non-null value. |
rules |
For regular expression (regex)-type parsers, there is a set of rules with these fields:
|
This is an example of a regex parser:
{ "name": "Test Regex Parser", "version": "0.1", "type": "regex", "hints": [ { "typeName": "tag.equals", "value": "test" } ], "rules": [ { "name": "Rule test 1" "regex": "test (\\S+)", "tags": { "event_name": "concat('test 1')", "customfield_0": "map(1)" } }, { "name": "Rule test 2" "contains": ["test2"], "regex": "test2 (?<src>\\S+) (?<dst>\\S+)", "tags": { "event_description": "concat('test 2')", "source_username": "map('src')", "destination_username": "map('dst')" } } ] }