BlueApps Parser Syntax

BlueApps use parsers to extract and normalize data received from different data sources. A parser in USM Anywhere is a JavaScript Object Notation (JSON) file that defines the method of dividing input into different pieces, and then mapping those pieces to the specific fields of a normalized event. The generic parser looks like this:

{ "name": "", "type": "", "version": "", "enrichmentScript": "", "device": "", "vendor": "", "deviceType": "", "family": "", "parentName": "", "parentVersion": "", "app": "", "hints" : [ ], "highlight_fields": "properties" : { "separator.pair" : "a", "separator.groupings" : "b" }, "dictionaries": { "main": { "load": "main-dictionary-0.1.json" }, "additional": { "contents": { "val1": [ 'a', 'b', 'c' ], "val2": [ 'a', 'b', 'c' ], "val3": [ 'a', 'b', 'c' ], "val4": [ 'a', 'b', 'c' ] } } }, "tags": { "field1": [ "map('key1') == '' ? map('key2') : map('key1')" ], "field2": [ "dict('main', map('howdy'), 1)", "regexp(/(d+)(.*)/, dict('additional', map('howdy'), 1), 1)" ], "field3": [ "split(dict('additional', tag('field2')), ',', 2)" ] } }

The following table includes each field and its description that a parser uses:

Fields and Description Used by a Parser
Field Description
name Name of the parser.
type Log type. The value depends on the log format for the specific data source. Some valid values are these: regex, CEF, CLF, CSV, GELF, JSON, keyvalue, LEEF, split, w3c, XML.
version Version of the parser.
enrichmentScript Specify the Lua script used to process a log line.
device Data source that is sending the logs.
vendor Data source vendor.
deviceType Data source type (for example, firewall, router).
parentName If a parentName is declared for the parser, a copy of the parent parser will be made and the child parser will overwrite that copy.
parentVersion Version of the parent parser.
app Name displayed under Data Sources > BlueApps.
hints References to unique information within a syslog message that identify the data source sending the logs. BlueApps that contain hints will process the message when the information in the log message matches the criteria set within the parser. See BlueApps and Data Sources for more information.
highlight_fields The most important fields shown in the principal event view.
properties This field describes the different properties of the parser, depending on the type.
dictionaries For each declared dictionary, you can either call out to an external file by name (with the assumption that the path is relative to the parser file) or you can declare the contents of the dictionary inline. Every entry in the dictionary is defined as a key and a series of values.
tags Tags define how different pieces in a log line map to the fields of a normalized event. For each tag that is defined, the USM Anywhere Sensor begins by evaluating the first code line. If the first code line returns a value, the field in event will be populated with that value. Otherwise, it evaluates the next code line until one returns a non-null value.

For regular expression (regex)-type parsers, there is a set of rules with these fields:

  • name: name of the rule
  • contains: pre-match filter
  • regex: regular expression
  • tags: tags to capture

This is an example of a regex parser:

{ "name": "Test Regex Parser", "version": "0.1", "type": "regex", "hints": [ { "typeName": "tag.equals", "value": "test" } ], "rules": [ { "name": "Rule test 1" "regex": "test (\\S+)", "tags": { "event_name": "concat('test 1')", "customfield_0": "map(1)" } }, { "name": "Rule test 2" "contains": ["test2"], "regex": "test2 (?<src>\\S+) (?<dst>\\S+)", "tags": { "event_description": "concat('test 2')", "source_username": "map('src')", "destination_username": "map('dst')" } } ] }