Sending USM Anywhere Notifications to PagerDuty

PagerDuty provides a mechanism to create services that include integrations to its Events API as a way to post data from external sources into PagerDuty incidents. The service configuration determines how PagerDuty handles the incoming incident. You must first create the integration key for a PagerDuty service before you set up the configuration in USM Anywhere to send these notifications.

To create a PagerDuty service and integration for USM Anywhere

1. Log in to your PagerDuty account.
2. In the top menu, select Configuration > Services.

3. At the top of the page, click Add New Service.

4. In the General Settings, enter a name for the new service (such as AT&T Cybersecurity).

5. In Integration Settings, set the type and name for the integration.

   • Choose Use our API Directly and select Events API v2.
   • Enter an Integration Name (such as USM Anywhere).

   

6. Set the Incident Settings and Incident Behavior according to how you want PagerDuty to handle the incidents (notifications) from USM Anywhere.
7. Click Add Service.

8. In the Integrations tab, copy the Integration Key for the new integration.

   

   Make sure to copy the key value to a secured location.

After you have created the PagerDuty integration and copied the key, you can configure USM Anywhere for PagerDuty notifications. After this configuration is in place, any orchestration rules set up for PagerDuty notification will send the triggered notification to the PagerDuty service for incident handling.

To configure the connection between PagerDuty and USM Anywhere

1. In the USM Anywhere web UI, go to Settings > Notifications.
2. In the left navigation panel, click PagerDuty.

3. In the PagerDuty integration key field, paste the key value that you copied from your PagerDuty service integration.

   

4. Click Save Credentials.

Create an orchestration rule to match new alarms or events and trigger a notification to your PagerDuty service. You can use an existing alarm or event with the desired characteristics to easily set the matching conditions for the rule.

To create an orchestration rule to trigger a PagerDuty notification

1. Go to Activity > Alarms or Activity > Events.
2. Click the alarm or event to open the details.

3. Click Create Rule and select Create Notification Rule.

   

4. You have already suggested property values to create a matching condition, but if you want to add new property values, click Add Condition.

   Note: If the field is related to the name of a country, you should use the country code defined by the ISO 3166.

   Note: The Sources or Destinations field needs to match the universally unique identifier (UUID) of the event or alarm. You can use the Source Name or Destination Name field instead.

   Important: Instead of using the equals and equals, case insensitive operators for array fields, AT&T Cybersecurity recommends the use of the in or contains operators.

   Note: If you need to add a property value that maps with a property key, you need to know the mapping of the field. See Determining the Mapping of a Field for more information.

5. (Optional.) Click Add Group to group your conditions.

   Note: See Operators in the Orchestration Rules for more information.

Note: The current rule box shows you the syntax of your rule, and the rule verification box reviews that syntax before saving the rule.

6. Click Next.

   

   Important: A dialog box opens if there are warning messages. Click Cancel to review the warning messages, or click Accept to continue creating the rule.

7. Enter a name for the rule.
8. (Optional.) Enter a description for identifying this rule.
9. For Notification Method, select the Slack option.

10. Enter the Slack Alert Username.

    The username must be a valid team member for the Slack channel.

    

11. Modify these two options:

    • Occurrences: Specify the number of event occurrences that produce a match on the conditional expression to trigger the rule. You can enter the number of occurrences or use the arrow to scroll the value up or down. You need to enter a number between 1 and 100.

    • Length: Specify the length of the timespan used to identify a match for multiple occurrences. Enter the number and choose a value of seconds, minutes, or hours.

      This duration identifies the amount of time that transpires from the beginning to end of the occurrence. If the number of occurrences is not met within this period, the rule is not a match.

      

      In this example, the rule applies when the configured conditions happen five times every three hours.

    These two options function together to specify the number of occurrences within a time period that will produce a match for the rule. For example, you can define a rule to trigger an alarm Alarms provide notification of an event or sequence of events that require attention or investigation. for an unauthorized access An incident-type categorization that may be a precursor to other actions or stages of an attack. attempt when a failed SSH Program to securely log into another computer over a network, execute commands in a remote machine, and move files from one machine to another through Secure Copy (SCP). login Log in (verb): Process in which an individual gains access to a computer system after providing sufficient credentials to authenticate their unique identity. Login (noun): User credentials, typically a username and matching password. occurs three times within a five-minute window.

12. Click Save.

    The created rule displays in the list of rules. You can see it from Settings > Rules > Orchestration Rules. See Orchestration Rules for more information.

Important: It takes a few minutes for an orchestration rule to become active.

5. For Notification Method, select the PagerDuty option.

   

6. At the bottom of the dialog box, set the rule condition parameters to specify the criteria for a matching alarm or event to trigger the rule.

   

   • This section provides suggested property/value pairs from the selected alarm or event that you can use as conditions for the rule. Click the icon to delete the items that you do not want to include in the matching conditions. You can also add other conditions that are not suggested.
   • If you create the rule from the Rules page, you must use the Add Condition and Add Group functions to define the property/value pairs that you want to use as conditions for the rule.
   • At the bottom of the dialog box, click More to display the optional multiple occurrence and window-length parameters.
   Conditional Expression

   Select an operator and add one or more conditions to form the conditional expression. You can include a condition group to evaluate a subset of conditions. The Current Rule pane displays the constructed expression in standard syntax. The box displays a red border if the expression is syntactically invalid as currently specified. A valid expression is required to save the rule definition.

   Select the operator used to determine the match for multiple conditions:

   • AND: Match all conditions.
   • OR: Match any one condition.
   • AND NOT: Exclude items matching all conditions after the first.
   • OR NOT: Include all items that do not match any conditions after the first.

   Click Add Condition to add a condition. For each condition, specify the field name, evaluator, and value. If the evaluation returns true for the condition, it is a match.

   Click Add Group to add a condition group. A new group includes a condition and its own operator used to match the conditions within the group. You can nest condition groups.

   Occurrences

   Specify the number of event or alarm occurrences that produce a match on the conditional expression to trigger the rule. The default value is 1. You can enter the number of occurrences or use the arrow to scroll the value up or down.

   USM Anywhere uses this in conjunction with the Length option to specify the number of occurrences within a time period that will trigger the rule. For example, you can define a rule to trigger for an unauthorized access attempt when a failed SSH Program to securely log into another computer over a network, execute commands in a remote machine, and move files from one machine to another through Secure Copy (SCP). login occurs three times within a five-minute window.

   Length

   Specify the length of the window to identify a match for multiple occurrences. Enter the number and choose a time unit value of seconds, minutes, or hours. This time period identifies the amount of time that transpires from the first occurrence to the last occurrence. If the number of occurrences is not met within this period, the rule does not trigger.

7. Click Save Rule.

PagerDuty creates incidents for the service from the notifications that USM Anywhere sends. You can review and respond to these incidents from your PagerDuty incidents dashboard. When you expand the details for the incident, in the Client area, click the link to open the event or alarm in USM Anywhere.