Setting Up Alarm Forwarding

Important: Note that the configurations outlined in this section are very important. This is the section that most of our partners have problems with. Not taking this section seriously can result in massive headaches and turn into a time-suck. Please proceed with care.

Forwarding and Federation

LevelBlue uses the term forwarding to describe the transfer of information from one USM Appliance Server to another. Forwarding can send alarms and/or events not only to other USM Appliance Servers but to USM Appliance Loggers as well. In a federated setup, the best practice is to forward alarms only, because forwarding events in such a scenario can lead to performance degradation and there is no effective way to distinguish one client's events from the other. 

After you have set up alarm forwarding, you will see a combined view of all alarms on your Federated Server. Alarms Alarms provide notification of an event or sequence of events that require attention or investigation. will have a tag attached to them with their origin. This will help you identify which customers generate each alarm.

Important: All servers involved in alarm forwarding, the Federation Server and the customer's USM Appliance Server, must be on the same version.

Using this method, all alarms from the customer's USM Appliance Server are forwarded to the Federation Server. This is the default behavior and should be used in all MSSP architectures.

In order for a customer's USM Appliance Server to forward alarms to the Federation Server, you need to add the customer's USM Appliance Server in the Federation Server and then configure the alarm forwarding on the customer's USM Appliance Server.

All configurations are done through the web UI. Use the VPN IP address in the configuration steps.

To add the customer USM Appliance Server

  1. Login to Federation Server.
  2. Go to Configuration > Deployment > Servers.
  3. Select New and add the customer's USM Appliance Server.
  4. Click Apply Changes.

To configure alarm forwarding

  1. Login to customer USM Appliance Server.

  2. Go to Configuration > Deployment > Servers.

    You should see both servers listed (customer's USM Appliance Server and the Federation Server).

  3. Choose the customer's USM Appliance Server and click Modify.
    1. Keep Forward Alarms to Yes.

    2. Change Forward Events to No.

      Important: Make sure to do this because you should only forward alarms to the Federation Server.

    3. Under the Forward Servers section, choose Add Server and add the Federation Server, and then click Add New.

      Keep Priority as 1 for the Federation Server, which means the highest priority.

      Note: USM Appliance uses the Priority setting to determine which server to send when you add multiple servers to receive events/alarms. When servers have the same priority such as Priority 1, events/alarms are sent to both. When servers have different priorities, events/alarms will be sent in the order of priorities (Priority 1 being the highest), stopping when a server has successfully received the event/alarm.

    4. Keep the other settings as default and click Save.
  4. Click Apply Changes.

After the configuration is finished, on the Federation Server, if you go to the Configuration > Deployment > Server page and click the Server Hierarchy link, a diagram displays showing both servers with an arrow pointing from the customer's USM Appliance Server to the Federation Server, indicating that the customer's USM Appliance Server is forwarding to the Federation Server.

After setting up forwarding you can either wait for alarms to come in automatically or generate events that are guaranteed to produce an alarm.

You can try one of the following on the customer's USM Appliance:

  • Generate consecutive failed SSH logins. (Make sure you have enabled the SSH plugin.)
  • Use a malware pcap and tcpreplay, such as the Zeus malware pcap samples available online, to inject malicious traffic into the network stream.

  • Use OTX Indicators of Compromise (IoC):

    Search OTX subscriptions (Configuration > Open Threat Exchange) for a pulse that has a domain name listed under Indicators of Compromise. Then on the customer's USM Appliance, run:

    nslookup <domain name of the subscribed OTX pulse>

  • Run an Nmap SYN Stealth Scan:

    nmap –sS <customer-USM Appliance-ip> –D 8.8.8.8

    This will generate an alarm on the customer's USM Appliance.

Subsequently, you should see the relevant alarm appear in the Federation Server. If not, try the steps below on the customer's USM Appliance:

  1. Connect to the LevelBlue Console through SSH and use your credentials to log in.

    The LevelBlue Setup menu displays.

  2. On the LevelBlue Setup main menu, select Jailbreak System to gain command line access.

    Select Yes when prompted. You will be in the root directory.

  3. Check the log file, /var/log/alienvault/forward/forward.log, to see if the alarm was forwarded.

  4. If needed, restart two services:

    1. Type exit to go back to the LevelBlue Setup menu.
    2. Select Maintenance & Troubleshooting.
    3. Select Restart System Services.
    4. Select Restart LevelBlue Alarm Forward Service.
    5. Confirm when prompted. After it is finished, press Enter to return to the Restart System Services screen.
    6. Select Restart LevelBlue Server Service.
    7. Confirm when prompted.

Verity that the alarm shows up in the Federation Server. If it is still not working, contact LevelBlue Support.