USM Anywhere™

Protecting Your Sensor's Performance with EPS Adaptive Response

To protect the health of your system, USM Anywhere monitors the rate of events being sent to your sensor. If that rate, measured in events per second (EPS), threatens to impact your sensor's capacity USM Anywhere will engage EPS Adaptive Response. EPS Adaptive Response enables your system to take more time to process events coming in by throttling your EPS, keeping your system running without risking event loss.

Note: See Reaching the Monthly Usage Limit to read more about sensor capacity and USM Anywhere tier limits.

EPS Adaptive Response may cause delays in the correlation of alarms. While activated, events will enter an adaptive processing queue on sensors or the agent pipeline to assist product stability. When deactivated, the queue will be processed at normal speeds. USM Anywhere views events in the user interface (UI) based on the time the actual event occurred and not the time the event was received by the sensor. When EPS Adaptive Response is ended and queued events get forwarded, they will be backfilled into the appropriate timeslots.

Important: After EPS Adaptive Response has ended, alarms may be delayed while throttled events are being forwarded.

EPS Adaptive Response Scenarios

There are two scenarios in which EPS Adaptive Response may be engaged:

  • The sensor volume is about to fill up: When your sensor's disk space approaches full, throttling engages to preserve remaining disk space.

  • The customer is over tier: If the customer is projected to be over tier, throttling slows down the event ingestion in both sensors and agents until the customer is back within their tier limits.

Note: Every time EPS Adaptive Response is engaged or disengaged, your USM Anywhere sensor will create a system event. In addition, a system event is created when throttling rates change.

You may also create custom events around throttling to best suit your environment's needs.

Disk Space

If your sensor's disk volume ever filled up completely, the sensor would stop being able to process events. To prevent this, EPS Adaptive Response slows down your sensor's EPS, giving your system time to process events coming in. As your disk partition continues to fill, the rate of EPS throttling will increase to preserve what remains of your disk space and your sensor's operations.

Throttling Rates per Percentage of Disk Used
Disk Use (%) Sensor Throttling (ms)
88 0.25
90

0.5

92

2

95

5

98

10

99 100

Over Tier

When your USM Anywhere Sensor is projected to go over tier, meaning either 5% over your allotment or over 50GB in total, it will analyze the rate of traffic coming through the sensor and agent data pipeline. Then it will engage EPS Adaptive Response to slow your data rate and keep your USM Anywhere Sensor operational until data ingestion can be decreased, or your tier is upgraded.

If you have more than one sensor or agent, USM Anywhere will begin by throttling only your sensors with the highest EPS. This is determined by retrieving your system's EPS per minute for every sensor and maintaining a rolling EPS average. Every 10 minutes, once your system determines that throttling is necessary, EPS will be engaged on any sensor sending more EPS than 75% of this average.

When your USM Anywhere is projected to go over tier, EPS throttling is progressive, starting at 1 ms and increasing up to 250 ms as necessary until data ingestion decreases or your tier changes. Once throttling has been engaged, the throttling rate will be sampled regularly. If the tier decrement is smaller than 5%, then the throttling factor is doubled. Otherwise, it remains the same until throttling is no longer necessary.

EPS Adaptive Response System Events

Every time the event throttling value changes in a sensor, a new system event is generated.

There are two system event types:

  • EPS throttling has been engaged: Your sensor's EPS is being throttled.

  • EPS throttling has ended: Your sensor's EPS is no longer being throttled.

Each throttling system event type has a number of possible event keys, specifying which type of event has been triggered.

EPS Throttling System Events and Their Meanings
Event Type Event Key Event Value
Sensor is being throttled event_action SENSOR_THROTTLING
event_name

Sensor is being throttled

sensor_uuid

Sensor ID

customheader_0

Throttling value

customfield_0

Throttling value in milliseconds

Sensor throttling is over event_action SENSOR_THROTTLING
event_name Sensor throttling is over
sensor_uuid Sensor ID
customheader_0 Throttling value
customfield_0 0