USM Anywhere™

Protecting Your Sensor's Performance with EPS Throttling

To protect the health of your system, USM Anywhere monitors the rate of events being sent to your sensor. If that rate, measured in events per second (EPS), threatens to impact your sensor's capacity USM Anywhere will engage EPS throttling. Throttling allows your system to take more time to process events coming in, keeping your system running without risking event loss.

Throttling Scenarios

There are two scenarios in which EPS throttling may be engaged:

  • The sensor volume is about to fill up: When your sensor's disk space approaches full, EPS throttling engages to preserve remaining disk space.

  • The customer is over tier: If the customer is projected to be over tier, EPS throttling slows down the event ingestion in both sensors and agents until the customer is back within their tier limits.

Note: Every time EPS throttling is engaged or disengaged, your USM Anywhere sensor will create a system event. In addition, a system event is created when EPS throttling rates change.

You may also create custom events around EPS throttling to best suit your environment's needs.

Disk Space

If your sensor's disk volume ever filled up completely, the sensor would stop being able to process events. To prevent this EPS throttling slows down your sensor's EPS, giving your system time to process events coming in. As your disk partition continues to fill, the rate of EPS throttling will increase to preserve what remains of your disk space and your sensor's operations.

Throttling Rates per Percentage of Disk Used
Disk Use (%) Sensor Throttling (ms)
88 0.25
90

0.5

92

2

95

5

98

10

99 100

Over Tier

When your USM Anywhere sensor is over tier, meaning either 5% over your allotment or over 50GB in total, it will analyze the rate of traffic coming through the sensor and agent and engage EPS throttling to slow your data rate and keep your USM Anywhere Sensor operational until data ingestion can be decreased, or your tier is upgraded.

If you have more than one sensor or agent, USM Anywhere will begin by throttling only your sensors with the highest EPS. This is determined by retrieving your system's EPS per minute for every sensor and maintaining a rolling EPS average. Every 10 minutes, once your system determines that throttling is necessary, EPS will be engaged on any sensor sending more EPS than 75% of this average.

When your USM Anywhere is over tier, EPS throttling is progressive, starting at 1 ms and increasing up to 250 ms as necessary until data ingestion decreases or your tier changes. Once throttling has been engaged, the throttling rate will be sampled regularly. If the tier decrement is smaller than 5%, then the throttling factor is doubled. Otherwise, it remains the same until throttling is no longer necessary.

EPS Throttling System Events

Every time the event throttling value changes in a sensor, a new system event is generated.

There are two system event types:

  • EPS throttling has been engaged: Your sensor's EPS is being throttled

  • EPS throttling has ended: Your sensor's EPS is no longer being throttled

Each throttling system event type has a number of possible event keys, specifying which type of event has been triggered.

EPS Throttling System Events and Their Meanings
Event Type Event Key Event Value
Sensor is being throttled event_action SENSOR_THROTTLING
event_name

Sensor is being throttled

sensor_uuid

Sensor ID

customheader_0

Throttling value

customfield_0

Throttling value in milliseconds

Sensor throttling is over event_action SENSOR_THROTTLING
event_name Sensor throttling is over
sensor_uuid Sensor ID
customheader_0 Throttling value
customfield_0 0