Scaling
When ingesting data with network sources or agents, you might need to scale the pipelines to process additional records. With Chronosphere Telemetry Pipeline, all ingested data is automatically load balanced between a group of pipeline replicas.
Add autoscaling
Use one of the following methods to set up autoscaling for your pipelines:
For replica counts with a volume over 100 TB a day, contact Chronosphere Support prior to scaling up to ensure stability.
-
Sign in to the Telemetry Pipeline web interface (opens in a new tab).
-
Click Core Instances, and then choose the Core Instance which has the Data Pipeline you want to autoscale.
If the pipeline doesn't exist, click Create a custom pipeline or Add new to create a new pipeline.
-
Click the Pipeline Name of the pipeline to autoscale. The pipeline summary appears.
-
Click Advanced Settings.
-
Increase the Number of replicas to the desired amount and then click Apply.
Horizontal pod autoscaling
Requires Core Instance version 2.12.3 or later.
Pipelines support horizontal pod autoscaling (opens in a new tab) (HPA), which enables Telemetry Pipeline to scale pipelines up or down as needed.
Enable horizontal pod autoscaling
To enable HPA, create or update a pipeline
through Pipeline CLI and include both the --scale-up-value and
--scale-down-value flags. You must assign a value greater than 0 to both flags,
but these values don't need to be identical to each other.
Configure horizontal pod autoscaling
To configure HPA, use the following flags when you create or update a pipeline through Pipeline CLI:
| Flag | Description | Value |
|---|---|---|
--min-replicas | The minimum number of replicas to which the autoscaler can scale down. | Example: 3 |
--scale-up-type | The type of scale-up policy to implement. | Pods orPercentz |
--scale-up-value | Required. The amount of change to implement through scale-up policies. | Must be greater than 0. |
--scale-up-period-seconds | The window of time for which the scale-up policy should hold true, in seconds. | Example: 30 |
--scale-down-type | The type of scale-down policy to implement. | Pods or Percent |
--scale-down-value | Required. The amount of change to implement through scale-down policies. | Example: 15 |
--scale-down-period-seconds | The window of time for which the scale-down policy should hold true, in seconds. | Example: 30 |
--utilization-cpu-average | The target percentage value for average CPU utilization. The autoscaler attempts to maintain equilibrium around this value by scaling pipelines up or down accordingly. | Example: 25 |
--utilization-memory-average | The target percentage value for average memory utilization. The autoscaler attempts to maintain equilibrium around this value by scaling pipelines up or down accordingly. | Example: 30 |
Disable horizontal pod autoscaling
To disable HPA, update a pipeline through Pipeline CLI
and set both the --scale-up-value and --scale-down-value flags to 0. This
prevents the autoscaler from scaling pipelines up or down, which means other HPA
settings won't affect your pipelines.