Resource profiles

TELEMETRY PIPELINE

Resource profiles

When sending data, you might need to adjust settings to balance performance and reliability. Chronosphere Telemetry Pipeline offers resource profiles, which

are templates that contain optimal settings for pipelines. You can edit these templates to change behavior related to buffering and resource consumption. Resource profiles are applied to pipelines when you deploy them.

Profile specifications

Telemetry Pipeline includes these profile specifications:

Type	Description
Best Effort Low Resource (`best-effort-low-resource`)	- Persistent storage - Buffering configured - No checksum - Maximum of 5 MB of memory when processing backlog data
High-Performance Optimal Throughput (`high-performance-optimal-throughput`)	- Minimal persistent storage - In-memory only option - No buffering
High-Performance Guaranteed Delivery (`high-performance-guaranteed-delivery`)	- Larger persistent storage - Buffering configured - 256 MB memory for backlog data

Resource profiles only support storage configurations. Memory and CPU limits can be set in the profile, but by default no limits are set.

Profile properties

Use the following settings to modify your profile:

Setting name	Configuration key	Possible values	Description
`StorageVolumeSize`	-	Volume size in `Mi`/`Gi`/`Ti`	Size of the volume to be mounted in the pod for local storage processing.
`StorageSyncFull`	`storage.sync`	`true`, `false`	Use normal synchronization mode (`false`) or full (`true`)
`StorageBacklogMemLimit`	`storage.backlog.mem_limit`	Memory in MB (for example: `5M`)	Maximum amount of memory to use when processing chunks from the backlog.
`StorageMaxChunksUp`	`storage.max_chunks_up`	Integers > `0`	Sets the maximum number of Chunks that can be up in memory.

Use a built-in resource profile during deployment

Specify any of the default profiles by using the following command:

calyptia create pipeline --core-instance CORE_INSTANCE --resource-profile PROFILE

Replace CORE_INSTANCE with the unique ID or name of the Core Instance, and PROFILE with the profile name.

Define a new resource profile

Use the following command to define your own resource profiles in Telemetry Pipeline:

calyptia create resource_profile --core-instance CORE_INSTANCE --name NEW_NAME --spec JSON_FILE

Replace CORE_INSTANCE with the unique ID or name of the Core Instance, and JSON_FILE with the JSON configuration file.

Resource profile JSON files follow this schema:

best-effort-low-resourcehigh-performance-optimal-throughputhigh-performance-guaranteed-delivery

This is an example of a resource profile definition:

{
 "resources": {
  "storage": {
   "syncFull": false,
   "backlogMemLimit": "5M",
   "volumeSize": "250Mi",
   "maxChunksUp": 128,
   "maxChunksPause": false
  },
  "cpu": {
   "limit": "0",
   "request": "0"
  },
  "memory": {
   "limit": "0",
   "request": "0"
  }
 }
}

A 0 value is defined as unlimited.

List available resource profiles

Use the following command to list the resource profiles defined within a Core Instance:

calyptia get resource_profiles --core-instance CORE_INSTANCE

Replace CORE_INSTANCE with the unique ID or name of the Core Instance.

Resource labels Scaling