Resource profiles

TELEMETRY PIPELINE

Resource profiles

Chronosphere Telemetry Pipeline offers templates called resource profiles that let you strike a balance between pipelines’ performance and reliability. These templates include different combinations of settings optimized for various scenarios. You can also edit resource profiles to customize behavior related to buffering and resource consumption.

Resource profiles are applied to pipelines when you deploy them.

Profile specifications

Telemetry Pipeline includes these profile specifications:

Type	Description
Best Effort Low Resource (`best-effort-low-resource`)	- Persistent storage - Buffering configured - No checksum - Maximum of 5 MB of memory when processing backlog data
High-Performance Optimal Throughput (`high-performance-optimal-throughput`)	- Minimal persistent storage - In-memory only option - No buffering
High-Performance Guaranteed Delivery (`high-performance-guaranteed-delivery`)	- Larger persistent storage - Buffering configured - 256 MB memory for backlog data

Resource profiles only support storage configurations. Memory and CPU limits can be set in the profile, but by default no limits are set.

Profile properties

Use the following settings to modify your profile:

Setting name	Configuration key	Possible values	Description
`StorageVolumeSize`	-	Volume size in `Mi`/`Gi`/`Ti`	Size of the volume to be mounted in the pod for local storage processing.
`StorageSyncFull`	`storage.sync`	`true`, `false`	Use normal synchronization mode (`false`) or full (`true`)
`StorageBacklogMemLimit`	`storage.backlog.mem_limit`	Memory in MB (for example: `5M`)	Maximum amount of memory to use when processing chunks from the backlog.
`StorageMaxChunksUp`	`storage.max_chunks_up`	Integers > `0`	Sets the maximum number of Chunks that can be up in memory.

Pipeline CLI commands

Use the following Pipeline CLI commands to interact with resource profiles.

Use a built-in resource profile during deployment

Specify any of the default profiles by using the following command:

calyptia create pipeline --core-instance CORE_INSTANCE --resource-profile PROFILE

Replace CORE_INSTANCE with the unique ID or name of the Core Instance, and PROFILE with the profile name.

Define a new resource profile

Use the following command to define your own resource profiles in Telemetry Pipeline:

calyptia create resource_profile --core-instance CORE_INSTANCE --name NEW_NAME --spec JSON_FILE

Replace CORE_INSTANCE with the unique ID or name of the Core Instance, and JSON_FILE with the JSON configuration file.

Resource profile JSON files follow this schema:

best-effort-low-resourcehigh-performance-optimal-throughputhigh-performance-guaranteed-delivery

This is an example of a resource profile definition:

{
 "resources": {
  "storage": {
   "syncFull": false,
   "backlogMemLimit": "5M",
   "volumeSize": "250Mi",
   "maxChunksUp": 128,
   "maxChunksPause": false
  },
  "cpu": {
   "limit": "0",
   "request": "0"
  },
  "memory": {
   "limit": "0",
   "request": "0"
  }
 }
}

A 0 value is defined as unlimited.

List available resource profiles

Use the following command to list the resource profiles defined within a Core Instance:

calyptia get resource_profiles --core-instance CORE_INSTANCE

Replace CORE_INSTANCE with the unique ID or name of the Core Instance.

Get details about a resource profile

Use the following command to get details about a specific resource profile:

calyptia get resource_profile PROFILE

Replace PROFILE with the unique ID or name of that resource profile.

Delete a resource profile

Use the following command to delete a specific resource profile:

calyptia delete resource_profile PROFILE

Replace PROFILE with the unique ID or name of that resource profile.

Pipeline configuration files Scaling and replicas