Resource profiles
Chronosphere Telemetry Pipeline offers templates called resource profiles that let you strike a balance between pipelines’ performance and reliability. These templates include different combinations of settings optimized for various scenarios. You can also edit resource profiles to customize behavior related to buffering and resource consumption.
Resource profiles are applied to pipelines when you deploy them.
Profile specifications
Telemetry Pipeline includes these profile specifications:
Type | Description |
---|---|
Best Effort Low Resource ( best-effort-low-resource ) | - Persistent storage - Buffering configured - No checksum - Maximum of 5 MB of memory when processing backlog data |
High-Performance Optimal Throughput ( high-performance-optimal-throughput ) | - Minimal persistent storage - In-memory only option - No buffering |
High-Performance Guaranteed Delivery ( high-performance-guaranteed-delivery ) | - Larger persistent storage - Buffering configured - 256 MB memory for backlog data |
Resource profiles only support storage configurations. Memory and CPU limits can be set in the profile, but by default no limits are set.
Profile properties
Use the following settings to modify your profile:
Setting name | Configuration key | Possible values | Description |
---|---|---|---|
StorageVolumeSize | - | Volume size in Mi /Gi /Ti | Size of the volume to be mounted in the pod for local storage processing. |
StorageSyncFull | storage.sync | true , false | Use normal synchronization mode (false ) or full (true ) |
StorageBacklogMemLimit | storage.backlog.mem_limit | Memory in MB (for example: 5M ) | Maximum amount of memory to use when processing chunks from the backlog. |
StorageMaxChunksUp | storage.max_chunks_up | Integers > 0 | Sets the maximum number of Chunks that can be up in memory. |
Pipeline CLI commands
Use the following Pipeline CLI commands to interact with resource profiles.
Specify the resource profile for a new pipeline
Set the resource profile for a new pipeline by using the following command:
calyptia create pipeline --core-instance INSTANCE --name PIPELINE --resource-profile PROFILE
Replace the following:
INSTANCE
: The name of your Core Instance.PIPELINE
: A name for your new pipeline.PROFILE
: The name of the resource profile to use.
Update the resource profile for an existing pipeline
Update the resource profile for an existing pipeline by using the following command:
calyptia update pipeline --name PIPELINE --resource-profile PROFILE
Replace the following:
PIPELINE
: The name or unique ID of your pipeline.PROFILE
: The name of the resource profile to use.
Define a new resource profile
Use the following command to define your own resource profiles in Telemetry Pipeline:
calyptia create resource_profile --core-instance INSTANCE --name PROFILE --spec FILE_NAME
Replace the following:
INSTANCE
: The name of your Core Instance.PROFILE
: A name for your new resource profile.FILE_NAME
: The JSON configuration file that defines your resource profile.
Resource profile JSON files follow this schema:
best-effort-low-resourcehigh-performance-optimal-throughputhigh-performance-guaranteed-delivery
This is an example of a resource profile definition:
{
"resources": {
"storage": {
"syncFull": false,
"backlogMemLimit": "5M",
"volumeSize": "250Mi",
"maxChunksUp": 128,
"maxChunksPause": false
},
"cpu": {
"limit": "0",
"request": "0"
},
"memory": {
"limit": "0",
"request": "0"
}
}
}
A 0
value is defined as unlimited
.
List available resource profiles
Use the following command to list the resource profiles defined within a Core Instance:
calyptia get resource_profiles --core-instance CORE_INSTANCE
Replace CORE_INSTANCE
with the name of the Core Instance.
Get details about a resource profile
Use the following command to get details about a specific resource profile:
calyptia get resource_profile PROFILE
Replace PROFILE
with the name of that resource profile.
Delete a resource profile
Use the following command to delete a specific resource profile:
calyptia delete resource_profile PROFILE
Replace PROFILE
with the name of that resource profile.