When ingesting data with network sources or agents, you may want to scale the pipeline to process additional records. With Calyptia Core, all ingested data is automatically load balanced within a pipeline replica.

Within Advanced Settings, increase the number of replicas to the desired amount.

To see these steps in real time, follow along with an interactive demo (opens in a new tab).

For replica count recommendations over 100 TB a day, contact the Calyptia team prior to scaling up to ensure proper stability.

Calyptia CLI

Scaling up an existing Calyptia Core pipeline can be performed using the update command:

calyptia update pipeline PIPELINE_NAME_OR_ID --replicas NUM_OF_REPLICAS

To examine the number of replicas for a given pipeline, run:

calyptia get pipelines --core-instance CORE_INSTANCE

The output is similar to the following:

 NAME                 REPLICAS  STATUS   AGE
 pipeline-name               3  STARTED  15 hours