> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and modify pipelines

You can use Chronosphere Telemetry Pipeline to create, modify, and delete
pipelines.

## Create a pipeline

Select from the following methods to create a pipeline.

<Tabs>
  <Tab title="Web interface" id="ui-create">
    To create a pipeline using the Telemetry Pipeline web interface:

    1. Sign in to [Telemetry Pipeline](https://core.calyptia.com/).
    2. Open the project where you want to deploy a new pipeline.
    3. Go to **Core Instances**, and then click the name of the Core Instance running
       in the environment where you want to deploy a new pipeline.
    4. In the **Data Pipelines** section, click **Create Data Pipeline**.
    5. In the dialog that appears, configure the following options:
       * **Name**: The name to give your pipeline.
       * **Service type**: The [Service type](/ingest/pipeline/v2/configure/networking#service-types)
         to use for your pipeline.
       * **Workload type**: The [Workload type](/ingest/pipeline/v2/configure/kubernetes/workloads)
         to use for your pipeline.
       * **Storage class**: ([StatefulSet](/ingest/pipeline/v2/configure/kubernetes/workloads#additional-settings-for-statefulset-pipelines)
         pipelines only) The [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/)
         to use for each Pod's [PersistentVolume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/).
         After you set this value for a StatefulSet pipeline, you can't modify it.
       * **Retention policy when scaled**: ([StatefulSet](/ingest/pipeline/v2/configure/kubernetes/workloads#additional-settings-for-statefulset-pipelines)
         pipelines only) Specifies whether Kubernetes should retain or delete the
         [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
         (PVC) associated with each affected Pod if the StatefulSet pipeline is scaled down.
       * **Retention policy when deleted**: ([StatefulSet](/ingest/pipeline/v2/configure/kubernetes/workloads#additional-settings-for-statefulset-pipelines)
         pipelines only) Specifies whether Kubernetes should retain or delete each Pod's PVC
         if the StatefulSet pipeline is deleted.
    6. Click **Save and continue**.
    7. Add and configure your [source plugins](/ingest/pipeline/plugins/source-plugins) and
       [destination plugins](/ingest/pipeline/plugins/destination-plugins).
    8. Add [processing rules](/ingest/pipeline/processing-rules) to transform the telemetry
       data as it passes through your pipeline.
    9. Click **Save and Deploy** to finish creating your pipeline.
  </Tab>

  <Tab title="Pipeline CLI" id="cli-create">
    To create a pipeline using [Pipeline CLI](/ingest/pipeline/pipeline-cli):

    1. Create a [pipeline configuration file](/ingest/pipeline/v2/configure/config-files).
       For example, this configuration file specifies a pipeline that uses
       [mock data](/ingest/pipeline/plugins/source-plugins/mock-data) as a source
       and [standard output](/ingest/pipeline/plugins/destination-plugins/standard-output)
       as a destination:

       ```yaml theme={null}
       pipeline:
          inputs:
             - dummy: '{"message":"dummy"}'
               rate: "1"
               samples: "0"
               start_time_sec: "-1"
               start_time_nsec: "-1"
               name: dummy
               tag: dummy.92a99f85-50b8-401d-a619-a52b14288f21
          filters:
             - name: lua
               match: dummy.92a99f85-50b8-401d-a619-a52b14288f21
          outputs:
             - format: json
               Name: stdout
               Match_Regex: ^(?!.*_calyptia_cloud).*$
       ```

    2. Run the following command:

       ```shell /INSTANCE/ /PIPELINE/ /VERSION/ /CONFIG/ theme={null}
       calyptia create pipeline \
       --core-instance INSTANCE \
       --name PIPELINE \
       --image ghcr.io/chronosphereio/core/calyptia-fluent-bit:VERSION
       --config-file CONFIG \
       ```

       Replace the following:

       * *`INSTANCE`*: The name of your Core Instance.

       * *`PIPELINE`*: The name to give your new pipeline.

       * *`VERSION`*: Optional: The latest version of the pipeline agent.
         You can find this value in the **Pipeline agent** column of the
         [Telemetry Pipeline component versions](/ingest/pipeline/v2/component-versions) table.
         For example, if the latest version of the pipeline agent is 1.23.0, provide the
         value `1.23.0` to use the Red Hat UBI [agent image](/ingest/pipeline/v2/configure/agent-image)
         or `1.23.0-distroless` to use the distroless agent image.

         If you don't specify a version number, Telemetry Pipeline uses the latest version
         of the Red Hat UBI agent image by default.

       * *`CONFIG`*: Optional: The name of your
         [pipeline configuration file](/ingest/pipeline/v2/configure/config-files). If you don't
         specify a configuration file, Telemetry Pipeline creates an empty placeholder
         that you can [modify at a later time](/ingest/pipeline/v2/configure/config-files#modify-a-pipelines-configuration-file).

         If you specify a configuration file, you can also add the `--skip-config-validation`
         flag to skip the validation process for that file.

    Telemetry Pipeline assigns a full name to each pipeline using the
    `[core-instance-name]-[pipeline_name]` naming schema, where `[core-instance-name]`
    is the name of your Core Instance and `[pipeline-name]` is the short name you gave
    your pipeline during the `calyptia create` process.
  </Tab>
</Tabs>

## Modify a pipeline

Select from the following methods to modify a pipeline.

<Note>
  To upgrade a pipeline to the latest version, you must use Pipeline CLI. For
  more information, see [Upgrade a pipeline](/ingest/pipeline/v2/install/upgrade#upgrade-a-pipeline).
</Note>

<Tabs>
  <Tab title="Web interface" id="ui-modify">
    To modify a pipeline in the Telemetry Pipeline web interface:

    1. Sign in to [Telemetry Pipeline](https://core.calyptia.com/).
    2. Open the project that contains the pipeline that you want to modify.
    3. Go to **Core Instances**, then click the name of the Core Instance associated
       with the pipeline you want to modify.
    4. Under **Data Pipelines**, click the name of the pipeline that you want to modify.
    5. Click **Edit**.
    6. Configure the [source](/ingest/pipeline/plugins/source-plugins),
       [destination](/ingest/pipeline/plugins/destination-plugins),
       and [processing rules](/ingest/pipeline/processing-rules) for your pipeline.
    7. Click **Save and Deploy** to save your changes.
    8. To manage advanced pipeline settings, go to the
       <Icon icon="square-pen" /> **Advanced Settings** page.
       The settings on this page include:
       * [Replicas](/ingest/pipeline/v2/configure/scaling)
       * [Deployment strategy](/ingest/pipeline/v2/configure/deployment-strategies)
       * [Version](/ingest/pipeline/v2/component-versions) and [agent image type](/ingest/pipeline/v2/configure/agent-image)
       * [Files](/ingest/pipeline/v2/configure/files)
       * [Secrets](/ingest/pipeline/v2/configure/secrets)
  </Tab>

  <Tab title="Pipeline CLI" id="cli-modify">
    To modify a pipeline using Pipeline CLI, follow these guides:

    * [Modify a pipeline's configuration file](/ingest/pipeline/v2/configure/config-files#modify-a-pipelines-configuration-file)
    * [Add a file to a pipeline](/ingest/pipeline/v2/configure/files#add-a-file-to-a-pipeline)
    * [Add a secret to a pipeline](/ingest/pipeline/v2/configure/secrets#add-a-secret)
    * [Update a pipeline's parsers](/ingest/pipeline/v2/build/parsers#custom-parsers)
    * [Add or remove replicas](/ingest/pipeline/v2/configure/scaling#fixed-number-of-replicas)
    * [Enable horizontal pod autoscaling](/ingest/pipeline/v2/configure/scaling#horizontal-pod-autoscaling)
    * [Modify a pipeline's Service type](/ingest/pipeline/v2/configure/networking#modify-the-service-type-of-an-existing-pipeline)
    * [Modify a pipeline's deployment strategy](/ingest/pipeline/v2/configure/deployment-strategies)
    * [Create pipeline sidecars](/ingest/pipeline/v2/configure/kubernetes/sidecars#create-a-sidecar)
    * [Set pipeline tolerations](/ingest/pipeline/v2/configure/kubernetes/tolerations#pipeline-tolerations)
    * [Pause a pipeline](/ingest/pipeline/v2/configure/scaling#pause-a-pipeline)
  </Tab>
</Tabs>

## Delete a pipeline

Select from the following methods to delete a pipeline.

<Tabs>
  <Tab title="Web interface" id="ui-delete">
    To delete a pipeline in the Telemetry Pipeline web interface:

    1. Sign in to [Telemetry Pipeline](https://core.calyptia.com/).
    2. Open the project that contains the pipeline that you want to delete.
    3. Go to **Core Instances**, then click the name of the Core Instance associated
       with the pipeline you want to delete.
    4. Under **Data Pipelines**, find the pipeline that you want to delete, and
       then click **<Icon icon="settings" />Settings <span aria-label="and then">></span> <Icon icon="trash" />Delete**.
    5. In the confirmation dialog that appears, click <Icon icon="trash" />**Delete**
       to permanently delete your pipeline.
  </Tab>

  <Tab title="Pipeline CLI" id="cli-delete">
    To delete a pipeline using [Pipeline CLI](/ingest/pipeline/pipeline-cli):

    1. Run the following command:

       ```shell /PIPELINE/ theme={null}
       calyptia delete pipeline PIPELINE
       ```

       Replace *`PIPELINE`* with either the name or [unique ID](/ingest/pipeline/v2/configure/metadata#get-pipeline-ids)
       of the pipeline to delete.

    2. Optional: If you installed Telemetry Pipeline in a Kubernetes cluster, use
       `kubectl` to delete the CRDs associated with your pipeline. This ensures no
       stray pipeline resources persist in your cluster. To do so, run the following
       command:

       ```shell /NAME/ theme={null}
       kubectl delete pipeline NAME
       ```

       Replace *`NAME`* with the name of the pipeline you deleted in the previous
       step.
  </Tab>
</Tabs>