Skip to main content
In addition to the YAML configuration file that govern a pipeline’s settings, you can optionally add other kinds of files to pipelines, such as parsers, certificates, and custom Lua scripts.
Due to Kubernetes ConfigMap storage limitations, the total size of a pipeline and its associated resources can’t exceed 1 MiB. This limit includes the combined size of configuration files, secrets, parsers, and any other files.

​
Add a file to a pipeline

To add a file to a pipeline, use one of these methods:
  1. Sign in to Telemetry Pipeline.
  2. Open the project that contains the pipeline that you want to add a file to.
  3. Go to Core Instances, then click the name of the Core Instance associated with the pipeline that you want to add a file to.
  4. Under Data Pipelines, click the name of the pipeline that you want to add a file to.
  5. Click  Advanced Settings.
  6. Under Pipeline Files, click Upload File.
  7. Use one of these methods to upload a file:
    • Click Select File to choose a file from your device.
    • Enter a value in File name field and add content to the large text field to create a new file with that name and content.
  8. Optional: To encrypt your file, select the Encrypt file checkbox.
  9. Click Upload File.

​
Add a file to a Core Instance

Adding a file to a Core Instance doesn’t circumvent the 1 MiB resource limit for pipelines. If you add a file to a Core Instance, that file becomes a resource for every pipeline associated with that Core Instance, which increases the resource utilization for each pipeline accordingly.
You can also add a file to a Core Instance, which makes that file available to all pipelines associated with that Core Instance. To do so, use one of these methods:
  1. Sign in to Telemetry Pipeline.
  2. Open the project that contains the Core Instance that you want to add a file to.
  3. Go to Core Instances, then click the name of the Core Instance that you want to add a file to.
  4. Click  Advanced Settings.
  5. Under Pipeline Files, click Upload File.
  6. Use one of these methods to upload a file:
    • Click Select File to choose a file from your device.
    • Enter a value in File name field and add content to the large text field to create a new file with that name and content.
  7. Optional: To encrypt your file, select the Encrypt file checkbox.
  8. Click Upload File.

​
Reference a file

After you’ve added a file to a pipeline, you can reference it in a pipeline’s configuration file by using the following variable: 
{{ files.FILENAME }}
Replace FILENAME with the name of your file.

​
File encryption

For encrypted files, Telemetry Pipeline uses RSA‑OAEP encryption with a 2048-bit key. Files are unencrypted by default, but you can choose to encrypt a file when you add to it a pipeline or Core Instance.