> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt
> Use this file to discover all available pages before exploring further.

# GCP PubSub Input source plugin

export const entity_0 = "Google Cloud PubSub Input source plugin"

export const plugin_0 = "Google Cloud PubSub Input source plugin"

> Requires Chronosphere Telemetry Pipeline v2.72.0 or later.

The Google Cloud PubSub Input [source plugin](/ingest/pipeline/plugins/source-plugins)
(name: `gcp_pubsub`) ingests data from Google Cloud Pub/Sub into a telemetry pipeline.
You can use this plugin to ingest
[LogEntry](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry)
objects generated from a
[GCP Logging Sink](https://docs.cloud.google.com/logging/docs/routing/overview#sinks)
with a [pull-type](https://docs.cloud.google.com/pubsub/docs/subscriber)
topic subscription.

This is a [push-based](/ingest/pipeline/plugins/source-plugins#push-based-and-pull-based-source-plugins) source plugin.

## Supported telemetry types

The {plugin_0} for Chronosphere Telemetry Pipeline supports these telemetry types:

|                    Logs                    |             Metrics             |              Traces             |
| :----------------------------------------: | :-----------------------------: | :-----------------------------: |
| <Icon icon="circle-check" color="green" /> | <Icon icon="ban" color="red" /> | <Icon icon="ban" color="red" /> |

## Configuration parameters

Use the parameters in this section to configure the {entity_0}. The
Telemetry Pipeline web interface uses the items in the **Name** column to
describe these parameters. [Pipeline configuration files](/ingest/pipeline/v2/configure/config-files)
use the items in the **Key** column as YAML keys.

### General

| Name                | Key               | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | Default |
| ------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| **Subscription ID** | `subscription_id` | Required. This value can be either your [Pub/Sub subscription ID](https://docs.cloud.google.com/pubsub/docs/create-subscription) or your [fully qualified subscription name](https://www.google.com/url?q=https://docs.cloud.google.com/pubsub/docs/pubsub-basics%23resource_names\&sa=D\&source=docs\&ust=1765569146016537\&usg=AOvVaw3gxdFgpPaceHZPqRl3eZsC). However, if you don't use a fully qualified subscription name, you must also set the `project_id` parameter. | *none*  |
| **Project ID**      | `project_id`      | Your [Google Cloud project ID](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects). This parameter is required if the value of `subscription_id` isn't a fully qualified subscription name.                                                                                                                                                                                                                                                      | *none*  |

### Advanced

| Name   | Key             | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | Default |
| ------ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| *none* | `mem_buf_limit` | For pipelines with the Deployment or DaemonSet [workload](/ingest/pipeline/v2/configure/kubernetes/workloads) type only. Sets a limit for how much buffered data the plugin can write to memory, which affects [backpressure](/ingest/pipeline/v2/configure/backpressure). This value must follow Fluent Bit's rules for [unit sizes](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit#unit-sizes). If unspecified, no limit is enforced. In addition to this parameter, the subscription client for the Google Cloud PubSub Input source plugin has its own [queue limit](#queue-limit). | *none*  |

## Authentication

Use one of the following methods to authenticate with Google Cloud.

### Use GKE workload identity

Use this method to provide credentials to ADC through
[an attached service account](https://docs.cloud.google.com/docs/authentication/application-default-credentials#attached-sa).

<Note>
  This method is supported only if you installed Telemetry Pipeline in a GKE cluster.
  Additionally, this method assumes that Telemetry Pipeline is installed in the same
  GCP project as your Pub/Sub topic subscription.
</Note>

1. Identify the Kubernetes namespace and service account associated with your
   [Core Instance](/ingest/pipeline/v2/concepts#core-operators-and-core-instances).

2. In Google Cloud, create an IAM service account, or identify an existing IAM
   service account to associate with your Kubernetes service account.

3. Grant your IAM service account the `roles/pubsub.subscriber` role:

   ```shell theme={null}
   gcloud projects add-iam-policy-binding IAM_SA_PROJECT_ID \
       --member "serviceAccount:IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com" \
        --role "roles/pubsub.subscriber"
   ```

   Replace the following values:

   * `IAM_SA_PROJECT_ID`: The Google Cloud project ID associated with your IAM
     service account.
   * `IAM_SA_NAME`: The name of your IAM service account.

4. Create an IAM allow policy that gives your Kubernetes service account access
   to impersonate the IAM service account:

   ```shell theme={null}
   gcloud iam service-accounts add-iam-policy-binding IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com \
       --role roles/iam.workloadIdentityUser \
       --member "serviceAccount:IAM_SA_PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"
   ```

   Replace the following values:

   * `IAM_SA_PROJECT_ID:` The Google Cloud project ID associated with your IAM service account.
   * `IAM_SA_NAME:` The name of your IAM service account.
   * `NAMESPACE`: The Kubernetes namespace associated with your pipeline.
   * `KSA_NAME`: The Kubernetes service account associated with your pipeline.

5. Annotate your Kubernetes service account with the name and project ID of
   your IAM policy:

   ```shell theme={null}
   kubectl annotate serviceaccount KSA_NAME \
       --namespace NAMESPACE \
       iam.gke.io/gcp-service-account=IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com
   ```

   Replace the following values:

   * `IAM_SA_PROJECT_ID:` The Google Cloud project ID associated with your IAM
     service account.
   * `IAM_SA_NAME:` The name of your IAM service account.
   * `NAMESPACE`: The Kubernetes namespace associated with your pipeline.
   * `KSA_NAME`: The Kubernetes service account associated with your pipeline.

6. In Pipeline CLI, use the `--service-account` flag when you create or update a
   pipeline that contains this plugin:

   ```shell theme={null}
   calyptia {create|update} pipeline --service-account KSA_NAME
   ```

   Replace `KSA_NAME` with the Kubernetes service account associated with your pipeline.

### Use GCP service account credentials

Use this method to provide credentials to ADC through
[the `GOOGLE_APPLICATION_CREDENTIALS` environment variable](https://docs.cloud.google.com/docs/authentication/application-default-credentials#GAC).

<Note>
  This method is supported for all Telemetry Pipeline installation methods, including
  Telemetry Pipeline installations that aren't running in a GKE cluster.
</Note>

1. If you don't already have an IAM service account in GCP, create a new IAM
   service account.

2. [Create a new IAM service account key](https://docs.cloud.google.com/iam/docs/keys-create-delete)
   to use with Telemetry Pipeline.

3. Download the IAM key that you created. This download should be a JSON file.

4. In your GKE cluster, create a Kubernetes secret from the JSON file that contains
   your IAM key.

5. Use the `--secret-volume-mount` flag when you create or update the pipeline that
   contains this plugin:

   ```shell theme={null}
   calyptia {create|update} pipeline --secret-volume-mount SECRET:KEY=GOOGLE_APPLICATION_CREDENTIALS
   ```

   Replace the following values:

   * `SECRET`: The name of the secret you created.
   * `KEY`: The key that contains your credentials in that secret.

   Don't replace `GOOGLE_APPLICATION_CREDENTIALS` with any other value.

## Queue limit

The subscription client for the GCP PubSub source plugin has a 300 MB queue limit.
This means the plugin will pause fetching data from Google Pub/Sub if its
subscription queue exceeds 300 MB.

If other constraints or limits are placed on your pipeline, the GCP PubSub source
plugin might stop fetching data before this queue limit is reached. For example, if
you use `mem_buf_limit` or `resources.storage.backlogMemLimit` to enforce a memory
limit, and that memory limit is reached, those values take precedence over the
300 MB queue limit.