Ingest Google Cloud metrics

​

Set up Google Cloud integration

1. Request infrastructure from Chronosphere. Chronosphere Support must create infrastructure for your Google Cloud integration for use with your tenant. Before proceeding, contact Chronosphere Support.
2. Modify the Google Cloud domain restriction constraint. If your Google Cloud organization restricts identities by domain, you must add the Observability Platform customer identity (C04aozwp9) as an allowed value in your policy.
3. Create a Google Cloud service account.
   A Google Cloud service account isn’t an Observability Platform service account. Observability Platform requires a Google Cloud service account in the user’s Google Cloud project to provide Observability Platform access to metrics using service account impersonation.
   For each Google Cloud project, create a service account with the monitoring.viewer role. To instead grant a more restricted set of permissions, implementing the principle of least privilege, create a custom role with permissions for:
   • monitoring.timeSeries.list
   • monitoring.metricDescriptors.list
   • monitoring.metricDescriptors.get
   • monitoring.monitoredResourceDescriptors.list
   • monitoring.monitoredResourceDescriptors.get
   Chronosphere recommends setting your project’s quota to twice your normal usage to protect against spikes and unexpected usage. Use the GCP Timeseries API Quota Usage Per Project dashboard to monitor project usage.
4. Add the Observability Platform principal to the Google Cloud service account. Each Google Cloud service account must grant access to the Observability Platform principal to impersonate them. Verify with your account team to ensure you have the correct principal. The Observability Platform principal format is similar to:
   Copy

   Ask AI

   gcp-metrics-<tenant-name>@chronosphere-xyz.iam.gserviceaccount.com
   
   The xyz portion of the address is dictated by your tenant provisioning. Grant the principal the iam.serviceAccountTokenCreator role.
5. Configure the integration in Observability Platform. Initialize the integration using the Google Cloud integration API by applying the configuration with Chronoctl or Terraform. For each Google Cloud project, supply the Google Cloud service account email (SERVICE-ACCOUNT@PROJECT-ID.iam.gserviceaccount.com) in the request to configure the integration, along with specific metrics to ingest using metric prefixes. See Metric Prefix Search for examples.

​

Example configuration

locals {
  // Email address of Observability Platform tenant-specific principal.
  chronosphere_sa_email = "SERVICE-ACCOUNT@PROJECT-ID.iam.gserviceaccount.com"

  // Google Cloud project containing monitoring data to be ingested into
  // Observability Platform.
  monitoring_project_id = "PROJECT_ID"
}

// Service account that will allow Observability Platform tenant-specific principal
// to impersonate it.
resource "google_service_account" "chronosphere" {
  project    = local.monitoring_project_id
  account_id = "chronosphere"
}

// Custom role with minimal set of permissions for principle of least privilege
// resource "google_project_iam_custom_role" "chronosphere" {
  project     = local.monitoring_project_id
  role_id     = "Google Cloud Integration Metrics Viewer"
  title       = "Google Cloud Integration Metrics Viewer"
  description = "Role for granting view access for Google Cloud integration"
  permissions = [
    "monitoring.metricDescriptors.list",
    "monitoring.timeSeries.list",
  ]
}

// The service account has the minimal set of permissions for access to the project
// containing the metric data so that it can retrieve time series data from the
// Google Cloud API.
resource "google_project_iam_member" "chronosphere" {
  project = local.monitoring_project_id
  role    = google_project_iam_custom_role.chronosphere.id
  member  = google_service_account.chronosphere.member
}

// The service account provides the Observability Platform tenant-specific principal with
// roles/iam.serviceAccountTokenCreator access so that it can impersonate it. Only
// the Observability Platform tenant-specific principal will be able to perform this
// impersonation.
data "google_iam_policy" "chronosphere" {
  binding {
    role    = "roles/iam.serviceAccountTokenCreator"
    members = ["serviceAccount:${local.chronosphere_sa_email}"]
  }
}

// Assign the token creator permission to the service account.
resource "google_service_account_iam_policy" "chronosphere" {
  service_account_id = google_service_account.chronosphere.name
  policy_data        = data.google_iam_policy.chronosphere.policy_data
}

​

Use metrics scopes for projects

• Google Cloud API cost savings: With metrics scoping, Observability Platform ingests metrics using the least number of API requests to Google Cloud. Metrics scoped projects introduce a fan in/out model of ingesting metrics from the underlying Google Cloud projects that will pack time series data points for each metric into the smallest set of API responses. The Google Cloud API cost savings are significant, particularly for users managing multiple Google Cloud projects.
• Configuration simplicity: With metrics scoping, users can manage a smaller set of Google Cloud service accounts and configuration in Observability Platform. This simplifies management of the Google Cloud integration.

​

Set up Observability Platform to receive Google Cloud data

​

View Google Cloud integrations

chronoctl gcp-metrics-integrations list

chronoctl gcp-metrics-integrations read SLUG

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

​

Create or update a Google Cloud integration

chronoctl gcp-metrics-integrations create --filename FILENAME

chronoctl gcp-metrics-integrations update --filename FILENAME

api_version: v1/config
kind: GcpMetricsIntegration
spec:
  name: NAME
  slug: SLUG
  service_account:
    client_email: EMAIL
  metric_groups:
    - project_id: PROJECT_ID
      prefixes:
        - PREFIX

resource "chronosphere_gcp_metrics_integration" "chronosphere_gcp_metrics" {
  name = "GCP Metrics"
  slug = "gcp-metrics"
  service_account {
    client_email = "metric-scoped-service-account@<project>.iam.gserviceaccount.com" (
  }
  metric_groups {
    project_id = "PROJECT_ID"
    prefixes = [
      "PREFIX",
    ]
  }

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

• NAME: (string) The name of the Google Cloud integration.
• SLUG: (string) The unique identifier of the Google Cloud integration.
• EMAIL: (string) The email address of the customer’s Google Cloud service account of the form SERVICE_ACCOUNT@PROJECT-ID.iam.gserviceaccount.com.
• PROJECT_ID: (string) The ID of the customer’s Google Cloud project to ingest metrics from.
• PREFIX: (list(string)) A list of metric prefixes to ingest from the Google Cloud project.

​

Delete a Google Cloud integration

chronoctl gcp-metrics-integrations delete SLUG

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

​

Metric information

​

Metrics availability

• Latency delay: Most Google Cloud metrics have a delay before they’re made available for querying by Google Cloud API. Metrics with an ingest delay exceeding seven hours won’t be ingested into Chronosphere. For example, Firebase database metrics can exceed this latency delay and won’t be ingested.
• Sample rate: Google Cloud metrics sample at different rates. Most metrics sample at 60-second intervals, with some as long as once per day.

• networking.googleapis.com/pod_flow
• networking.googleapis.com/vm_flow
• networking.googleapis.com/node_flow

​

Metric names

gcp_<MONITORED-RESOURCE-NAME>_<FULLY-QUALIFIED-METRIC-NAME>

• MONITORED-RESOURCE-NAME is the monitored resource type in Google Cloud.
• FULLY-QUALIFIED-METRIC-NAME is the metric name in Google Cloud, with characters like . and / convert to _ to be Prometheus compatible. The following examples are fully qualified metric names in Google Cloud before metric name sanitization:
  • bigquery.googleapis.com/job/num_in_flight
  • cloudsql.googleapis.com/database/available_for_failover
  • kubernetes.io/container/accelerator/memory_total

​

Metric labels

​

Finding Google Cloud metrics in Metrics Explorer

• All Google Cloud metrics start with the prefix gcp_. Searching for this prefix displays all Google Cloud metrics in the platform.
• Substring search is possible. For example, if the original Google Cloud metric name contains a substring like domain_reachable, searching for the substring returns the Google Cloud metric, along with other metrics containing the substring.

​

Metric mappings

​

Metric prefix search