Documentation Index
Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt
Use this file to discover all available pages before exploring further.
Google Cloud Platform provides
sinks, which control
how you can route log data to supported destinations. Use the
Google Cloud Pub/Sub destination to route log
data from Google Cloud Platform to Chronosphere Observability Platform.
Complete the following steps to route logs from Google Cloud Platform:
- Create a sink for logs.
- Create a service account.
- Create a pull subscription.
- Contact Chronosphere Support.
If you want to use Terraform to manage routing Google logs to Observability Platform,
refer to the Terraform example. After applying this
configuration, Terraform generates the information that you supply to
Chronosphere Support.
This integration is only intended to handle throughput with low volume. To determine
whether this integration is suited to your use case, contact
Chronosphere Support.
If you want to route metrics data from Google Cloud Platform, see
Ingest Google Cloud metrics.
Create a sink for logs
First, you need to create a sink that defines the service type and destination to
route your logs.
-
In the Google Cloud Logs Console, in the left navigation, click Log Router
to open the Log router page.
-
Follow the steps in the Google Cloud documentation to
create a sink.
-
Select Cloud Pub/Sub topic as the sink service.
-
Optional: Enter a filter expression that matches the log entries you want to
include.
See the
Logging query language
in the Google Cloud documentation for information about how to construct queries.
-
Complete the remaining steps to create the sink.
Next, create a service account.
Create a service account
After creating a sink, you create a service account in Google Cloud that
Observability Platform impersonates.
Before starting this process, obtain the Observability Platform principal for
Google Cloud. Click your profile icon from the menu bar and select My
Account. Copy the value next to Google Cloud Logs IAM Principal.
Each Google Cloud service account must grant access to the Observability
Platform principal to impersonate them.
- In the Google Cloud console,
create a service account.
- Add the Observability Platform principal you obtained previously to the Google
Cloud service account.
- Grant the principal the
iam.serviceAccountTokenCreator role.
- Enter information for the remaining fields to finish creating the service account.
Next, create a pull subscription.
Create a pull subscription
Add a pull subscription
to the Pub/Sub destination you previously created. Observability Platform initiates
requests to the Pub/Sub server to retrieve messages.
-
In the Google Cloud console,
Create a pull subscription.
-
In the Delivery type menu, select Pull.
-
Configure the following recommended settings:
| Setting | Value |
|---|
| Message retention duration | 23 hours* |
| Expiration period | Never expire |
| Acknowledgement deadline | 60 seconds |
| Exactly once delivery | No |
| Message ordering | No |
| Dead lettering | No |
| Retry policy | Retry after exponential backoff delay |
*Set Message retention duration to less than 24 hours to avoid
storage costs.
-
Grant access to the Pub/Sub you created by assigning the following roles to your
new internal service account:
roles/pubsub.subscriber
roles/pubsub.viewer
Lastly, contact Chronosphere and provide the details
about your pull subscription.
After completing the previous steps, contact Chronosphere Support and
provide the following information:
-
The name of your new Google Cloud service account email, such as:
SERVICE-ACCOUNT@PROJECT-ID.iam.gserviceaccount.com
SERVICE-ACCOUNT is the name of the Google Cloud service account you created
in create a service account.
PROJECT-ID is the ID of your Google Cloud project.
-
The name of the logging sink subscription you created, such as
projects/PROJECT-ID/subscriptions/SUBSCRIPTION
Chronosphere can then enable the logging integration to start routing your Google
Cloud logs to Observability Platform.
The following code provides an example for creating a single Google Cloud service
account in the a Google Cloud project, and enables Observability Platform to
impersonate and gain access.
locals {
// Email address of your Chronosphere Logs tenant-specific principal.
chronosphere_sa_email = "gcp-logs-TENANT@chronosphere-production-b.iam.gserviceaccount.com"
// Google Cloud project containing logging data to be ingested into
// Observability Platform.
logging_project_id = "PROJECT_ID"
// Organization ID of the Google Cloud organization containing the project.
org_id = "ORGANIZATION_ID"
}
// Creates Pub/Sub topic for logging sink.
resource "google_pubsub_topic" "main" {
name = "logging_sink"
project = local.project_id
}
// Creates a new logging sink.
resource "google_logging_organization_sink" "main" {
org_id = local.org_id
name = "all_logs_sink"
destination = "pubsub.googleapis.com/${google_pubsub_topic.main.id}"
include_children = true
}
// Grants sink permission to roles so they can write to the Pub/Sub topic.
data "google_iam_policy" "topic" {
binding {
role = "roles/pubsub.publisher"
members = [
google_logging_organization_sink.main.writer_identity,
]
}
}
// Binds the role to a new topic.
resource "google_pubsub_topic_iam_policy" "main" {
project = local.project_id
topic = google_pubsub_topic.main.name
policy_data = data.google_iam_policy.topic.policy_data
}
// Creates a new Pub/Sub subscription.
resource "google_pubsub_subscription" "main" {
name = "all_logs_sink"
project = local.project_id
topic = google_pubsub_topic.main.name
message_retention_duration = "23h"
ack_deadline_seconds = 60
retry_policy {
minimum_backoff = "10s"
}
enable_message_ordering = false
enable_exactly_once_delivery = false
}
// Creates a policy granting the new Chronosphere logs service account access to
// the subscription.
data "google_iam_policy" "subscription" {
binding {
role = "roles/pubsub.viewer"
members = [
google_service_account.chronosphere_logs.member
]
}
binding {
role = "roles/pubsub.subscriber"
members = [
google_service_account.chronosphere_logs.member
]
}
}
// Binds the new policy to the subscription.
resource "google_pubsub_subscription_iam_policy" "main" {
project = local.project_id
subscription = google_pubsub_subscription.main.name
policy_data = data.google_iam_policy.subscription.policy_data
}
// Service account that lets an Observability Platform tenant-specific principal
// to impersonate it.
resource "google_service_account" "chronosphere_logs" {
project = local.project_id
account_id = "chronosphere-logs"
}
// The service account provides the Observability Platform tenant-specific principal with
// roles/iam.serviceAccountTokenCreator access so that it can impersonate it. Only
// the Observability Platform tenant-specific principal can perform this
// impersonation.
data "google_iam_policy" "chronosphere_logs" {
binding {
role = "roles/iam.serviceAccountTokenCreator"
members = ["serviceAccount:${local.chronosphere_sa_email}"]
}
}
// Assigns the token creator permission to the service account.
resource "google_service_account_iam_policy" "chronosphere_logs" {
service_account_id = google_service_account.chronosphere_logs.name
policy_data = data.google_iam_policy.chronosphere_logs.policy_data
}