Google Cloud Operations Suite destination plugin
Google Cloud Operations Suite, formerly known as Stackdriver, is a cloud-based management platform designed to provide observability and insight into the performance, uptime, and overall health of applications and infrastructure running on Google Cloud and other cloud environments.
The suite includes a range of integrated tools such as logging, monitoring, trace, debugging, and error reporting to help developers and IT teams identify, troubleshoot, and resolve issues quickly. The platform also provides real-time metrics and alerts, letting teams proactively monitor and optimize their applications and services for optimal performance and reliability.
The Google Cloud Operations Suite destination plugin in Calyptia Core lets you configure your pipeline to stream your log data and metrics to Google Cloud Operations Suite.
Configuration parameters
The Google Cloud Operations Suite destination plugin provides these configuration parameters.
General
| Key | Description |
|---|---|
| Google Service Credentials Path | The Service Credentials file lets Calyptia Core to communicate directly with Google Cloud Services. Read the following on how to set up service credentials: https://cloud.google.com/logging/docs/agent/logging/authorization#create-service-account (opens in a new tab). |
| Google Cloud Service Account Email | Account email associated to the service. Only available if no credentials file has been provided. |
| Google Cloud Service Account Secret | Private key content associated with the service account. Only available if no credentials file has been provided. |
| Metadata Server | Private key content associated with the service account. Only available if no credentials file has been provided. |
Advanced
| Key | Description |
|---|---|
| Region Location | The Google Cloud or AWS region in which to store data about the resource. If the resource type is one of the generic_node or generic_task, this field is required. |
| Namespace Identifier | A namespace identifier, such as a cluster name or environment. If the resource type is one of the generic_node or generic_task, this field is required. |
| Node ID Identifier | A unique identifier for the node within the namespace, such as hostname or IP address. If the resource type is generic_node, then this field is required. |
| Job Identifier | An identifier for a grouping of related task, such as the name of a microservice or distributed batch. If the resource type is generic_task, then this field is required. |
| Task ID Identifier | A unique identifier for the task within the namespace and job, such as a replica index identifying the task within the job. If the resource type is generic_task, then this field is required. |
| Google Cloud Project ID | The Google Cloud project that should receive these logs.Defaults to the project ID of the google_service_credentials file, or the project_id from Google's metadata.google.internal server. |
| Resource Type | Set resource type of data. Supported resource types: k8s_container, k8s_node, k8s_pod, global, generic_node, generic_task, and gce_instance. |
| Kubernetes Cluster Name | The name of the cluster that the container (node or pod based on the resource type) is running in. If the resource type is one of the k8s_container, k8s_node or k8s_pod, then this field is required. |
| Kubernetes Cluster Location | The physical location of the cluster that contains (node or pod based on the resource type) the container. If the resource type is one of the k8s_container, k8s_node or k8s_pod, then this field is required. |
| Labels Key | The value of this field is used by the Stackdriver output plugin to find the related labels from jsonPayload and then extract the value of it to set the LogEntry Labels. |
| Labels | Optional list of comma separated of strings specifying key=value pairs. The resulting labels will be combined with the elements in obtained from labels_key to set the LogEntry Labels. Elements from labels will override duplicate values from labels_key. |
| Log Name Key | The value of this field is used by the Stackdriver output plugin to extract logName from jsonPayload and set the logName field. |
| Tag Prefix | Set the tag_prefix used to validate the tag of logs with k8s resource type. Without this option, the tag of the log must be in format of k8s_container(pod/node).* in order to use the k8s_container resource type. Now the tag prefix is configurable by this option, being aware of the ending dot. |
| Severity Key | Specify the name of the key from the original record that contains the severity information. |
| Enable Cloud Trace Integration | Rewrite the trace field to include the projectID and format it for use with Cloud Trace. When this flag is enabled, the user can get the correct result by printing only the traceID (usually 32 characters). |
| Custom K8s Regex | Set a custom regex to extract field like pod_name, namespace_name, container_name and docker_id from the local_resource_id in logs. This is helpful if the value of pod or node name contains dots. |
Security and TLS
| Key | Description |
|---|---|
| TLS | Enable or disable TLS/SSL support. |
| TLS Certificate Validation | Turn TLS/SSL certificate validation on or off. TLS must be on for this setting to be enabled. |
| TLS Debug Level | Set TLS debug verbosity level. Accepts these values: 0 (No debug), 1 (Error), 2 (State change), 3 (Informational), 4 (Verbose). |
| CA Certificate File Path | Absolute path to CA certificate file. |
| Certificate File Path | Absolute path to certificate file. |
| Private key File Path | Absolute path to private key file. |
| Private Key Path Password | Optional password for tls.key_file file. |
| TLS SNI Hostname Extension | Hostname to be used for TLS SNI extension. |
Configuration
| Key | Description |
|---|---|
| DNS Mode | Select the primary DNS connection type (TCP or UDP). |
| DNS Resolver | Select the primary DNS connection type (TCP or UDP). |
| Prefer IPv4 | Prioritize IPv4 DNS results when trying to establish a connection. |
| Keepalive | Enable or disable Keepalive support. |
| Keepalive Idle Timeout | Set maximum time allowed for an idle Keepalive connection. |
| Max Connect Timeout | Set maximum time allowed to establish a connection, this time includes the TLS handshake. |
| Max Connect Timeout Log Error | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. |
| Max Keepalive Recycle | Set maximum number of times a keepalive connection can be used before it is retired. |
| Source Address | Specify network address to bind for data traffic. |