OpenSearch destination plugin

The OpenSearch destination plugin lets you configure Chronosphere Telemetry Pipeline to send your data to OpenSearch.

Supported telemetry types

This plugin supports these telemetry types:

Logs	Metrics	Traces

Configuration parameters

Use the parameters in this section to configure your plugin. The Telemetry Pipeline web interface uses the values in the Name column to describe the parameters. Items in the Key column are the YAML keys to use in pipeline configuration files.

Required

Name	Key	Description	Default
Host	`host`	IP address or hostname of the target OpenSearch instance.	`127.0.0.1`
Port	`port`	TCP port of the target OpenSearch instance.	`9200`
Index	`index`	Index name.	`calyptia-core`
Logstash Format	`logstash_format`	Enables Logstash format compatibility. Accepted values: `true`, `false`.	`false`

Security and TLS

Name	Key	Description	Default
TLS	`tls`	Enables or disables TLS/SSL support. Accepted values: `true`, `false`.	`false`
TLS Certificate Validation	`tls.verify`	Enables or disables TLS/SSL certificate validation. TLS must be enabled for certificates to be validated. Accepted values: `off`, `on`.	`on`
TLS Debug Level	`tls.debug`	Sets TLS debug verbosity level. Accepted values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), `4` (Verbose).	`1`
CA Certificate File Path	`tls.ca_file`	Absolute path to CA certificate file.	none
Certificate File Path	`tls.crt_file`	Absolute path to certificate file.	none
Private Key File Path	`tls.key_file`	Absolute path to private key file.	none
Private Key Path Password	`tls.key_passwd`	Password for private key file.	none
TLS SNI Hostname Extension	`tls.vhost`	Hostname to be used for TLS SNI extension.	none

Advanced

Name	Key	Description	Default
Replace Dots	`replace_dots`	When enabled, replaces field name dots with underscore, required by versions of OpenSearch. Accepted values: `true`, `false`.	`false`
Type	`type`	Type Name.	`_doc`
Suppress Type Name	`suppress_type_name`	If true, mapping types are removed. For v7.0.0 or later. Accepted values: `true`, `false`.	`true`
Buffer Size	`buffer_size`	Specifies the buffer size used to read the response from the OpenSearch HTTP service. This option is useful for debugging purposes where is required to read full responses. Response size grows depending of the number of records inserted. To set an unlimited amount of memory set this value to `false`, otherwise the value must be according to the Unit Size specification	`512k`
Path	`path`	OpenSearch accepts new data on HTTP query path `/_bulk`. It's also possible to serve Elasticsearch behind a reverse proxy on a sub path. This option defines such path on the Telemetry Pipeline side. It adds only a path prefix in the indexing HTTP POST URI.	none
Generate ID	`generate_id`	When enabled, generates `_id` for outgoing records. This prevents duplicate records when retrying ES. Accepted values: `true`, `false`.	false
Write Operation	`write_operation`	Operation to use to write in bulk requests.	`create`
ID Key	`id_key`	If set, `_id` will be the value of the key from incoming records.	none
Replace Dots	`current_time_index`	Uses the current time for index generation instead of the message record. Accepted values: `true`, `false`.	`false`
Logstash Prefix	`logstash_prefix`	When `logstash_format` is enabled, the index name is composed using a prefix and the date. For example, if `logstash_prefix` is `mydata`, your index becomes `mydata-YYYY.MM.DD`. The last string appended belongs to the date when the data is being generated.	none
Logstash Prefix Key	`logstash_prefix_key`	When included, the value in the record that belongs to the key will be looked up and over-write the `logstash_prefix` for index generation. If the key/value isn't found in the record, the `logstash_prefix` option acts as a fallback. Nested keys are supported through record accessor patterns.	none
Logstash Date Format	`logstash_dateformat`	Time format (based on `strftime`) to generate the second part of the index name.	`%Y.%m.%d`
Time Key	`time_key`	When `logstash_format` is enabled, each record gets a new timestamp field. The `time_key` property defines the name of that field.	`@timestamp`
Time Key	`time_key_format`	When `logstash_format` is enabled, this property defines the format of the timestamp.	none
Time Key Nanos	`time_key_nanos`	When `logstash_format` is enabled, enabling this property sends nanosecond precision timestamps. Accepted values: `true`, `false`.	`false`
Include Tag Key	`inclulde_tag_key`	When enabled, it appends the tag name to the record. Accepted values: `true`, `false`.	`false`
Tag Key	`tag_key`	When `inclulde_tag_key` is enabled, this property defines the key name of the tag in the message.	`flb-key`

Basic Authentication

Name	Key	Description	Default
HTTP Username	`http_user`	Basic auth username.	none
HTTP Password	`http_passwd`	Basic auth password. Requires `http_user` to be set.	none

AWS Authentication

Name	Key	Description	Default
Enable AWS Authentication	`aws_auth`	Enable AWS Sigv4 authentication. Accepted values: `true`, `false`.	`false`
AWS Region	`aws_region`	AWS region of your service.	none
AWS STS Endpoint	`aws_sts_endpoint`	Custom endpoint for the AWS STS API, used with the `aws_role_arn` option.	none
AWS IAM Assume Role	`aws_role_arn`	ARN of an IAM role to assume. For example, for cross-account access.	none
AWS External ID	`aws_external_id`	Specifies an external ID for the STS API. Can be used with the `aws_role_arn` parameter if your role requires an external ID.	none

Advanced Networking

Name	Key	Description	Default
DNS Mode	`net.dns.mode`	Selects the primary DNS connection type, which can be `TCP` or `UDP`.	none
DNS Resolver	`net.dns.resolver`	Selects the primary DNS connection type, which can be `LEGACY` or `ASYNC`.	none
Prefer IPv4	`net.dns.prefer_ipv4`	Prioritizes IPv4 DNS results when trying to establish a connection. Accepted values: `true`, `false`.	`false`
Keepalive	`net.keepalive`	Enables or disables Keepalive support. Accepted values: `true`, `false`.	`true`
Keepalive Idle Timeout	`net.keepalive_idle_timeout`	Sets the maximum time allowed for an idle Keepalive connection.	`30s`
Max Connect Timeout	`net.connect_timeout`	Sets the maximum time allowed to establish a connection, which includes the TLS handshake.	`10s`
Max Connect Timeout Log Error	`net.connect_timeout_log_error`	Specifies whether to log an error on connection timeout. When disabled, the timeout is logged as a debug message. Accepted values: `true`, `false`.	`true`
Max Keepalive Recycle	`net.keepalive_max_recycle`	Sets the maximum number of times a keepalive connection can be used before it's retired.	`2000`
Source Address	`net.source_address`	Specifies the network address to bind for data traffic.	none

Debugging

Name	Key	Description	Default
Trace Output	`trace_output`	When enabled, print the OpenSearch API calls to `stdout`. For diagnostics only. Accepted values: `true`, `false`.	`false`
Trace Error	`trace_error`	When enabled, print the OpenSearch exceptions to `stderr`. For diagnostics only. Accepted values: `true`, `false`.	`false`

Oracle OpenTelemetry