OpenSearch destination plugin
The OpenSearch destination plugin lets you configure Chronosphere Telemetry Pipeline to send your data to OpenSearch.
Supported telemetry types
This plugin supports these telemetry types:
Logs | Metrics | Traces |
---|---|---|
Configuration parameters
Use the parameters in this section to configure your plugin. The Telemetry Pipeline web interface uses the values in the Name column to describe the parameters. Items in the Key column are the YAML keys to use in pipeline configuration files.
Required
Name | Key | Description | Default |
---|---|---|---|
Host | host | IP address or hostname of the target OpenSearch instance. | 127.0.0.1 |
Port | port | TCP port of the target OpenSearch instance. | 9200 |
Index | index | Index name. | calyptia-core |
Logstash Format | logstash_format | Enable Logstash format compatibility. Accepted values: true , false . | false |
Security and TLS
Name | Key | Description | Default |
---|---|---|---|
TLS | tls | Enable or disable TLS/SSL support. Accepted values: true , false . | false |
TLS Certificate Validation | tls.verify | Turn TLS/SSL certificate validation on or off. TLS must be on for this setting to be enabled. | on |
TLS Debug Level | tls.debug | Set TLS debug verbosity level. Accepted values: 0 (No debug), 1 (Error), 2 (State change), 3 (Informational), 4 (Verbose). | 1 |
CA Certificate File Path | tls.ca_file | Absolute path to CA certificate file. | none |
Certificate File Path | tls.crt_file | Absolute path to certificate file. | none |
Private Key File Path | tls.key_file | Absolute path to private key file. | none |
Private Key Path Password | tls.key_passwd | Optional password for tls.key_file file. | none |
TLS SNI Hostname Extension | tls.vhost | Hostname to be used for TLS SNI extension. | none |
Advanced
Name | Key | Description | Default |
---|---|---|---|
Replace Dots | replace_dots | When enabled, replace field name dots with underscore, required by versions of OpenSearch. Accepted values: true , false . | false |
Type | type | Type Name. | _doc |
Suppress Type Name | suppress_type_name | If true, mapping types are removed. For v7.0.0 or later. Accepted values: true , false . | true |
Buffer Size | buffer_size | Specify the buffer size used to read the response from the OpenSearch HTTP service. This option is useful for debugging purposes where is required to read full responses. Response size grows depending of the number of records inserted. To set an unlimited amount of memory set this value to false , otherwise the value must be according to the Unit Size specification | 512k |
Path | path | OpenSearch accepts new data on HTTP query path /_bulk . It's also possible to serve Elasticsearch behind a reverse proxy on a sub path. This option defines such path on the Telemetry Pipeline side. It adds only a path prefix in the indexing HTTP POST URI. | none |
Generate ID | generate_id | When enabled, generate _id for outgoing records. This prevents duplicate records when retrying ES. Accepted values: true , false . | false |
Write Operation | write_operation | Operation to use to write in bulk requests. | create |
ID Key | id_key | If set, _id will be the value of the key from incoming records. | none |
Replace Dots | current_time_index | Use the current time for index generation instead of the message record. Accepted values: true , false . | false |
Logstash Prefix | logstash_prefix | When logstash_format is enabled, the index name is composed using a prefix and the date. For example, if logstash_prefix is mydata , your index becomes mydata-YYYY.MM.DD . The last string appended belongs to the date when the data is being generated. | none |
Logstash Prefix Key | logstash_prefix_key | When included, the value in the record that belongs to the key will be looked up and over-write the logstash_prefix for index generation. If the key/value isn't found in the record, the logstash_prefix option acts as a fallback. Nested keys are supported through record accessor patterns. | none |
Logstash Date Format | logstash_dateformat | Time format (based on strftime ) to generate the second part of the index name. | %Y.%m.%d |
Time Key | time_key | When logstash_format is enabled, each record gets a new timestamp field. The time_key property defines the name of that field. | @timestamp |
Time Key | time_key_format | When logstash_format is enabled, this property defines the format of the timestamp. | none |
Time Key Nanos | time_key_nanos | When logstash_format is enabled, enabling this property sends nanosecond precision timestamps. Accepted values: true , false . | false |
Include Tag Key | inclulde_tag_key | When enabled, it append the tag name to the record. Accepted values: true , false . | false |
Tag Key | tag_key | When inclulde_tag_key is enabled, this property defines the key name of the tag in the message. | flb-key |
Basic Authentication
Name | Key | Description | Default |
---|---|---|---|
HTTP Username | http_user | Basic auth username. | none |
HTTP Password | http_passwd | Basic auth password. Requires http_user to be set. | none |
AWS Authentication
Name | Key | Description | Default |
---|---|---|---|
Enable AWS Authentication | aws_auth | Enable AWS Sigv4 authentication. Accepted values: true , false . | false |
AWS Region | aws_region | AWS region of your service. | none |
AWS STS Endpoint | aws_sts_endpoint | Custom endpoint for the AWS STS API, used with the aws_role_arn option. | none |
AWS IAM Assume Role | aws_role_arn | ARN of an IAM role to assume. For example, for cross-account access. | none |
AWS External ID | aws_external_id | Specify an external ID for the STS API. Can be used with the aws_role_arn parameter if your role requires an external ID. | none |
Advanced Networking
Name | Key | Description | Default |
---|---|---|---|
DNS Mode | net.dns.mode | Select the primary DNS connection type, which can be TCP or UDP . | none |
DNS Resolver | net.dns.resolver | Select the primary DNS connection type, which can be LEGACY or ASYNC . | none |
Prefer IPv4 | net.dns.prefer_ipv4 | Prioritize IPv4 DNS results when trying to establish a connection. Accepted values: true , false . | false |
Keepalive | net.keepalive | Enable or disable Keepalive support. Accepted values: true , false . | true |
Keepalive Idle Timeout | net.keepalive_idle_timeout | Set maximum time allowed for an idle Keepalive connection. | 30s |
Max Connect Timeout | net.connect_timeout | Set maximum time allowed to establish a connection, which includes the TLS handshake. | 10s |
Max Connect Timeout Log Error | net.connect_timeout_log_error | On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. Accepted values: true , false . | true |
Max Keepalive Recycle | net.keepalive_max_recycle | Set maximum number of times a keepalive connection can be used before it's retired. | 2000 |
Source Address | net.source_address | Specify network address to bind for data traffic. | none |
Debugging
Name | Key | Description | Default |
---|---|---|---|
Trace Output | trace_output | When enabled, print the OpenSearch API calls to stdout . For diagnostics only. Accepted values: true , false . | false |
Trace Error | trace_error | When enabled, print the OpenSearch exceptions to stderr . For diagnostics only. Accepted values: true , false . | false |