Join records
The join records processing rule combines values from multiple records into an array of values within a single record.
To determine which values to combine, this rule checks the timestamp of each record, then combines values from records that occur within the same nanosecond. This timestamp is a piece of metadata that Chronosphere Telemetry Pipeline assigns to all records that pass through a pipeline, and is unrelated to any keys within the record itself.
For a processing rule with the opposite effect, see split record.
Configuration parameters
Use the parameters in this section to configure this processing rule. The Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.
| Name | Key | Description | Default |
|---|---|---|---|
| Source key | key | Required. The key that contains values to join in an array. The resulting array will be assigned to a key with the same name. | none |
| Skip records where the specified key is missing checkbox | skipMissing | Indicates whether to modify the record if the key isn't found. | Not selected |
| Comment | comment | A custom note or description of the rule's function. This text is displayed next to the rule's name in the Actions list in the processing rules interface. | none |
Example
Using the join records processing rule lets you restructure your telemetry data by turning multiple standalone events into a combined array. For example, given these sample logs:
{"log":{"user_id":3,"action":"purchase"}}
{"log":{"user_id":4,"action":"click"}}
{"log":{"user_id":1,"action":"view"}}
{"log":{"user_id":5,"action":"click"}}
{"log":{"user_id":2,"action":"click"}}A processing rule with the Source key value log returns the following result:
{"log":[{"user_id":3,"action":"purchase"},{"user_id":4,"action":"click"},{"user_id":1,"action":"view"}]}
{"log":[{"user_id":5,"action":"click"},{"user_id":2,"action":"click"}]}This rule combined the log key from multiple standalone records into a series
of unified log arrays. Because the first three events occurred within the same
nanosecond, their log values were combined within one array, and because the
fourth and fifth events occurred one nanosecond later, their log values were
combined within another array.