Skip to main content
The nest keys processing rule moves the value of a specified source key into an object nested under a specified destination key.

​
Configuration parameters

Use the parameters in this section to configure the . The Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.
NameKeyDescriptionDefault
Source keysrcRequired. The key to nest under Destination key. This can be either a flat key/value pair or the key to an object.none
Destination keydstRequired. The key of the object to store your nested data. If no matching key exists, create a new key and set its value accordingly.none
Match case checkboxmatchCaseIndicates whether the regular expression is case-sensitive.Not selected
Regex engineregexEngineRequired. The engine to parse your regular expression. Accepted values: GNU, Oniguruma, PCRE2, POSIX, TRE.PCRE2
Regex mode checkboxregexModeIndicates whether to treat the value of Source key as a regular expression.Not selected / false
Merge checkboxmergeIf selected, and if Destination key is the key to an existing object that already contains data, the nest keys rule moves the new nested data into Destination key without overwriting its original data. However, if Destination key is a flat key/value pair, its value is overwritten and replaced with a new object.Not selected
CommentcommentA custom note or description of the rule’s function. This text is displayed next to the rule’s name in the Actions list in the processing rules interface.none

​
Examples

Using the nest keys processing rule lets you restructure your telemetry data by storing key/value pairs inside a parent object.

​
New object

For example, given this sample log data: 
{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"page_id":50,"action":"view"}
A processing rule with the Source key value user_id and the Destination key value account returns the following result: 
{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"account":{"user_id":3},"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"account":{"user_id":4},"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"account":{"user_id":1},"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"account":{"user_id":5},"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"account":{"user_id":1},"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"account":{"user_id":2},"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"account":{"user_id":3},"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"account":{"user_id":1},"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"account":{"user_id":2},"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"account":{"user_id":2},"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"account":{"user_id":1},"page_id":50,"action":"view"}
This rule moved the user_id key/value pair into a new object named account.

​
Merge into existing object

You can also use the Merge setting to nest data within an existing object. For example, given this sample log data: 
{"timestamp":"2023-03-28T09:08:41.64283645Z","user":{"account":{"vip":"no","id":3},"device":"mobile"},"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user":{"account":{"vip":"yes","id":4},"device":"desktop"},"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user":{"account":{"vip":"no","id":1},"device":"desktop"},"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user":{"account":{"vip":"yes","id":5},"device":"mobile"},"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user":{"account":{"vip":"no","id":1},"device":"other"},"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user":{"account":{"vip":"yes","id":2},"device":"mobile"},"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user":{"account":{"vip":"no","id":3},"device":"desktop"},"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user":{"account":{"vip":"no","id":1},"device":"desktop"},"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user":{"account":{"vip":"yes","id":2},"device":"desktop"},"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user":{"account":{"vip":"yes","id":2},"device":"other"},"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user":{"account":{"vip":"no","id":1},"device":"mobile"},"page_id":50,"action":"view"}
A processing rule with the Source key value action, the Destination key value user, and the Merge setting enabled returns the following result: 
{"user":{"account":{"id":3,"vip":"no"},"action":"purchase","device":"mobile"},"timestamp":"2023-03-28T09:08:41.64283645Z","page_id":30,"action":"purchase"}
{"user":{"account":{"id":4,"vip":"yes"},"action":"purchase","device":"desktop"},"timestamp":"2023-03-28T09:08:42.643343109Z","page_id":10,"action":"purchase"}
{"user":{"account":{"id":1,"vip":"no"},"action":"click","device":"desktop"},"timestamp":"2023-03-28T09:08:48.643600498Z","page_id":50,"action":"click"}
{"user":{"account":{"id":5,"vip":"yes"},"action":"purchase","device":"mobile"},"timestamp":"2023-03-28T09:08:50.643773688Z","page_id":40,"action":"purchase"}
{"user":{"account":{"id":1,"vip":"no"},"action":"purchase","device":"other"},"timestamp":"2023-03-28T09:08:51.643932272Z","page_id":30,"action":"purchase"}
{"user":{"account":{"id":2,"vip":"yes"},"action":"click","device":"mobile"},"timestamp":"2023-03-28T09:08:56.644080944Z","page_id":40,"action":"click"}
{"user":{"account":{"id":3,"vip":"no"},"action":"click","device":"desktop"},"timestamp":"2023-03-28T09:09:03.64425954Z","page_id":30,"action":"click"}
{"user":{"account":{"id":1,"vip":"no"},"action":"view","device":"desktop"},"timestamp":"2023-03-28T09:09:03.644317046Z","page_id":20,"action":"view"}
{"user":{"account":{"id":2,"vip":"yes"},"action":"purchase","device":"desktop"},"timestamp":"2023-03-28T09:09:10.64447719Z","page_id":50,"action":"purchase"}
{"user":{"account":{"id":2,"vip":"yes"},"action":"view","device":"other"},"timestamp":"2023-03-28T09:09:17.644810963Z","page_id":10,"action":"view"}
{"user":{"account":{"id":1,"vip":"no"},"action":"view","device":"mobile"},"timestamp":"2023-03-28T09:09:20.644994805Z","page_id":50,"action":"view"}
This rule moved the action key/value pair into the user object, but kept the existing key/value pairs that were already stored in user. For a processing rule with the opposite effect, see flatten subrecord or lift submap.