Decode JSON
The decode JSON processing rule transforms an escaped JSON string into a structured JSON object. The resulting output includes both the original string and a new JSON object assigned to the key of your choosing.
For a processing rule with the opposite effect, see encode JSON. For a processing rule with a similar effect that works on any string, see extract keys/values.
Configuration parameters
Use the parameters in this section to configure this processing rule. The Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.
| Name | Key | Description | Default |
|---|---|---|---|
| Source key | src | Required. The key that contains the escaped JSON string to transform. | log |
| Destination key | dst | Required. The key to store your transformed JSON object. If a key with this name already exists, the decode JSON rule will overwrite it. | decoded_json |
| Comment | comment | A custom note or description of the rule's function. This text is displayed next to the rule's name in the Actions list in the processing rules interface. | none |
Example
Using the decode JSON rule lets you extract embedded data from a string and turn it into parsable key/value pairs. You can then use these key/value pairs in other processing rules or for general storage and analysis.
For example, given the following sample website log data:
{"log": "{\"timestamp\":\"2023-03-28T09:08:41.64283645Z\",\"user_id\":3,\"page_id\":30,\"action\":\"purchase\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:08:42.643343109Z\",\"user_id\":4,\"page_id\":10,\"action\":\"purchase\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:08:48.643600498Z\",\"user_id\":1,\"page_id\":50,\"action\":\"click\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:08:50.643773688Z\",\"user_id\":5,\"page_id\":40,\"action\":\"purchase\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:08:51.643932272Z\",\"user_id\":1,\"page_id\":30,\"action\":\"purchase\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:08:56.644080944Z\",\"user_id\":2,\"page_id\":40,\"action\":\"click\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:09:03.64425954Z\",\"user_id\":3,\"page_id\":30,\"action\":\"click\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:09:03.644317046Z\",\"user_id\":1,\"page_id\":20,\"action\":\"view\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:09:10.64447719Z\",\"user_id\":2,\"page_id\":50,\"action\":\"purchase\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:09:17.644810963Z\",\"user_id\":2,\"page_id\":10,\"action\":\"view\"}\n"}
{"log": "{\"timestamp\":\"2023-03-28T09:09:20.644994805Z\",\"user_id\":1,\"page_id\":50,\"action\":\"view\"}\n"}A processing rule with the Source key value log and the Destination key
value output returns the following result:
{"log":"{\"timestamp\":\"2023-03-28T09:08:41.64283645Z\",\"user_id\":3,\"page_id\":30,\"action\":\"purchase\"}\n","output":{"timestamp":"2023-03-28T09:08:41.64283645Z","page_id":30,"user_id":3,"action":"purchase"}}
{"log":"{\"timestamp\":\"2023-03-28T09:08:42.643343109Z\",\"user_id\":4,\"page_id\":10,\"action\":\"purchase\"}\n","output":{"timestamp":"2023-03-28T09:08:42.643343109Z","page_id":10,"user_id":4,"action":"purchase"}}
{"log":"{\"timestamp\":\"2023-03-28T09:08:48.643600498Z\",\"user_id\":1,\"page_id\":50,\"action\":\"click\"}\n","output":{"timestamp":"2023-03-28T09:08:48.643600498Z","page_id":50,"user_id":1,"action":"click"}}
{"log":"{\"timestamp\":\"2023-03-28T09:08:50.643773688Z\",\"user_id\":5,\"page_id\":40,\"action\":\"purchase\"}\n","output":{"timestamp":"2023-03-28T09:08:50.643773688Z","page_id":40,"user_id":5,"action":"purchase"}}
{"log":"{\"timestamp\":\"2023-03-28T09:08:51.643932272Z\",\"user_id\":1,\"page_id\":30,\"action\":\"purchase\"}\n","output":{"timestamp":"2023-03-28T09:08:51.643932272Z","page_id":30,"user_id":1,"action":"purchase"}}
{"log":"{\"timestamp\":\"2023-03-28T09:08:56.644080944Z\",\"user_id\":2,\"page_id\":40,\"action\":\"click\"}\n","output":{"timestamp":"2023-03-28T09:08:56.644080944Z","page_id":40,"user_id":2,"action":"click"}}
{"log":"{\"timestamp\":\"2023-03-28T09:09:03.64425954Z\",\"user_id\":3,\"page_id\":30,\"action\":\"click\"}\n","output":{"timestamp":"2023-03-28T09:09:03.64425954Z","page_id":30,"user_id":3,"action":"click"}}
{"log":"{\"timestamp\":\"2023-03-28T09:09:03.644317046Z\",\"user_id\":1,\"page_id\":20,\"action\":\"view\"}\n","output":{"timestamp":"2023-03-28T09:09:03.644317046Z","page_id":20,"user_id":1,"action":"view"}}
{"log":"{\"timestamp\":\"2023-03-28T09:09:10.64447719Z\",\"user_id\":2,\"page_id\":50,\"action\":\"purchase\"}\n","output":{"timestamp":"2023-03-28T09:09:10.64447719Z","page_id":50,"user_id":2,"action":"purchase"}}
{"log":"{\"timestamp\":\"2023-03-28T09:09:17.644810963Z\",\"user_id\":2,\"page_id\":10,\"action\":\"view\"}\n","output":{"timestamp":"2023-03-28T09:09:17.644810963Z","page_id":10,"user_id":2,"action":"view"}}
{"log":"{\"timestamp\":\"2023-03-28T09:09:20.644994805Z\",\"user_id\":1,\"page_id\":50,\"action\":\"view\"}\n","output":{"timestamp":"2023-03-28T09:09:20.644994805Z","page_id":50,"user_id":1,"action":"view"}}This rule retained the original log string and created a new JSON object
called output that contains a structured version of the same key/value pairs
that are embedded in log.