Skip to main content
The extract keys/values processing rule uses a regular expression to search for key/value pairs inside a string, then creates a structured object to store those key/value pairs. Additionally, the resulting output always includes a string of the original unstructured data. The resulting output includes both the original JSON data and a string of comma-separated values assigned to a key you specify.

​
Configuration parameters

Use the parameters in this section to configure the . The Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.
NameKeyDescriptionDefault
Source keysrcRequired. The key whose value contains key/value pairs to extract.none
Destination keydstRequired. The key of the object to store your structured key/value pairs. If you specify the name of an existing key, the original value of the key is overwritten.none
RegexregexRequired. The regular expression for extracting key/value pairs from the value of Source key. This expression must have two capture groups: the result of the first capture group becomes the name of a key, and the result of the second capture group becomes that key’s value.none
Regex engineregexEngineRequired. The engine to parse your regular expression. Accepted values: GNU, Oniguruma, PCRE2, POSIX, TRE.PCRE2
CommentcommentA custom note or description of the rule’s function. This text is displayed next to the rule’s name in the Actions list in the processing rules interface.none

​
Example

Using the extract keys/values rule lets you extract embedded data from a string and turn it into parsable key/value pairs. You can then use these key/value pairs in other processing rules or for general storage and analysis. For example, given the following sample website log data: 
{"log": "user_id:3,page_id:30,action:purchase"}
{"log": "user_id:4,page_id:10,action:purchase"}
{"log": "user_id:1,page_id:50,action:click"}
{"log": "user_id:5,page_id:40,action:purchase"}
{"log": "user_id:1,page_id:30,action:purchase"}
{"log": "user_id:2,page_id:40,action:click"}
{"log": "user_id:3,page_id:30,action:click"}
{"log": "user_id:1,page_id:20,action:view"}
{"log": "user_id:2,page_id:50,action:purchase"}
{"log": "user_id:2,page_id:10,action:view"}
{"log": "user_id:1,page_id:50,action:view"}
A processing rule with the Source key value log, the Destination key value extracted, the Regex value (\w+):(\w+), and the Regex engine value PCRE2 returns the following result: 
{"extracted":{"action":"purchase","user_id":"3","page_id":"30"},"log":"user_id:3,page_id:30,action:purchase"}
{"extracted":{"action":"purchase","user_id":"4","page_id":"10"},"log":"user_id:4,page_id:10,action:purchase"}
{"extracted":{"action":"click","user_id":"1","page_id":"50"},"log":"user_id:1,page_id:50,action:click"}
{"extracted":{"action":"purchase","user_id":"5","page_id":"40"},"log":"user_id:5,page_id:40,action:purchase"}
{"extracted":{"action":"purchase","user_id":"1","page_id":"30"},"log":"user_id:1,page_id:30,action:purchase"}
{"extracted":{"action":"click","user_id":"2","page_id":"40"},"log":"user_id:2,page_id:40,action:click"}
{"extracted":{"action":"click","user_id":"3","page_id":"30"},"log":"user_id:3,page_id:30,action:click"}
{"extracted":{"action":"view","user_id":"1","page_id":"20"},"log":"user_id:1,page_id:20,action:view"}
{"extracted":{"action":"purchase","user_id":"2","page_id":"50"},"log":"user_id:2,page_id:50,action:purchase"}
{"extracted":{"action":"view","user_id":"2","page_id":"10"},"log":"user_id:2,page_id:10,action:view"}
{"extracted":{"action":"view","user_id":"1","page_id":"50"},"log":"user_id:1,page_id:50,action:view"}
This rule extracted key/value pairs from the string stored in the log key and stored those key/value pairs in a new structured object named extracted. For a processing rule that performs a similar operation on escaped JSON strings, see decode JSON. For a processing rule that performs a similar operation on data not already formatted as key/value pairs, see parse.