Skip to main content
The search/replace value processing rule uses regular expressions to search for a value inside a string, then replaces that value with a different specified value.

​
Configuration parameters

Use the parameters in this section to configure the . The Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.
NameKeyDescriptionDefault
KeykeyRequired. The key whose value contains the string to replace.none
RegexregexRequired. The regular expression that determines which values to replace. You can use capture groups to specify more than one value.none
Match case checkboxmatchCaseIndicates whether the regular expression is case-sensitive.Not selected
Maximum number of substitutionssubmaxThe maximum number of substitutions that this rule will perform within a given record.none
Replacement valuereplacementRequired. The regular expression substitution pattern to replace the values specified by Regex. You can use the % character to use capture groups in your substitution pattern.none
Regex engineregexEngineRequired. The engine to parse your regular expression. Accepted values: GNU, Oniguruma, PCRE2, POSIX, TRE.PCRE2
CommentcommentA custom note or description of the rule’s function. This text is displayed next to the rule’s name in the Actions list in the processing rules interface.none

​
Examples

Using the search/replace value rule lets you modify and overwrite your telemetry data.

​
Basic substitution

For example, given these sample logs: 
{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"page_id":50,"action":"view"}
A processing rule with the Key value action, the Regex value purchase, and the Replacement value checkout returns the following result: 
{"user_id":3,"page_id":30,"timestamp":"2023-03-28T09:08:41.64283645Z","action":"checkout"}
{"user_id":4,"page_id":10,"timestamp":"2023-03-28T09:08:42.643343109Z","action":"checkout"}
{"user_id":1,"page_id":50,"timestamp":"2023-03-28T09:08:48.643600498Z","action":"click"}
{"user_id":5,"page_id":40,"timestamp":"2023-03-28T09:08:50.643773688Z","action":"checkout"}
{"user_id":1,"page_id":30,"timestamp":"2023-03-28T09:08:51.643932272Z","action":"checkout"}
{"user_id":2,"page_id":40,"timestamp":"2023-03-28T09:08:56.644080944Z","action":"click"}
{"user_id":3,"page_id":30,"timestamp":"2023-03-28T09:09:03.64425954Z","action":"click"}
{"user_id":1,"page_id":20,"timestamp":"2023-03-28T09:09:03.644317046Z","action":"view"}
{"user_id":2,"page_id":50,"timestamp":"2023-03-28T09:09:10.64447719Z","action":"checkout"}
{"user_id":2,"page_id":10,"timestamp":"2023-03-28T09:09:17.644810963Z","action":"view"}
{"user_id":1,"page_id":50,"timestamp":"2023-03-28T09:09:20.644994805Z","action":"view"}
This rule searched for the string purchase within the value of the action key, then replaced any matching strings with the value checkout.

​
Capture groups

You can also use capture groups for more complex replacements. For example, given these sample logs: 
{"event":"Sign-on detected from user 'Alice' to server 83ad48."}
{"event":"Sign-on detected from user 'Bob' to server d6f442."}
{"event":"Sign-on detected from user 'Carol' to server fb65ee."}
{"event":"Sign-on detected from user 'Dave' to server e6ec7d."}
{"event":"Sign-on detected from user 'Erin' to server b0692c."}
{"event":"Sign-on detected from user 'Frank' to server 1c7cd2."}
{"event":"Sign-on detected from user 'Grace' to server 9626a3."}
A processing rule with the Key value event, the Regex value Sign-on detected from user '(.*)' to server (.*)., and the Replacement value login: %1 at %2 returns the following result: 
{"event":"login: Alice at 83ad48"}
{"event":"login: Bob at d6f442"}
{"event":"login: Carol at fb65ee"}
{"event":"login: Dave at e6ec7d"}
{"event":"login: Erin at b0692c"}
{"event":"login: Frank at 1c7cd2"}
{"event":"login: Grace at 9626a3"}
This rule searched for username and server name strings within the value of the event key, then replaced the value of event with a shorter message that still retained the names contained in the original message.