Block records

The block records processing rule removes any records that contain a key whose value matches a specified regular expression and preserves all other records.

For a processing rule with the opposite effect, see allow records.

Configuration parameters

ParameterDescriptionDefault
KeyRequired. The key whose value to validate against. If this key isn't present in a given record, that record is preserved.log
RegexRequired. The regular expression that determines which records to remove. This expression is evaluated against the value of the specified key.none
Match case checkboxIf selected, the regular expression is case-sensitive.Not selected
Regex engineRequired. The engine to parse your regular expression. Accepted values: GNU, Oniguruma, PCRE2, POSIX, TRE.PCRE2
CommentA custom note or description of the rule's function. This text is displayed next to the rule's name in the Actions list in the processing rules interface.none

Example

Using the block records rule lets you pare down telemetry data by eliminating records with data you don't want to keep.

For example, given this sample log data:

{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"page_id":50,"action":"view"}

A processing rule with the Key value action and the Regex value purchase returns the following result:

{"user_id":1,"page_id":50,"action":"click","timestamp":"2023-03-28T09:08:48.643600498Z"}
{"user_id":2,"page_id":40,"action":"click","timestamp":"2023-03-28T09:08:56.644080944Z"}
{"user_id":3,"page_id":30,"action":"click","timestamp":"2023-03-28T09:09:03.64425954Z"}
{"user_id":1,"page_id":20,"action":"view","timestamp":"2023-03-28T09:09:03.644317046Z"}
{"user_id":2,"page_id":10,"action":"view","timestamp":"2023-03-28T09:09:17.644810963Z"}
{"user_id":1,"page_id":50,"action":"view","timestamp":"2023-03-28T09:09:20.644994805Z"}

This rule removed the records associated with user purchases and preserved the records associated with other actions, such as pageviews or clicks.