Allow records

The allow records processing rule preserves any records that contain a key whose value matches a specified regular expression and removes all other records.

For a processing rule with the opposite effect, see block records.

Configuration

  • Key: the key whose value to check against. If this key isn't present in a given record, that record will be removed.
  • Regex: the regular expression that determines which records to preserve. This expression is evaluated against the value of the specified key.
  • Regex engine: the engine to parse your regular expression.
  • Match case: (Optional) indicates if the regular expression is case-sensitive.

Example usage

Using the allow records rule lets you pare down telemetry data by retaining only the records with key-value pairs that you explicitly choose to keep.

For example, given this sample log data:

{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"page_id":50,"action":"view"}

A processing rule with the Key value action and the Regex value purchase returns the following result:

{"user_id":3,"page_id":30,"action":"purchase","timestamp":"2023-03-28T09:08:41.64283645Z"}
{"user_id":4,"page_id":10,"action":"purchase","timestamp":"2023-03-28T09:08:42.643343109Z"}
{"user_id":5,"page_id":40,"action":"purchase","timestamp":"2023-03-28T09:08:50.643773688Z"}
{"user_id":1,"page_id":30,"action":"purchase","timestamp":"2023-03-28T09:08:51.643932272Z"}
{"user_id":2,"page_id":50,"action":"purchase","timestamp":"2023-03-28T09:09:10.64447719Z"}

This rule retained the records associated with user purchases and removed the records associated with other actions, such as pageviews or clicks.