Custom pipelines
Processing rules
Hash key

Hash key

The hash key processing rule copies the value of a specified source key, hashes that value, then stores the hashed copy in a specified destination key. This destination can be either a new or existing key.

Configuration parameters

  • Source key: Required. The key whose value you'd like to hash. The value of this key must be a string and not a number, Boolean, array, or object.
  • Destination key: Required. The key that stores the hashed value of the source key. If you specify the name of an existing key, the hash keys rule overwrites the value of that key. If you don't specify a key name, the hash key rule creates a new key with the name _SOURCE_HASH_, where SOURCE is the name of your Source Key and HASH is the name of your Hash algorithm.
  • Output scheme: Required. The scheme to store your hashed value. Options include hexadecimal, binary, and base64.
  • Hash algorithm: Required. The algorithm used to hash your value. Options include SHA256 and MD5.
  • Comment: A custom note or description of the rule's function. This text is displayed next to the rule's name in the Actions list in the processing rules interface.

Example

Using the hash keys rule lets you store values in hashed format, which can help validate the authenticity and integrity of your data.

For example, given the following sample website log data:

{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"page_id":50,"action":"view"}

A processing rule with the Source key value action, the Output scheme value Hexadecimal, and the Hash algorithm value SHA256 returns the following result:

{"user_id":3,"page_id":30,"action":"purchase","action_sha256":"7048dc8aa7663e713097535171b4ed72ab015705a885de9636e66d4b1fa18804","timestamp":"2023-03-28T09:08:41.64283645Z"}
{"user_id":4,"page_id":10,"action":"purchase","action_sha256":"7048dc8aa7663e713097535171b4ed72ab015705a885de9636e66d4b1fa18804","timestamp":"2023-03-28T09:08:42.643343109Z"}
{"user_id":1,"page_id":50,"action":"click","action_sha256":"e5c7ffac26fed654fe62045898f55b551a0dc120badf3d116bcd364418f3ec16","timestamp":"2023-03-28T09:08:48.643600498Z"}
{"user_id":5,"page_id":40,"action":"purchase","action_sha256":"7048dc8aa7663e713097535171b4ed72ab015705a885de9636e66d4b1fa18804","timestamp":"2023-03-28T09:08:50.643773688Z"}
{"user_id":1,"page_id":30,"action":"purchase","action_sha256":"7048dc8aa7663e713097535171b4ed72ab015705a885de9636e66d4b1fa18804","timestamp":"2023-03-28T09:08:51.643932272Z"}
{"user_id":2,"page_id":40,"action":"click","action_sha256":"e5c7ffac26fed654fe62045898f55b551a0dc120badf3d116bcd364418f3ec16","timestamp":"2023-03-28T09:08:56.644080944Z"}
{"user_id":3,"page_id":30,"action":"click","action_sha256":"e5c7ffac26fed654fe62045898f55b551a0dc120badf3d116bcd364418f3ec16","timestamp":"2023-03-28T09:09:03.64425954Z"}
{"user_id":1,"page_id":20,"action":"view","action_sha256":"2bcb43cbc8f6b7ef66331532881143fcbae60a879db3a8fb853f645bb24c2b3c","timestamp":"2023-03-28T09:09:03.644317046Z"}
{"user_id":2,"page_id":50,"action":"purchase","action_sha256":"7048dc8aa7663e713097535171b4ed72ab015705a885de9636e66d4b1fa18804","timestamp":"2023-03-28T09:09:10.64447719Z"}
{"user_id":2,"page_id":10,"action":"view","action_sha256":"2bcb43cbc8f6b7ef66331532881143fcbae60a879db3a8fb853f645bb24c2b3c","timestamp":"2023-03-28T09:09:17.644810963Z"}
{"user_id":1,"page_id":50,"action":"view","action_sha256":"2bcb43cbc8f6b7ef66331532881143fcbae60a879db3a8fb853f645bb24c2b3c","timestamp":"2023-03-28T09:09:20.644994805Z"}

This rule hashed the value of action from each log entry and stored the hashed value in a new key named action_sha256.

Flatten subrecordJoin records