Redact Sensitive Data

Description

The Redact Sensitive Data processor detects and redacts sensitive data.

Supported Types

Metrics	Logs	Traces	Bindplane Collector
✓	✓	✓	v1.81.0+

Configuration Table

Parameter	Type	Default	Description
telemetry_types	strings	[Metrics, Logs, Traces]	Which types of telemetry to apply redaction rules to.
redaction_strategy*	enum	Censor with Asterisks	The approach used to redact sensitive data. Censor with Asterisks will replace all detected sensitive data with asterisks (e.g. ******). Hashing will replace all detected sensitive data with their hashed values based on the hash_function parameter.
hash_function	enum	sha3	The hash function used to redact sensitive data. Only relevant if redaction_strategy is set to Hashing. Must be one of sha3, sha1, or md5. sha3 is highly recommended when dealing with sensitive data as it is secure and future-proof. sha1 and md5 are faster hashing algorithms, but are insecure and susceptible to attacks.
redaction_rules	enums	See default rules	Commonly used redaction rule presets.
custom_rules	strings		A list of regular expressions representing custom redaction rules. Any data that matches any rule will be redacted according to the redaction_strategy parameter.
ignored_values	strings		A list of regex patterns. Data that matches will not be redacted regardless of its value.
redacted_keys	strings		A list of regex patterns corresponding to keys of resource attributes, attributes, or log body fields. Keys that match will have their values redacted regardless of their value.
ignored_keys	strings		A list of names (not regex patterns) corresponding to keys of resource attributes, attributes, or log body fields. Keys that match will not have their values redacted regardless of their value.
added_summary_attributes*	enum	None	The summary attributes added to the telemetry. Must be one of None, Redacted Field Counts, or Redacted Field Counts and Names. Redacted Field Counts will add attributes to the outgoing telemetry with the count of redacted telemetry. Redacted Field Counts and Names will also add attributes with the name of the attribute keys that were redacted. Note: Does not have an effect when configured within a Google SecOps Stream.

*required field

Default Rules Values

• Credit Card: \b(?:(?:(?:\d{4}[- ]?){3}\d{4}|\d{15,16}))\b

• Date of Birth: \b(?:(?:19|20)?\d{2}[-/])?(?:0?[1-9]|1[0-2])-/(?:-/?\d{2})?\b

• Email: \b[a-zA-Z0-9._/+-—|]+@[A-Za-z0-9-—|]+.[a-zA-Z|]{2,6}\b

• International Bank Account Number (IBAN): \b[A-Z]{2}\d{2}(?: ?[A-Z0-9]){11,31}(?:\s[A-Z0-9])*\b

• IPv4 Address: \b(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b

• IPv6 Address: \b(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}\b

• MAC Address: \b([0-9A-Fa-f]{2}[:-]){5}[0-9A-Fa-f]{2}\b

• US Phone Number: \b((+|\b)[1l][-. ])?(?\b[\dOlZSB]{3,5}([-. ]|) ?)[\dOlZSB]{3}[-. ][\dOlZSB]{4}\b

• International Phone Number: +[1-9]\d{0,2}(?:[-.\s]?(?\d+)?(?:[-.\s]?\d+)*)\b

• Social Security Number (SSN): \b\d{3}[- ]\d{2}[- ]\d{4}\b

• US City, State: \b[A-Z][A-Za-z\s\.]+,\s{0,1}[A-Z]{2}\b

• US Street Address: \b\d+\s[A-z]+\s[A-z]+(\s[A-z]+)?\s*\d*\b

• US Zip Code: \b\d{5}(?:[-\s]\d{4})?\b

• UUID/GUID: \b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-5][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}\b

Basic Example Configuration

Below is an example of a basic Redact Sensitive Data processor configuration using a subset of the preset redaction rules.

Standalone Processor

apiVersion: bindplane.observiq.com/v1
kind: Processor
spec:
    parameters:
        - name: telemetry_types
          value:
            - Logs
            - Metrics
            - Traces
        - name: redaction_strategy
          value: Censor with Asterisks
        - name: redaction_rules
          value:
            - Social Security Number (SSN)
            - Credit Card
            - IPv4 Address
            - US Street Address
            - US City, State
            - US Zip Code

Web Interface

Advanced Example Configuration

Below is an example of an advanced Redact Sensitive Data configuration using custom redaction rules, and allowing/ignoring specific values & keys.

Standalone Processor

apiVersion: bindplane.observiq.com/v1
kind: Processor
spec:
    parameters:
        - name: telemetry_types
          value:
            - Logs
            - Metrics
            - Traces
        - name: redaction_strategy
          value: Hashing
        - name: hash_function
          value: sha3
        - name: redaction_rules
          value: []
        - name: custom_rules
          value:
            - ^\d{9}$
            - ^[a-zA-Z]+@[a-zA-Z]+.com$
        - name: ignored_values
          value:
            - ^[a-zA-Z][email protected]$
        - name: redacted_keys
          value:
            - ^service.*$
        - name: ignored_keys
          value:
            - service.name
            - service.env
        - name: added_summary_attributes
          value: Redacted Field Counts and Names

Web Interface

Redaction Strategy

This configuration sets the redaction_strategy parameter to Hashing and the hash_function parameter to sha3. This replaces any matching sensitive data with a sha3 hash of its value as seen in the Web Interface screenshot above.

Custom Rules

This example does not select any of the preset redaction rules, and instead configures two custom rules to match passport IDs (nine digit numbers), and alphabetical emails that end in .com.

Ignored Values

This example configures the ignored_values parameter to process example emails (ending in "example.com") without redaction, even though they will match the custom email rule.

Redacted Keys

This example configures the redacted_keys parameter to always redact the value of any keys that begin with "service." Note that this applies to all levels of keys (resource attributes, attributes, and log body keys).

Ignored Keys

The above example configures two keys to be ignored by the redaction processor: "service.name" and "service.env". These two values would have normally been redacted due to the Redacted Keys configuration, but will now be ignored. Note that this applies to all levels of keys (resource attributes, attributes, and log body keys).

Added Summary Attributes

The above example sets the value of the added_summary_attributes parameter to Redacted Field Counts and Names. This adds various metrics in the form of attributes (at both the resource & span level). These metrics track the count of redacted attributes and the names of each attribute that is redacted, as seen in the Web Interface screenshot above.