Sensitive Data Redaction

resolve can redact data to ensure no leaked sensitive data in your logs or traces even reaches resolve ai's storage layer the redaction approach you choose depends on the selected integration, whether using an on prem setup or a cloud based observability tool sensitive data includes personally identifiable information (pii), customer identifiable information (cii), cardholder data (chd) protecting this data ensures compliance with industry standards, including pci, dss, and gdpr there are 3 options for sensitive data redaction cloud advanced settings in the resolve page for a cloud based integration satellite data redaction capabilities with standard regexes sidecar deploy alongside the satellite to customize redaction logic data subject to redaction the following categories of sensitive information are subject to redaction logs entire raw log messages spans / traces attributes and tags metrics all metric names and label values redacted data patterns redaction is based on a list of regex expressions on common sensitive data see the full list below email phone number dob credit card number bank account card expiration date ssn ein ip address us street address option 1 redaction for cloud integrations sensitive data redaction can be enabled on a per integration basis open the desired integration, such as datadog enable the redaction config within the integration settings by default, resolve ai redacts the all json paths listed in the "supported data types and observability platforms" section above you can also specify additional custom target json paths to redact as needed e g; $ data\[ ] attributes spans\[ ] mytags option 2 on prem redaction from satellite sidecar sensitive data redaction can be enabled on a per integration basis from the resolve satellite https //docs resolve ai/resolve satellite prerequisites the resolve satellite should already be installed in your environment if you need assistance with installation, refer to the instructions provided in resolve satellite https //docs resolve ai/resolve satellite section if you need assistance with the installation, refer to the instructions provided in the respective observability tool integration documentation the specific integration you want to redact data for should be set up through the resolve satellite update integration configuration update your helm values override file (ex resolve values yaml ) with the redactionconfig configuration for a specific integration (ex grafanaonprem ) resolve values yaml integrations grafanaonprem type grafana create true secretname \<name from your secret manager> connection region \<enum> accesskeyid \<string> secretaccesskey \<string> redactionconfig # enable redaction config enabled true install the satellite and apply the updates to the values file (ex resolve values yaml ) visit dockerhub for the latest helm chart https //hub docker com/r/resolveaihq/satellite chart/tags and satellite https //hub docker com/r/resolveaihq/satellite/tags apply config to satellite and redeploy helm upgrade install oci //registry 1 docker io/resolveaihq/satellite chart version \<latestchart> values resolve values yaml set image tag=\<latestimage> once your satellite is deployed, resolve automatically redacts sensitive data specific to its configured integration verify redaction is enabled login to https //app0 resolve ai/ https //app0 resolve ai/ go to the integrations page and and select the integration for which you enabled sensitive data redaction you should see an automatically created integration based on the provided configuration, including whether redaction is enabled enabled redaction config security sidecar the security sidecar is a lightweight http proxy container deployed alongside the resolve satellite https //docs resolve ai/resolve satellite in the same pod it provides two capabilities beyond what the resolve ai web app offers by default secret resolution — resolves kubernetes secrets into http headers before requests leave the pod, so the satellite process never sees raw api keys response redaction — strips sensitive data (credit cards, emails, ssns, etc ) from api responses before the satellite processes them auditable redaction logic — source code is available for review and customization (contact help\@resolve ai mailto\ help\@resolve ai to request access) the sidecar image is publicly available on docker hub https //hub docker com/r/clouddatalabs/security sidecar how it works secret resolution integration credentials can be stored as placeholders (e g , $ load datadog api key ) in the satellite's integration config instead of plain text values the satellite sends outbound http requests through the sidecar proxy running in the same pod the sidecar detects headers containing the $ load prefix for each match, the sidecar resolves the actual secret value from a kubernetes secret (injected as an environment variable or mounted as a file) the sidecar replaces the placeholder with the resolved value and forwards the request to the target service response redaction on the return path, the sidecar applies regex based redaction to api response bodies before they reach the satellite this uses the same patterns listed in the redacted data patterns docid\ s6 zyfp8qrvtahpix7v2m section above redaction is enabled by default and controlled by the enable redaction environment variable prerequisites the resolve satellite https //docs resolve ai/resolve satellite is already installed in your environment at least one on prem integration is configured through the satellite install the security sidecar 1\ update values file add the sidecarproxy block to your resolve values yaml resolve values yaml ingest token \<your resolve satellite token> clustername \<your cluster name> environment \<your environment> \# enable the security sidecar httpapiclientproxyurl "http //localhost 8080" sidecarproxy enableredaction "true" container image repository clouddatalabs/security sidecar tag v13 pullpolicy always resources limits cpu 1 memory 8gi requests cpu 1 memory 8gi 2\ configure secrets use secretkeyrefs to make kubernetes secret values available to the sidecar each entry maps a key from a kubernetes secret to an environment variable name that corresponds to a $ load \<name> placeholder in your integration config for example, to resolve datadog api and app keys resolve values yaml sidecarproxy secretsconfig secretkeyrefs \ envname datadog api key secretname datadog resolve token secretkey apikey \ envname datadog app key secretname datadog resolve token secretkey appkey with this config, a header containing $ load datadog api key in the satellite's integration config will be resolved to the value of the apikey key in the datadog resolve token kubernetes secret the satellite itself never sees the raw credential advanced mount secrets as files as an alternative to environment variable injection, secrets can be mounted as files this may be preferable when using external secret store csi drivers, or to prevent secrets from appearing in the container's environment (e g , kubectl describe pod output) the secret keys must match the $ load placeholders exactly first, create the kubernetes secret the key names must match the $ load placeholders you use in your integration config datadog resolve token yaml apiversion v1 kind secret type opaque metadata name datadog resolve token stringdata datadog api key "\<your datadog api key>" datadog app key "\<your datadog app key>" then mount it in your resolve values yaml resolve values yaml sidecarproxy secretsconfig mountpath "/etc/secrets" secretmounts \ name datadog creds mountpath /etc/secrets \# you must also define the corresponding volumes \# the volume name must match the pattern sidecar \<secretmounts\[] name> volumes \ name sidecar datadog creds secret secretname datadog resolve token the integration config uses the same $ load datadog api key and $ load datadog app key placeholders as the environment variable approach \ 3\ apply changes update the satellite with the new resolve values yaml file helm upgrade install resolve satellite \\ oci //registry 1 docker io/resolveaihq/satellite chart \\ \ values resolve values yaml 4\ verify the sidecar is running confirm that the pod has two containers ( satellite and sidecar proxy ) kubectl get pods l app kubernetes io/name=satellite o jsonpath='{ items\[ ] spec containers\[ ] name}' you should see output containing both container names you can also check the sidecar logs kubectl logs \<satellite pod name> c sidecar proxy full example below is a complete resolve values yaml that enables the sidecar with secret resolution for a datadog on prem integration resolve values yaml ingest token \<your resolve satellite token> clustername prod main environment production httpapiclientproxyurl "http //localhost 8080" sidecarproxy enableredaction "true" container image repository clouddatalabs/security sidecar tag v13 pullpolicy always resources limits cpu 1 memory 8gi requests cpu 1 memory 8gi secretsconfig secretkeyrefs \ envname datadog api key secretname datadog resolve token secretkey apikey \ envname datadog app key secretname datadog resolve token secretkey appkey integrations datadogonprem type datadog create true secretname datadog resolve token connection site datadoghq com servicemapenabled true redactionconfig enabled true