Databricks: Mask Cardholder PANs in Responses

What this policy does

Direction: egress (tool_post_invoke) Default: allow (transform-only — never denies) Package: databricks.egress.mask_pan

What it does

Masks payment-card numbers (PANs) in Databricks tool responses before the agent receives them. Lakehouse tables routinely hold cardholder data, and a PAN surfaces in the response payload of the async SQL/Genie poll tools and the free-text AI Search index tools — not in the submit call. This egress policy Luhn-validates every 13-to-19-digit card-shaped candidate in the response and rewrites each confirmed PAN to BIN-plus-last4: the first six digits (the issuer BIN) and the last four are kept, and every digit between is replaced with *, e.g. 4111 1111 1111 1111 → 411111******1111. BIN+last4 is the maximum display format PCI DSS permits for personnel without a business need to see the full PAN.

Luhn validation keeps the false-positive rate far below a bare digit-length regex: ordinary long numbers (order IDs, row counts, epoch timestamps, join keys) fail the checksum and are left intact, so only numbers that actually satisfy the card-number check digit are masked.

The policy never blocks a call. When at least one PAN is found the response content blocks are rewritten via transformed_payload; when nothing matches, the transform rule is undefined and the response passes through byte-identical.

Callers whose input.subject.claims.groups contains the documented placeholder group pci-full-pan receive unmasked responses. The exemption is fail-closed: a caller with no subject, no claims, no groups claim, or a malformed groups claim is never exempt and always gets masked output.

Compliance alignment

• PCI DSS 3.4.1 — supports masking of PAN when displayed: the agent channel shows at most BIN+last4, with full-PAN visibility limited to a defined role (pci-full-pan).
• PCI DSS 3.4.2 — supports preventing PAN copy/relocation via remote-access technologies: an agent that only ever receives the masked form of a query result cannot re-post the full PAN into other tools, tickets, notebooks, or files.
• PCI DSS 12.10.7 — supports PAN-where-not-expected incident procedures: a PAN returned from an unexpected lakehouse column is a classic trigger, and the gateway's decision/transform audit events for this policy give the incident process a concrete signal to work from.
• CCPA/CPRA §1798.150 — supports reducing nonredacted-PI breach exposure: card numbers read out of the lakehouse are masked before they reach the agent by default.
• SOC 2 CC6.7 — supports restricting the transmission and movement of information: cardholder PANs read out of the lakehouse are masked to BIN+last4 on the agent channel before they can be moved into other tools, tickets, notebooks, or files.

This family also aligns with ISO/IEC 27001 A.8.11 (data masking) on the MCP read path, and complements — rather than duplicates — SSN/email/phone redaction (see Composition).

Tool name matching

The policy scopes to the Databricks surfaces that return row/document data in their response, matched case-insensitively on the tool name after normalizing _ to - so both underscore (as the servers publish them) and hyphenated (as some gateways deliver them) forms match. The tool name is resolved from input.resource.name (PARC), then input.tool_metadata.name, then input.payload.name — all three carry the same value on tool_post_invoke, and taking whichever is populated keeps the scope check from failing open on a gateway that omits resource.name on egress (the pre-PARC path). The surfaces:

• poll_sql_result — the managed Databricks SQL server's async result tool. The execute_sql / execute_sql_read_only submit calls return only a statement handle; the row data egresses here, so this is the tool to mask (verified name — Databricks docs + community article).
• genie_poll_response — the Genie One (Beta) async answer tool. Genie answers are grounded in Unity Catalog data, so this is the natural-language exfiltration path for the same tables (verified name — Databricks docs).
• execute_sql_query — the community RafaelCartenet/mcp-databricks-server synchronous SQL passthrough, which returns rows directly in its own response (verified name — that server's source).
• AI Search index tools — the managed AI Search server exposes one tool per index, named dynamically {CATALOG}__{SCHEMA}__{INDEX_NAME} with a double underscore between segments. These indexes frequently hold support tickets and documents with free-text card numbers. Because the exact names are per-deployment (the double-underscore scheme is documented but the concrete names are not), the policy matches any tool whose name contains __ as an AI Search index tool.

The DTwo gateway prefixes tool names with the configured MCP server name (e.g. databricks-sql-poll_sql_result), and that prefix is not standardized — suffix matching on the fixed verbs and the __ signature for AI Search keep the policy portable. Verify the exact names your gateway sends with the dump-input debug technique before relying on this in production.

Deliberately out of scope: execute_sql / execute_sql_read_only (managed) return a handle, not data; genie_ask and the SQL submit tools are ingress-side; metadata tools (describe_uc_table, list_uc_catalogs, list_clusters, …) do not return card data.

Patterns matched

Conservative, anchored PAN shapes only — each is commented in the Rego, and every candidate must additionally pass the Luhn check before it is masked:

• 16-digit PANs grouped 4-4-4-4 with space or dash separators (Visa/Mastercard/Discover print format).
• 15-digit American Express PANs grouped 4-6-5, constrained to the 34/37 IIN range.
• Unseparated 13-19-digit runs (the ISO/IEC 7812 PAN length range) — the dominant shape for a PAN stored in a lakehouse column and serialized into a SQL result. Runs of 20+ digits never match: there is no word boundary inside a digit run, so a longer identifier is never partially masked.

Response shape

Egress tool output arrives as content blocks in input.payload.text (an array; entries are typically strings of plain text, markdown, or serialized JSON — a poll_sql_result result set is a JSON string block). The policy scans each string block, replaces every Luhn-valid match with its own BIN+last4 form, and emits transform.transformed_payload with the original payload's text replaced by the masked blocks. Non-string blocks pass through unmodified. Because matching is string-level, PANs are masked wherever they appear — result rows, Genie answer prose, AI Search snippets — without parsing each tool's specific JSON shape.

Examples

Transformed (masked)

{
  "input": {
    "action": "tool_post_invoke",
    "mode": "output",
    "resource": { "name": "databricks-sql-poll_sql_result", "type": "tool" },
    "payload": {
      "name": "databricks-sql-poll_sql_result",
      "text": ["{\"rows\":[[\"acct-889\",\"4111 1111 1111 1111\"]]}"]
    },
    "subject": { "sub": "google-apps|casey@acme.com", "claims": { "groups": ["support"] } }
  }
}

allow = true; the agent sees the row with 411111******1111.

Allowed unmasked (exempt group)

{
  "input": {
    "action": "tool_post_invoke",
    "mode": "output",
    "resource": { "name": "databricks-sql-poll_sql_result", "type": "tool" },
    "payload": {
      "name": "databricks-sql-poll_sql_result",
      "text": ["{\"rows\":[[\"acct-889\",\"4111111111111111\"]]}"]
    },
    "subject": { "sub": "google-apps|pci-analyst@acme.com", "claims": { "groups": ["pci-full-pan"] } }
  }
}

allow = true, no transform — the caller is in the pci-full-pan group.

Passthrough (no PAN)

A Luhn-invalid digit run (a row count, an epoch timestamp, an order number) produces no transform; the response is returned byte-identical.

Composition

One policy, one job. This policy masks cardholder PANs only; it is designed to run alongside — not merge with — the other Databricks egress family:

• redact-pii-egress (egress) handles SSN, email, and phone. Keeping PAN separate lets the two families use different exemption groups (PCI full-PAN role vs. a privacy role) and independent tuning. Attach both to the same egress direction for round-trip coverage.
• guard-warehouse-sql (ingress) blocks DML/DDL and forces read-only SQL; this policy masks card data in the results of the reads you do allow.
• A default-deny-unknown-tools (PF-28) ingress policy is recommended alongside AI Search / UC-function deployments, since those tool names are dynamic.

Known limitations

• Luhn-valid non-card numbers are masked too. The Luhn check eliminates most row counts, timestamps, and IDs, but some non-card identifiers (certain IMEIs and other checksummed numbers) are Luhn-valid and will be masked. The masked form keeps first-six/last-four, so such false positives usually stay recognizable.
• Obfuscated PANs are missed. Card numbers with separators other than space/dash (dots, unicode spaces), split across lines or content blocks, spelled out in words, or base64-encoded do not match. Card numbers typed with non-ASCII digits (e.g. Unicode fullwidth) also do not match: the RE2 \d class is ASCII-only. Grouped formats other than 4-4-4-4 and Amex 4-6-5 match only in their unseparated form.
• A PAN glued directly to a word character is missed. Every pattern is \b-anchored, and the underscore counts as a word character in RE2, so a digit run immediately preceded or followed by a letter, digit, or underscore with no separator (e.g. acct_4111111111111111) has no word boundary and is not masked. This is the deliberate cost of the same \b anchoring that stops a 20+-digit identifier from being partially masked. The same applies to a grouped PAN with a stray digit glued to its first or last group (e.g. 4000 0000 0000 00021 or 94000 0000 0000 0002): the grouped pattern's leading/trailing \b fails, and the single-space separators break the run below the 13-contiguous-digit floor of the unseparated pattern, so the whole PAN-shaped string passes through unmasked.
• The __ AI-Search signature also matches non-index tools whose gateway server name contains __. The AI Search branch puts any tool whose resolved name contains a double underscore in scope. If you name an MCP server with a __ in it (e.g. my__srv), even its metadata/SQL tools (list_clusters, execute_sql, …) are pulled into scope and their responses are scanned and masked. This is over-masking, not a leak — the policy only ever masks Luhn-valid card shapes and never denies — but it can surprise. Avoid __ in gateway server names, or pin the concrete AI Search tool names and drop the __ heuristic if the collateral scanning is unwanted.
• Adjacent digit groups can shadow a grouped PAN. In pathological sequences like 1234 5678 4111 1111 1111 1111, the leftmost 4-4-4-4 window is consumed first (and fails Luhn), so the real PAN inside it is not matched. Unseparated PANs are unaffected.
• Substring collisions between two detected PANs. Replacements are applied per distinct matched string in unspecified order; if one detected PAN is a literal substring of another in the same block (both Luhn-valid), more than BIN+last4 of the longer one can remain visible. Middle digits of every match still get masked.
• Structured (non-string) content blocks and non-array text are not masked — fail-open. The policy scans and rewrites only string entries of input.payload.text, and only when text is a JSON array. A PAN carried inside a content block delivered as a JSON object, or a payload.text delivered as a bare string, passes through unmasked. In the DTwo egress shape observed to date tool output arrives as an array of string blocks, and serialized JSON inside a string block is scanned; only native object shapes and non-array text evade it. Confirm your gateway/server delivers string blocks with the dump-input technique before relying on this.
• The synchronous community execute_sql is not covered. This policy matches execute_sql_query (RafaelCartenet) but not the JustTryAI server's synchronous execute_sql, to avoid colliding with the managed execute_sql submit tool (which returns only a handle). If you run the JustTryAI server, add its execute_sql suffix to content_tool_suffixes.
• The Genie Space (GA) single-invoke tool is not covered. Only Genie One (Beta) is masked, via its async genie_poll_response egress tool. The GA-track Genie Space server (/api/2.0/mcp/genie/{space_id}) exposes a single synchronous invoke tool that returns the Unity-Catalog-grounded answer inline in its own response — but its name is not published, does not end in poll-sql-result / genie-poll-response / execute-sql-query, and carries no __ signature, so it is out of scope and its responses pass through unmasked. Because it is GA (Genie One is still Beta), it is the more likely production surface. Do not assume "Genie is covered": once you learn the concrete tool name for your space with the dump-input technique, add its suffix to content_tool_suffixes, and run the default-deny-unknown-tools (PF-28) companion (see Composition) so a new or unpinned Genie/AI-Search tool is denied rather than silently leaking.
• Egress masking only. The full card number still exists in the lakehouse and in the Databricks UI; this policy controls what the agent sees on the MCP path.
• Tool names are partly unverified. poll_sql_result, genie_poll_response, and execute_sql_query are verified from the landscape research; the AI Search per-index tool name is not published (the double-underscore scheme is documented but the concrete name is per-deployment), so the __ match is a heuristic — verify with dump-input.
• Group names are placeholders — replace pci-full-pan with your IdP's group name at import time. The exemption reads input.subject.claims.groups and requires it to be an array of strings; every other shape (string, object, number, null, or missing) fails closed to masked output. Confirm your IdP emits a groups claim as a string array for your tenant before relying on the exemption.

Compliance note. This policy supports alignment with the cited framework controls on the MCP path only. No policy or bundle makes an organization compliant with any framework; web-UI, native-API, and in-app access are outside the gateway's reach by design. Validate against your own compliance program before relying on it.

Policy source (Rego)

package databricks.egress.mask_pan

# Transform-only policy — never denies, only masks Luhn-valid card numbers in
# Databricks result/answer/search responses to BIN+last4.
default allow := true

# -----------------------------------------------------------------------------
# Tool matching — Databricks surfaces that return row/document data in their
# response. The gateway prefixes tool names with the configured server name, so
# we match on the suffix to stay portable. Suffixes are hyphenated; the incoming
# name is normalized `_` -> `-` first so both `poll_sql_result` and
# `poll-sql-result` deliveries match.
# -----------------------------------------------------------------------------

content_tool_suffixes := [
    # Managed Databricks SQL server: async result of execute_sql* (data egresses
    # here, not in the submit call).
    "poll-sql-result",
    # Genie One (Beta): async NL->SQL answer payload.
    "genie-poll-response",
    # Community RafaelCartenet/mcp-databricks-server: synchronous SQL passthrough
    # that returns rows directly in its own response.
    "execute-sql-query",
]

# The egress tool name can arrive under resource.name (PARC), tool_metadata.name
# (the legacy egress-only source), or payload.name — all three carry the same
# value. Collect every populated form (lowercased) so a gateway that does not
# populate resource.name on egress (the pre-PARC path) still scopes correctly
# instead of failing open and masking nothing.
tool_name_candidates := [lower(raw) |
    some raw in [
        object.get(object.get(input, "resource", {}), "name", ""),
        object.get(object.get(input, "tool_metadata", {}), "name", ""),
        object.get(object.get(input, "payload", {}), "name", ""),
    ]
    raw != ""
]

# Suffixes are hyphenated; each candidate is normalized `_` -> `-` before the
# suffix compare so both `poll_sql_result` and `poll-sql-result` deliveries match.
is_in_scope_tool if {
    some cand in tool_name_candidates
    some suffix in content_tool_suffixes
    endswith(replace(cand, "_", "-"), suffix)
}

# AI Search index tools are named dynamically `{CATALOG}__{SCHEMA}__{INDEX_NAME}`
# with a double underscore between segments. The concrete names are
# per-deployment, so we match any tool whose name carries that `__` signature.
is_in_scope_tool if {
    some cand in tool_name_candidates
    contains(cand, "__")
}

# -----------------------------------------------------------------------------
# PAN candidate shapes — anchored with \b word boundaries so digit runs inside
# longer identifiers are never partially matched. Every candidate must also pass
# the Luhn check below before it is masked.
# -----------------------------------------------------------------------------

pan_pattern := concat("|", [
    # 16-digit PANs grouped 4-4-4-4 with space or dash separators
    # (Visa / Mastercard / Discover print format, e.g. 4111 1111 1111 1111).
    `\b\d{4}[ -]\d{4}[ -]\d{4}[ -]\d{4}\b`,
    # 15-digit American Express PANs grouped 4-6-5 with space or dash separators,
    # constrained to the 34/37 IIN range (e.g. 3782 822463 10005).
    `\b3[47]\d{2}[ -]\d{6}[ -]\d{5}\b`,
    # Unseparated 13-19 digit runs — the ISO/IEC 7812 PAN length range. Runs of
    # 20+ digits never match: there is no word boundary inside a digit run, so
    # this cannot partially mask a longer identifier.
    `\b\d{13,19}\b`,
])

# -----------------------------------------------------------------------------
# Luhn check — filters card-shaped candidates so timestamps, order numbers, row
# counts, and other digit runs that merely look like PANs are left alone.
# -----------------------------------------------------------------------------

digits_only(s) := regex.replace(s, `[^0-9]`, "")

luhn_contribution(d, parity) := d if { parity == 0 }

luhn_contribution(d, parity) := 2 * d if {
    parity == 1
    (2 * d) < 10
}

luhn_contribution(d, parity) := (2 * d) - 9 if {
    parity == 1
    (2 * d) >= 10
}

luhn_valid(digits) if {
    chars := split(digits, "")
    n := count(chars)
    total := sum([v |
        some i, c in chars
        v := luhn_contribution(to_number(c), (n - 1 - i) % 2)
    ])
    total % 10 == 0
}

# All card-shaped substrings of t that pass the Luhn check.
pan_candidates(t) := {c |
    some c in regex.find_n(pan_pattern, t, -1)
    luhn_valid(digits_only(c))
}

# -----------------------------------------------------------------------------
# Masking — each match is rewritten to BIN+last4: first six digits (issuer BIN)
# and last four kept, everything between masked with `*`. Separators are dropped
# in the masked form (e.g. `4111 1111 1111 1111` -> `411111******1111`).
# -----------------------------------------------------------------------------

mask_pan(c) := masked if {
    d := digits_only(c)
    n := count(d)
    masked := concat("", [
        substring(d, 0, 6),
        # Replace every middle digit with `*` (RE2 has no repeat builtin, so we
        # mask the middle substring char-by-char instead of building a `*` run).
        regex.replace(substring(d, 6, n - 10), `\d`, "*"),
        substring(d, n - 4, 4),
    ])
}

# Rewrite every Luhn-valid candidate in a string block to its masked form.
mask_block(b) := out if {
    is_string(b)
    replacements := {c: mask_pan(c) | some c in pan_candidates(b)}
    count(replacements) > 0
    out := strings.replace_n(replacements, b)
}

mask_block(b) := b if {
    is_string(b)
    count(pan_candidates(b)) == 0
}

# Non-string content blocks (structured/JSON object blocks) pass through
# unmodified.
mask_block(b) := b if { not is_string(b) }

# -----------------------------------------------------------------------------
# Full-PAN exemption — callers in the placeholder group see unmasked content.
# Fail-closed: missing subject, missing claims, missing groups, or a malformed
# groups claim all leave this rule undefined, so masking applies. The is_array
# guard is load-bearing: without it a groups claim shaped as an object (e.g.
# {"role":"pci-full-pan"}) would iterate its *values* and match, granting the
# exemption to a caller who never held the group in an array. Requiring an array
# keeps every non-array shape (string, object, number, null) fail-closed.
# Replace "pci-full-pan" with your IdP's group name at import time.
# -----------------------------------------------------------------------------

caller_may_view_full_pan if {
    claims := object.get(object.get(input, "subject", {}), "claims", {})
    groups := object.get(claims, "groups", [])
    is_array(groups)
    some group in groups
    group == "pci-full-pan"
}

# -----------------------------------------------------------------------------
# Transform — emitted only when in scope, the caller is not exempt, and at least
# one block actually changed. Otherwise the rule is undefined and the aggregator
# skips this policy, returning the response byte-identical.
# -----------------------------------------------------------------------------

text_blocks := object.get(input.payload, "text", [])

masked_blocks := [out |
    some block in text_blocks
    out := mask_block(block)
]

transform := {
    "transformed_payload": object.union(input.payload, {"text": masked_blocks}),
} if {
    input.mode == "output"
    is_in_scope_tool
    not caller_may_view_full_pan
    is_array(text_blocks)
    masked_blocks != text_blocks
}

Canonical source: policy.md on GitHub · raw · raw on this site (.md)

Related policies