PII Redaction
Design pipelines that detect, minimize, redact, and govern personally identifiable information.
Start with minimization
The safest personal data is data the workflow never receives. Define the task’s required fields, remove unrelated content, and collect only what is necessary for the stated purpose. Map where data enters, is transformed, logged, cached, sent to providers, and deleted. Minimization reduces exposure and makes later redaction more reliable, but it does not eliminate the need for access controls.
Types of personal information
PII can include names, emails, phone numbers, addresses, identity numbers, account IDs, faces, voices, locations, and combinations that identify someone indirectly. Sensitive data may require stronger controls depending on context and jurisdiction. Create a project-specific inventory rather than relying on a generic list. Include data in prompts, attachments, tool arguments, errors, traces, analytics, and backups.
import re
text = "Contact anita@example.com or call +91 98765 43210."
text = re.sub(r"[\w.+-]+@[\w.-]+\.\w+", "[EMAIL]", text)
text = re.sub(r"\+?\d[\d ()-]{8,}", "[PHONE]", text)
print(text)Detection methods
Use deterministic patterns for well-formed numbers and addresses, named-entity detectors for contextual text, and specialized tools for documents, faces, or audio. No detector is perfect: formats vary, OCR introduces errors, and names overlap with ordinary words. Combine methods, preserve confidence and location, and send uncertain cases to review instead of treating one regex as complete protection.
Redaction and pseudonyms
Replace values with stable placeholders when the model needs to understand relationships, such as matching the same customer across messages. Keep the mapping in a separate protected store with a short retention period. Irreversible redaction is safer when the value is not needed later. Ensure placeholders cannot accidentally resemble real credentials or become instructions in a prompt.
Logging boundaries
A redacted user interface does not help if the original prompt is copied into logs, traces, crash reports, or analytics. Apply redaction before observability code receives the data, and review third-party SDK behavior. Mask secrets and identifiers in exceptions and metrics labels. Restrict investigation access and document when an authorized operator may view original content.
Verification and deletion
Test detection and redaction on realistic formats, languages, OCR output, nested JSON, screenshots, audio transcripts, and adversarial spacing. Verify that original values cannot be recovered from caches, backups, embeddings, indexes, or generated exports beyond the approved retention period. Deletion should cover every copy and provide evidence of completion where governance requires it.
Human review and governance
Some workflows need a reviewer to resolve ambiguous entities or confirm that a document is safe to process. Give reviewers the minimum context and permissions needed, and record decisions without creating another uncontrolled copy. Define ownership, retention, provider terms, incident response, and user rights before launch. Redaction is a process spanning product, engineering, security, and operations.