Your RAG System Can't Forget: The Right to Erasure Problem Nobody Engineered For

The short answer

Where a single sentence about a person ends up

Follow one document through a standard ingestion pipeline. A client emails a PDF — a settlement agreement mentioning two employees and one customer by name. The pipeline does its job well, and by the next morning that one document exists in six places.

1. 01

   The source store. The original file in object storage, plus the parser's intermediate output. This is the copy your current deletion process knows about.

2. 02

   The chunk store. The document split into passages — typically dozens per file — sized for retrieval, with boundaries drawn by token count and headings. A person's name now sits inside chunks that also describe other people.

3. 03

   The vector index. Each chunk embedded into a high-dimensional vector, stored with its metadata and, in most setups, the chunk text itself. Embeddings can be inverted back to readable text, which is why treating them as anonymized numbers fails on contact with an inversion attack.

4. 04

   The logs. Retrieval logs record which chunks matched which queries. Prompt logs carry the retrieved passages verbatim into every LLM request. Completion logs restate the substance in the model's answer. Tracing platforms often duplicate all three.

5. 05

   The caches. Semantic caches store finished answers keyed by query similarity, so a cached response built from deleted data keeps serving until someone invalidates it.

6. 06

   Backups and replicas. Nightly snapshots of everything above, on their own retention schedule, restorable long after the live copies are gone.

Six is the floor. Evaluation datasets, fine-tuning exports, and analytics warehouses each add a copy when a pipeline feeds them.

None of these copies escapes the law by being derived. The Israeli Privacy Protection Authority’s draft AI guidance states that the Privacy Protection Law covers information AI systems infer or derive from personal data, and European analysis of the GDPR’s “any information” standard reaches the same place: an erasure request should be able to remove a vector along with its source chunk.

An embedding you can’t find and can’t delete remains personal data. It has simply become personal data you’ve lost control of.

Why deleting the row doesn’t delete the data

Chunks ignore people

Chunking splits documents by structure and token budget. People appear wherever the text puts them. One chunk of a settlement agreement can name three individuals, which means a person-level erasure request maps onto chunk boundaries badly in both directions: deleting whole documents removes more than the requester’s data, while deleting nothing below the document level removes less. Without a mapping from person to documents to chunks, every deletion request starts with a discovery project across your own index.

Vector indexes bury their dead

Logs and caches answer for the dead

Prompt logs are the store teams forget first, and the one that holds the data in its most readable form: full retrieved passages, inlined into every request, timestamped and searchable. Delete the source, the chunks, and the vectors, and a grep through last quarter’s LLM traffic still returns the requester’s settlement terms in plain text. Semantic caches behave the same way with a shorter fuse — a cached answer derived from deleted content keeps serving matches until an invalidation reaches it, and cache keys built on query similarity make targeted invalidation genuinely hard.

Backups are the honest delay

Backups get a defensible timeline where the other stores get none. The workable pattern: erase from live systems now, let snapshots expire on a documented retention schedule, and keep a re-deletion list that runs after any restore — because a restore without one silently resurrects every record you’ve certified as gone. Israeli guidance closes the loop explicitly: the PPA’s draft DPO guidelines require disposal to cover backup systems and decommissioned equipment, which pulls restored-snapshot hygiene inside the compliance perimeter.

What regulators have already said

The doctrine is younger than the architecture, but the pieces on the board point one way.

Fine-tuning: where deletion stops being an option

RAG earns its reputation as the compliance-friendly pattern only after the deletion pipeline below exists. Before that, it concentrates the same personal data in more places than the system it replaced.

Engineering a RAG that can forget: the deletion propagation pattern

This is the pattern we implement in our own RAG builds. It costs a few design decisions at ingestion time and turns an erasure request from a search project into a keyed operation.

Lineage before anything

Every artifact the pipeline creates carries the identifier of its source: source_id stamped on chunks, on vector metadata, on log lines, on cache entries. Beside it, an identity map connects people to the documents that mention them, maintained at ingestion while the knowledge is fresh. With those two structures, “delete everything about this person” compiles into a set of keyed lookups. Without them, the first real deletion request begins with an unbounded search through your own systems — the expensive end of the retrofit curve, where rebuilding embeddings sits.

One request, many acknowledgements

Deletion runs as an orchestrated job, and the job is only done when every store has answered. The sequence we use: pause ingestion for the affected sources, delete chunks by key, delete vectors by metadata filter, then verify the physical cleanup rather than trusting the API’s return code — trigger or await the tombstone purge and confirm it ran. Invalidate the caches. Redact or expire the log entries. Append the identifiers to the re-deletion list that runs after any backup restore. Notify downstream processors. Each stage acknowledges or retries; failures land in a dead-letter queue that pages someone, because a silent partial deletion is a certification you can’t stand behind. The whole run fits inside the statutory window with room to spare.

Crypto-shredding for the stores you can’t purge

Some stores resist physical deletion — append-only logs, snapshot chains, cold archives. For those, encrypt per source or per epoch and make key destruction the deletion primitive: destroy the key and every copy, including the ones in old snapshots and filesystem free pages, becomes unreadable at once. The ghost-vectors researchers reached the same conclusion from the attack side — their epoch key rotation scheme cut recovery of deleted records to 0% at a cost of 0.005 milliseconds per record, while generating cryptographically signed proof of erasure. Deletion this way produces evidence by construction.

Receipts, never residue

The erasure log records facts about the deletion and nothing from inside it: which identifiers, which stores, which timestamps, the job hash, the key-destruction proof. Signed, bundled, and ready for whoever asks — a regulator, an enterprise buyer’s security review, your own incident response. This is the same evidence bar we describe in Security & Controls: the claim “we deleted it” backed by an artifact rather than an assurance.

The vendor edge

A RAG pipeline exports data with every API call. The embedding provider and the model provider are recipients under Article 19, which makes their retention windows part of your erasure surface: know how long prompts persist, whether zero-retention modes exist for your tier, what the training-use default is, and how subprocessor deletion is handled. Put deletion SLAs in the contract and propagation of erasure requests in the runbook. A pipeline that can prove deletion internally while its API partner holds thirty days of prompt history has proven half of something.

A 30-minute self-test

Ten questions. Each maps to a store or a control from this article, and each has a yes or a no.

1. 01

   Can you name every store that holds a copy of a customer's data after ingestion?

2. 02

   Does a mapping exist from a person to the documents that mention them?

3. 03

   Does every chunk and vector carry a source_id you could delete by?

4. 04

   When vectors are deleted, does anyone verify the physical cleanup completed?

5. 05

   Do prompt and completion logs have a retention limit and a redaction path?

6. 06

   Does a deletion event invalidate the semantic cache?

7. 07

   Is backup retention documented, and does a re-deletion list run after restores?

8. 08

   Do you know your embedding and LLM providers' retention windows, and whether zero-retention is available?

9. 09

   Does a completed deletion produce a receipt you could show an auditor?

10. 10

    Is fine-tuning on personal data either excluded or logged record-by-record?

Seven or more honest yes answers puts you ahead of most production deployments we’ve reviewed. Three or fewer means the next erasure request sets your roadmap for you.

If you run a RAG in production

A RAG system that can prove what it forgot is rarer than one that can explain what it knows. The first is becoming the more valuable of the two.

Frequently asked questions