Repository navigation
feat(vault): hold and erase personal data records - #481
Open
LKSNDRTMLKV wants to merge 1 commit into
Open
LKSNDRTMLKV wants to merge 1 commit into
LKSNDRTMLKV wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #446. Stacked on #479, so the base is
chore/repin-core-0-22-0and the diff is only this slice. Merge after #479; retarget tomainonce it lands.Finishes what #479 left of #446: the erasable store for the records a
held: outsidestatement names, the route that erases one, and the dry run's personal-data findings.What it does
PersonalDataPorton PostgreSQL, in its own tableodal.personal_data_record(migration 0043). It is not a column ofodal.passport; the archive of past versions, the audit chain, the snapshots, the outboxes and the back-up copy never read or write it, and no route reads a record.DELETE /vault/api/v1/personal-data/{record}erases one and answers the receipt (record, passport, field,erasedAt). A retry answers the first receipt again. A record this node never held is404. It touches neither the passport nor the statement in it.POST /dpp/validatenow runs the same personal-data check create runs, so a preview cannot pass what create refuses. It reads no store.What a tombstone retains, and why it needs a retention basis
Erasure drops the content and keeps the row: the record identifier, the passport and field it belonged to, and the instant it was erased. None of the content. But a passport identifier can relate to whoever owns the item, and information that can be linked to a person is personal data (GDPR Recital 26), so the tombstone is not free of it. It is protected like the passport's individual-item data, and the operator needs a basis to keep it. Showing that it erased (Art. 5(2)) is one. Nothing in this node deletes a tombstone, and the app role has no
DELETEon the table, so the DELETE set inops/pg/README.mdis unchanged.Erasure overwrites the content in place. PostgreSQL's own write-ahead log, replicas and backups hold earlier row versions, and no application role can reach them. How long those live is the operator's retention, and the CHANGELOG says so.
Design choices worth a look
UPDATE, not aDELETE, and the database enforces that it is one-way. Column-levelUPDATE (content, erased_at)for the app role only, aCHECKthat a row is present or erased and never both, and a trigger that refuses any change to a tombstone (and any rewrite of a present record) even for a superuser. A retried erasure returns the first receipt because theUPDATE ... WHERE erased_at IS NULLmatches nothing the second time; two racing calls are handed one receipt (tested with eight).odal.passport. A record cannot be held for a passport this node has no row for;storeanswersNotFoundfor one. Passports are never deleted, so it cannot block anything. Core's in-memory holder accepts any passport id, so this is stricter than the stub.404, including a string that is not a UUID. The route does not tell "malformed" from "unknown".AppState.personal_datais required, not optional. A node that cannot erase a record it was given cannot honour a withdrawn consent, and a501that only surfaces when someone needs it is the wrong failure.assemble_passportis split out ofcreate_handlerand both call it. Consequence worth knowing: the preview now also refuses a malformedcommodityCode, which create refused and the preview passed.Not done, on purpose
storeorrecords_foryet; the issue asked for the store and the erasure path. How content enters (this API, or the operator's own system writing through the port) is a decision for the founder, and the listing route belongs with it: without one an operator has no way to learn the id of a record whose statement was never written.Passport::validate; it never did.Tests
Real PostgreSQL (
dpp-dal/tests/pg_personal_data.rs): round trip, JSONnullcontent stays present, a tombstone with no content anywhere in the row, a retried erasure returns the first receipt, concurrent erasures handed one receipt, a record whose identifier was lost is found byrecords_forand erased, listing order, unknown and non-UUID ids, the foreign key, the app role cannotDELETEor repoint a record, a superuser cannot resurrect or rewrite one, theCHECK. Parity withodal_appcan read every table.Over HTTP (
dpp-vault/tests/personal_data_route.rs): the receipt and the retry, the passport and statement byte-identical after an erasure, write- and read-scoped and unauthenticated callers refused and the record still present afterwards,404against already-erased200, and the dry run against create for each finding (no statement, a statement about an unmarked field, a basis the act does not admit, an outside statement naming no record) plus the tariff code. I disabled the dry run's check and watched the parity test go red.Through the assembled node (
dpp-node/tests/smoke.rs): a published passport with a record, read through the public, authenticated and credentialed views, history, versions, evidence and the list, none carrying the content; then every table in theodalandidentityschemas searched as whole rows, and the content is inodal.personal_data_recordand nowhere else; erasing changes nothing the passport serves. The real S3 back-up adapter was not driven with a record present; the back-up port takes aPassport, which carries only the statement, and the table search covers everything the database holds.just checkwith every step exceptaudit,just lint-integrationandjust openapi-checkpass. The dal tier (106) and vault tier (489) pass in full. The node tier passes 337 of 349; the 12 that fail ares3_backup,snapshot_static_tierandnats_event_bus, none of which this touches, and every one failed pullingrustfs/rustfs:1.0.0ornats:2from Docker Hub (HTTP 429), so those three suites are unverified here.auditis red onmainfor wasmtime and a yankedzip, unrelated.