Skip to content

feat(vault): hold and erase personal data records - #481

Open
LKSNDRTMLKV wants to merge 1 commit into
chore/repin-core-0-22-0from
feat/personal-data-store-446
Open

LKSNDRTMLKV wants to merge 1 commit into
chore/repin-core-0-22-0from
feat/personal-data-store-446

Conversation

@LKSNDRTMLKV

Copy link
Copy Markdown
Member

Refs #446. Stacked on #479, so the base is chore/repin-core-0-22-0 and the diff is only this slice. Merge after #479; retarget to main once it lands.

Finishes what #479 left of #446: the erasable store for the records a held: outside statement names, the route that erases one, and the dry run's personal-data findings.

What it does

  • PersonalDataPort on PostgreSQL, in its own table odal.personal_data_record (migration 0043). It is not a column of odal.passport; the archive of past versions, the audit chain, the snapshots, the outboxes and the back-up copy never read or write it, and no route reads a record.
  • DELETE /vault/api/v1/personal-data/{record} erases one and answers the receipt (record, passport, field, erasedAt). A retry answers the first receipt again. A record this node never held is 404. It touches neither the passport nor the statement in it.
  • POST /dpp/validate now runs the same personal-data check create runs, so a preview cannot pass what create refuses. It reads no store.
  • OpenAPI for the route and its responses, bundle regenerated, CHANGELOG entry (and the "not yet done" paragraph from chore(deps)!: repin dpp-core to 0.22.0 #479 removed).

What a tombstone retains, and why it needs a retention basis

Erasure drops the content and keeps the row: the record identifier, the passport and field it belonged to, and the instant it was erased. None of the content. But a passport identifier can relate to whoever owns the item, and information that can be linked to a person is personal data (GDPR Recital 26), so the tombstone is not free of it. It is protected like the passport's individual-item data, and the operator needs a basis to keep it. Showing that it erased (Art. 5(2)) is one. Nothing in this node deletes a tombstone, and the app role has no DELETE on the table, so the DELETE set in ops/pg/README.md is unchanged.

Erasure overwrites the content in place. PostgreSQL's own write-ahead log, replicas and backups hold earlier row versions, and no application role can reach them. How long those live is the operator's retention, and the CHANGELOG says so.

Design choices worth a look

  • Erasure is an UPDATE, not a DELETE, and the database enforces that it is one-way. Column-level UPDATE (content, erased_at) for the app role only, a CHECK that a row is present or erased and never both, and a trigger that refuses any change to a tombstone (and any rewrite of a present record) even for a superuser. A retried erasure returns the first receipt because the UPDATE ... WHERE erased_at IS NULL matches nothing the second time; two racing calls are handed one receipt (tested with eight).
  • Admin scope, not write. Every other route that removes something is admin (API keys, webhooks, facilities, operator identifiers), the write scope produces and moves passports and nothing it does destroys data, and this cannot be undone. Loosening the gate later breaks nobody and tightening it would. Say if an integration that handles withdrawals should hold a narrower key.
  • A foreign key to odal.passport. A record cannot be held for a passport this node has no row for; store answers NotFound for one. Passports are never deleted, so it cannot block anything. Core's in-memory holder accepts any passport id, so this is stricter than the stub.
  • An identifier this store did not mint is 404, including a string that is not a UUID. The route does not tell "malformed" from "unknown".
  • AppState.personal_data is required, not optional. A node that cannot erase a record it was given cannot honour a withdrawn consent, and a 501 that only surfaces when someone needs it is the wrong failure.
  • The dry run builds the draft create builds. assemble_passport is split out of create_handler and both call it. Consequence worth knowing: the preview now also refuses a malformed commodityCode, which create refused and the preview passed.

Not done, on purpose

  • No route stores a record, and none lists a passport's records. Nothing in this node calls store or records_for yet; the issue asked for the store and the erasure path. How content enters (this API, or the operator's own system writing through the port) is a decision for the founder, and the listing route belongs with it: without one an operator has no way to learn the id of a record whose statement was never written.
  • The dry run does not call Passport::validate; it never did.

Tests

Real PostgreSQL (dpp-dal/tests/pg_personal_data.rs): round trip, JSON null content stays present, a tombstone with no content anywhere in the row, a retried erasure returns the first receipt, concurrent erasures handed one receipt, a record whose identifier was lost is found by records_for and erased, listing order, unknown and non-UUID ids, the foreign key, the app role cannot DELETE or repoint a record, a superuser cannot resurrect or rewrite one, the CHECK. Parity with odal_app can read every table.

Over HTTP (dpp-vault/tests/personal_data_route.rs): the receipt and the retry, the passport and statement byte-identical after an erasure, write- and read-scoped and unauthenticated callers refused and the record still present afterwards, 404 against already-erased 200, and the dry run against create for each finding (no statement, a statement about an unmarked field, a basis the act does not admit, an outside statement naming no record) plus the tariff code. I disabled the dry run's check and watched the parity test go red.

Through the assembled node (dpp-node/tests/smoke.rs): a published passport with a record, read through the public, authenticated and credentialed views, history, versions, evidence and the list, none carrying the content; then every table in the odal and identity schemas searched as whole rows, and the content is in odal.personal_data_record and nowhere else; erasing changes nothing the passport serves. The real S3 back-up adapter was not driven with a record present; the back-up port takes a Passport, which carries only the statement, and the table search covers everything the database holds.

just check with every step except audit, just lint-integration and just openapi-check pass. The dal tier (106) and vault tier (489) pass in full. The node tier passes 337 of 349; the 12 that fail are s3_backup, snapshot_static_tier and nats_event_bus, none of which this touches, and every one failed pulling rustfs/rustfs:1.0.0 or nats:2 from Docker Hub (HTTP 429), so those three suites are unverified here. audit is red on main for wasmtime and a yanked zip, unrelated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant