Skip to content

Storage and Data Lifecycle

Elman Mohammadi edited this page Oct 6, 2026 · 1 revision

Storage and Data Lifecycle

Secure Package uses filesystem-backed runtime storage for packages, jobs, uploads, catalog data, accounts, audit logs, rate limits, tokens, and temporary restore material.

Storage roots

The application configuration defines these logical areas:

storage/
├── packages
├── temp
├── uploads
├── catalog
├── accounts
├── audit
├── rate-limits
├── chunks
└── upload-inventory

Additional runtime directories are created on demand, including:

storage/secrets/
storage/temp/download-tokens/
storage/temp/restore-tokens/

Packages

The canonical server package is the portable .spkg14 file.

During creation, the builder first creates an internal package directory:

header.json
manifest.enc
complete.json
blobs/

The API then archives it and removes the unpacked package directory to avoid retaining two copies of the encrypted bytes.

Catalog

The package catalog stores metadata such as:

  • package ID
  • owner hash
  • name hash
  • creation timestamp
  • expiration timestamp
  • revocation timestamp

The catalog does not need to store the decrypted project contents.

Upload lifecycle

Uploads move through these broad states:

initialized
  -> partial/chunked
  -> checksum-verified
  -> finalized ZIP
  -> archive job

Upload state is owner-bound and expires after the configured TTL.

Archive jobs

Archive processing uses a temporary job directory with state metadata, source storage, archive staging, and queued nested archives.

Job IDs are 24 random bytes represented by 48 lowercase hexadecimal characters.

Jobs are also owner-bound.

Temporary restore lifecycle

A successful restore produces a temporary restore directory.

A short-lived token is issued to allow the restored content to be downloaded.

The restore token is deleted after consumption or expiration.

Operational cleanup tooling should remove expired temporary state regularly.

File permissions

The implementation attempts to create sensitive directories with 0700 permissions and sensitive files with 0600 permissions.

This is defense in depth. The operating system, PHP process account, filesystem ACLs, container permissions, and deployment architecture still control the final protection boundary.

Secrets directory

When SPK_NAME_PEPPER is not explicitly configured, the application creates:

storage/secrets/name-pepper.bin

It contains a persistent random 256-bit secret.

The entire storage/ tree must remain private and must not be served directly by the web server.

Audit log lifecycle

The audit log is append-oriented JSONL:

storage/audit/events.jsonl
storage/audit/events.jsonl.head

The head file stores the latest hash-chain value.

Deleting or editing historical audit records breaks the chain and should be treated as an integrity event.

Access and restore tokens

Token files store hashed token filenames and the minimum state required for lifecycle handling.

The raw token is returned to the caller but is not used directly as a storage filename.

Data deletion

Package deletion removes the package archive, package directory when present, associated access tokens, and catalog metadata.

Deletion is intentionally destructive and should be treated accordingly.

Backups

Backups must include whatever is necessary to preserve the application's operational state while maintaining credential separation.

In particular, a backup strategy should consider:

  • package archives
  • catalog metadata
  • account records
  • the configured/generated server name pepper
  • audit data
  • deployment secrets managed outside the repository

A recovery key should not be stored merely because the application data is backed up. It is a separate credential and should remain independently protected.

Privacy principle

The encrypted package should be the container for protected project content. Operational metadata should reveal as little sensitive information as practical, and secret values must not be copied into logging or telemetry paths.

Clone this wiki locally