Testing and Validation

Testing and Validation

V14 includes a dedicated set of security and regression tests because the package reader and restore path are security-critical.

Test layout

The tests/ directory includes coverage for:

Cryptography and format

• CryptoRoundTripTest.php
• KdfParameterTest.php
• CanonicalJsonTest.php
• CanonicalFuzzTest.php
• V14DescriptorTest.php
• V14HeaderBindingTest.php
• V14PackageRoundTripTest.php
• V14TamperHeaderTest.php
• V14BlobTamperTest.php
• V14RecoveryTest.php

Package and restore safety

• V14PackageArchiveTest.php
• V14PackageArchiveStorageTest.php
• V14RestoreSafetyTest.php
• TamperDetectionTest.php
• FileRoundTripTest.php
• PackageCatalogTest.php
• PackageAuthorizationTest.php

Authentication/security state

• ApplicationSecretsTest.php
• SecurityDiagnosticsTest.php
• SecurityInvariantTest.php
• ReplayGuardTest.php
• ReplayRaceTest.php
• SignedStateTest.php
• RestoreTokenTest.php
• NameRegistryTest.php

Uploads and archive policy

• ResumableUploadTest.php
• UploadChecksumTest.php
• V13UploadTest.php
• ZipPolicyTest.php

Release/security tooling

• ReleaseIntegrityTest.php
• ReleaseVerifierTrustRootTest.php
• SourceSecurityAuditTest.php

Browser compatibility

• BrowserV13RoundTripTest.mjs
• V13DescriptorTest.php

Fast vs full security suites

Default:

php bin/security-test.php

Full:

php bin/security-test.php --full

The fast suite runs a curated security-focused list. The full suite executes all tests/*Test.php files discovered by the script.

Syntax checks

PHP syntax can be checked across the active source tree with:

find . -type f -name '*.php' -not -path './vendor/*' -print0 | xargs -0 -n1 php -l

Browser JavaScript syntax can be checked with:

find public/client -type f -name '*.js' -print0 | xargs -0 -n1 node --check

The project validation record also checks active PHP and JavaScript syntax.

Validation record: 2026-10-01

The repository's validation document reports:

• 103 active PHP files passed php -l.
• Active JavaScript files passed node --check.
• php bin/self-check.php passed the required PHP/Sodium/randomness/Argon2id checks.
• php bin/security-test.php passed the fast security suite.
• php bin/source-audit.php passed.
• V14 package round-trip passed.
• V14 recovery restore passed.
• V14 header binding and strict schema tests passed.
• V14 header/completion tamper tests passed.
• V14 blob tamper and pre-write integrity tests passed.
• V14 restore-safety and no-overwrite tests passed.
• Replay race, upload checksum, signed state, authorization, and existing security regression tests passed.

Known environment limitations in that record

The validation record also documents limitations:

• The validation sandbox did not have the PHP Zip extension, so V14PackageArchiveTest.php reported a skip.
• Composer was not installed in that sandbox, so composer validate and composer audit were not run there.
• No synthetic composer.lock was generated; release CI deliberately requires a real lockfile.
• The development tree was unsigned, so tests that require an actual Ed25519 release artifact remained skipped until a real signing key was supplied.

These limitations are important. A test report should preserve both successful evidence and the conditions under which some checks could not run.

Security testing philosophy

Tests focus on invariants rather than only happy-path examples.

Useful adversarial invariants include:

modified header -> reject
modified ciphertext -> reject
missing blob -> reject
unreachable blob -> reject
wrong Merkle root -> reject
unsafe restored name -> reject
existing destination -> reject
replay identifier -> reject
invalid release signature -> reject
untrusted external release key -> reject

Fuzzing and future work

The repository contains canonical JSON fuzz coverage and parser-focused validation, but the project still documents the need for continued adversarial testing, reproducible release validation, and independent review.

Interpreting test status

A green test suite means the tested assertions passed. It does not mean:

• the code is mathematically proven secure;
• every platform filesystem behaves identically;
• every PHP/Sodium version is equivalent;
• the host is trusted;
• the dependencies are risk-free;
• a third-party audit has occurred.

The project's security posture should therefore be communicated as explicit threat-model coverage plus evidence, not as a numeric security score.