Skip to content

Commit 86e311d

Browse files
committed
feat: compile per-class hydrators/serializers, warmable cache, lazy collections
Performance refactor: from() and toArray() now execute a specialized closure generated per data class (plain properties become inline array reads; casts, enums, nested DTOs, collections, and pipes delegate to the existing runtime via captured metadata — behavior is unchanged). Steady-state throughput: hydration ~2.6x, serialization ~2.2x over the interpreted path. - Add HydratorCompiler and SerializerCompiler (eval once per class per process, in-memory registries, flush hooks); remove the interpreted Hydrator; extract ValueNormalizer for compiled serializers - Precompute ParameterMeta::$isPlain so hot paths skip ValueCaster; inline the is_array() input check; use it in with() overrides too - Cache format v2: .meta.php files now carry the compiled hydrator and serializer alongside the metadata — a warmed FPM worker pays neither reflection nor eval (opcache serves the whole file); legacy v1 files still load - Add vendor/bin/sdo-warm + Support\CacheWarmer: scans sources (PSR-4 dirs from composer.json by default) for concrete BaseData subclasses and pre-builds the cache on deploy; fails fast on invalid DTO definitions, reports non-exportable classes as skipped - Add BaseData::lazyCollection() for streaming large iterables with a flat memory profile (~0.26 MB peak for 50k rows vs ~13 MB materialized) - Docs: compiled hot path, pre-warming guide, streaming collections, README performance section 245 tests, 100% coverage.
1 parent 9daafda commit 86e311d

20 files changed

Lines changed: 950 additions & 146 deletions

README.md

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,12 +23,27 @@ composer require std-out/simple-data-objects
2323

2424
| | Simple Data Objects |
2525
|---|---|
26-
| Reflection | Once per class, then zero — file cache compiled by opcache |
26+
| Hot path | Compiled per-class closures — zero reflection, zero dispatch overhead |
2727
| Boilerplate | None — constructor props + attributes |
2828
| Roundtrip | `from(toArray())` always works, mapped keys included |
2929
| Standalone | Validation works without a Laravel app |
3030
| Pipelines | Middleware-style input preprocessing, class or property level |
3131

32+
### Performance
33+
34+
Benchmarked against **the most popular full-featured data-object library in the PHP/Laravel ecosystem** — identical DTO shapes, 20,000 iterations per scenario, PHP 8.4:
35+
36+
| Scenario | Simple Data Objects | Popular alternative | Advantage |
37+
|---|---|---|---|
38+
| Hydration — flat DTO | ~4,500,000 ops/s | ~130,000 ops/s | **~35× faster** |
39+
| Hydration — nested DTO | ~2,200,000 ops/s | ~74,000 ops/s | **~30× faster** |
40+
| Hydration — collection of 20 | ~220,000 ops/s | ~7,500 ops/s | **~29× faster** |
41+
| Serialization — flat DTO | ~7,400,000 ops/s | ~200,000 ops/s | **~37× faster** |
42+
| Serialization — nested DTO | ~4,000,000 ops/s | ~117,000 ops/s | **~34× faster** |
43+
| Peak memory — streaming 50,000 rows | 0.26 MB with `lazyCollection()` | ~13 MB | **~50× less memory** |
44+
45+
Absolute numbers vary with hardware; the ratios stay stable across runs. CPU time per operation follows the same ratios — less CPU burned per request means more headroom per server.
46+
3247
---
3348

3449
## Quick Look
@@ -109,12 +124,30 @@ final class UpperCasePipe implements ValuePipe
109124

110125
### Zero-reflection in production
111126

127+
`from()` and `toArray()` compile a specialized closure per class — plain properties become direct array reads. Enable the file cache and the compiled code persists between requests:
128+
112129
```php
113130
// bootstrap / AppServiceProvider — run once
114131
MetadataRegistry::setStoragePath(storage_path('framework/data-objects'));
115132
```
116133

117-
First access writes a PHP file per class. Every subsequent request opcache serves it — **zero reflection**.
134+
Pre-warm it on deploy so even the first request is hot:
135+
136+
```bash
137+
vendor/bin/sdo-warm storage/framework/data-objects app/Data
138+
```
139+
140+
Every worker then starts with opcache-compiled metadata **and** hydration/serialization code — zero reflection, zero compilation at runtime.
141+
142+
### Streaming large datasets
143+
144+
`lazyCollection()` hydrates one item at a time as the collection is consumed — peak memory stays flat no matter how many rows flow through:
145+
146+
```php
147+
foreach (UserData::lazyCollection($csvRows) as $user) {
148+
$importer->process($user); // 50k rows, ~0.26 MB peak instead of ~13 MB
149+
}
150+
```
118151

119152
### Immutable copies with `with()`
120153

bin/sdo-warm

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
#!/usr/bin/env php
2+
<?php
3+
4+
declare(strict_types=1);
5+
6+
// Locate the autoloader. Order matters: composer's bin proxy provides the
7+
// exact path; otherwise prefer the project the command is run FROM (cwd) —
8+
// with a path-repository symlink __DIR__ resolves inside the package source,
9+
// where the relative guesses would pick the wrong vendor tree.
10+
$candidates = [
11+
$_composer_autoload_path ?? null,
12+
getcwd().'/vendor/autoload.php',
13+
__DIR__.'/../../../autoload.php',
14+
__DIR__.'/../vendor/autoload.php',
15+
];
16+
17+
foreach ($candidates as $autoload) {
18+
if ($autoload !== null && is_file($autoload)) {
19+
require $autoload;
20+
break;
21+
}
22+
}
23+
24+
use StdOut\SimpleDataObjects\Support\CacheWarmer;
25+
26+
$args = array_slice($argv, 1);
27+
28+
if ($args === []) {
29+
fwrite(STDERR, "Pre-builds the metadata + compiled hydrator/serializer cache for every\n");
30+
fwrite(STDERR, "concrete BaseData subclass found in the scanned sources.\n\n");
31+
fwrite(STDERR, "Usage: sdo-warm <cache-dir> [<src-path> ...]\n\n");
32+
fwrite(STDERR, "Without <src-path> arguments, the PSR-4 directories from ./composer.json\n");
33+
fwrite(STDERR, "are scanned.\n\n");
34+
fwrite(STDERR, "Example: vendor/bin/sdo-warm storage/framework/cache/data-objects app/Data\n");
35+
exit(2);
36+
}
37+
38+
$cacheDir = array_shift($args);
39+
40+
if ($args === []) {
41+
$args = CacheWarmer::pathsFromComposer(getcwd().'/composer.json');
42+
43+
if ($args === []) {
44+
fwrite(STDERR, "No source paths given and no PSR-4 autoload directories found in ./composer.json.\n");
45+
exit(2);
46+
}
47+
48+
echo 'Scanning PSR-4 paths from composer.json: '.implode(', ', $args)."\n\n";
49+
}
50+
51+
try {
52+
$result = CacheWarmer::warm($cacheDir, $args);
53+
} catch (Throwable $e) {
54+
fwrite(STDERR, 'ERROR: '.$e->getMessage()."\n");
55+
exit(1);
56+
}
57+
58+
if ($result['warmed'] === [] && $result['skipped'] === []) {
59+
fwrite(STDERR, "WARNING: no concrete BaseData subclasses found under: ".implode(', ', $args)."\n");
60+
fwrite(STDERR, "Check that the classes are autoloadable from the current working directory.\n");
61+
}
62+
63+
foreach ($result['warmed'] as $class) {
64+
echo " warmed {$class}\n";
65+
}
66+
67+
foreach ($result['skipped'] as $class) {
68+
echo " skipped {$class} (metadata not exportable — in-memory cache only)\n";
69+
}
70+
71+
printf(
72+
"\n%d class(es) warmed, %d skipped → %s\n",
73+
count($result['warmed']),
74+
count($result['skipped']),
75+
$cacheDir,
76+
);
77+
78+
exit(0);

composer.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
"illuminate/http": "Required to use HasLaravelIntegration::fromRequest() and toResponse()",
2828
"illuminate/database": "Required to use HasLaravelIntegration::fromModel()"
2929
},
30+
"bin": ["bin/sdo-warm"],
3031
"autoload": {
3132
"psr-4": {
3233
"StdOut\\SimpleDataObjects\\": "src/"

docs/features/cache.md

Lines changed: 29 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ The first time a DTO class is hydrated, the library uses PHP Reflection to read
66

77
For subsequent calls in the **same PHP process**, reflection is skipped entirely. This covers long-running runtimes like **Laravel Octane**, **Swoole**, and **RoadRunner** with zero extra configuration.
88

9+
On top of the metadata, `from()` and `toArray()` compile a **specialized closure per class** on first use (kept in memory for the process): plain properties become direct array reads/writes, and only properties with casts, enums, nested DTOs, or pipes go through the richer runtime path. Behavior is identical to the metadata-driven path — this is purely an execution-speed optimization.
10+
911
## File-based Cache (PHP-FPM)
1012

1113
In traditional PHP-FPM environments, each request starts a fresh process. Enable the file cache to persist metadata between requests:
@@ -20,32 +22,47 @@ On first access, a PHP file is written for each class. Subsequent requests `requ
2022

2123
### Cache File Format
2224

23-
Cache files use `var_export()` with `__set_state()`, not `serialize()`:
25+
Cache files use `var_export()` with `__set_state()`, not `serialize()`, and carry the **compiled hydrator and serializer closures** alongside the metadata:
2426

2527
```php
2628
<?php
2729

28-
return StdOut\SimpleDataObjects\Support\ClassMeta::__set_state([
29-
'parameters' => [
30-
StdOut\SimpleDataObjects\Support\ParameterMeta::__set_state([
31-
'phpName' => 'name',
32-
'inputName' => 'name',
33-
'allowsNull' => false,
34-
// ...
35-
]),
36-
],
37-
]);
30+
$meta = \StdOut\SimpleDataObjects\Support\ClassMeta::__set_state([/* ... */]);
31+
$p = $meta->parameters;
32+
$pipes = $meta->pipes;
33+
34+
return [
35+
$meta,
36+
static function (array $d) use ($p, $pipes): \App\Data\UserData { /* compiled hydration */ },
37+
static function (\App\Data\UserData $o) use ($p): array { /* compiled serialization */ },
38+
];
3839
```
3940

4041
This means:
4142
- **No deserialization gadget chains** — no `unserialize()` call
42-
- **Opcache-friendly** — the file is compiled to opcodes once
43+
- **Opcache-friendly** — the whole file, closures included, is compiled to opcodes once; a warmed process pays neither reflection nor closure compilation
4344
- **Human-readable** — easy to inspect during debugging
4445

4546
### File Naming
4647

4748
Cache files are named `sha256(classname).meta.php`. There is no path traversal risk regardless of class naming, and the distinct `.meta.php` suffix guarantees cache clearing can never touch foreign files.
4849

50+
## Pre-warming on Deploy
51+
52+
Instead of letting the first request pay the build cost, generate the cache ahead of time with the bundled CLI:
53+
54+
```sh
55+
# scan specific paths
56+
vendor/bin/sdo-warm storage/framework/cache/data-objects app/Data
57+
58+
# or scan all PSR-4 directories from your composer.json automatically
59+
vendor/bin/sdo-warm storage/framework/cache/data-objects
60+
```
61+
62+
It scans the sources for **concrete** `BaseData` subclasses (abstract bases are skipped), builds each class's metadata **and compiled closures**, and writes the cache files. Add it to your deploy script right before the app goes live — every FPM worker then starts fully warm.
63+
64+
Classes whose metadata cannot be exported (see [Limitations](#limitations)) are reported as `skipped` and keep using the in-memory cache. A broken DTO definition (e.g. conflicting attributes) fails the command immediately — deploy-time is exactly when you want to find out.
65+
4966
## Clearing the Cache
5067

5168
```php

docs/features/collections.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,20 @@ $collection = UserData::collection([$user, [...]]);
3535
// $collection->first() === $user
3636
```
3737

38+
## Lazy Collections
39+
40+
`lazyCollection()` returns an `Illuminate\Support\LazyCollection` that hydrates items **one at a time as they are consumed**, instead of materializing everything upfront. Use it for large iterables — a DB cursor, a generator streaming a big CSV — where holding every hydrated instance in memory at once would be wasteful:
41+
42+
```php
43+
$names = UserData::lazyCollection($csvRowGenerator)
44+
->take(3)
45+
->map(fn (UserData $u) => $u->name)
46+
->all();
47+
// only 3 rows were ever hydrated, no matter how large the source is
48+
```
49+
50+
Like `collection()`, already-hydrated instances pass through unchanged.
51+
3852
## Nested Collections in DTOs
3953

4054
Use `#[DataCollection(ClassName::class)]` to declare a property as a typed collection:

docs/guide/installation.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,12 @@ use StdOut\SimpleDataObjects\Support\MetadataRegistry;
5353
MetadataRegistry::setStoragePath(storage_path('framework/data-objects'));
5454
```
5555

56+
Pre-warm the cache on deploy so the first request pays nothing (see [Metadata Cache](../features/cache.md#pre-warming-on-deploy)):
57+
58+
```bash
59+
vendor/bin/sdo-warm storage/framework/data-objects app/Data
60+
```
61+
5662
Clear on deploy:
5763

5864
```bash

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ features:
3131
details: "#[Cast], #[Rules], #[Flatten], #[Hidden], #[IgnoreIfNull] — all behaviour defined where the property is declared."
3232
- icon: 🚀
3333
title: Production-ready performance
34-
details: Reflection runs once per class and is cached in memory. Optional file cache lets opcache pre-compile all metadata.
34+
details: Hydration and serialization compile to per-class closures. The file cache plus the sdo-warm CLI give production zero reflection and zero runtime compilation.
3535
- icon: 🛡️
3636
title: Secure by default
3737
details: EncryptedCast uses XSalsa20-Poly1305 authenticated encryption. Validation throws before any hydration occurs.

0 commit comments

Comments
 (0)