Skip to content

Document the 1.4.0 release and host migration #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
luisleo526 merged 3 commits into from
Oct 8, 2026
Merged

Document the 1.4.0 release and host migration #191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fc4ede7
docs: prepare scoped 1.4.0 codegen release notes
luisleo526 Oct 8, 2026
c203463
docs: clarify 1.4.0 settings migration and HPO compatibility
luisleo526 Oct 8, 2026
9ed00c4
docs: correct failure classes and HPO option guidance
luisleo526 Oct 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 177 additions & 15 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,90 @@ supported as exact pairs; on the 0.x line they are independent. See the
the release commit. A prerelease note describes changes since the preceding
prerelease or stable tag; the final stable note consolidates the series.

## Unreleased
## 1.4.0 — Unreleased

A minor release of the exact engine/codegen pair, prepared from codegen
`bfc4ddce` and engine `b3192bfc`. It consolidates the seven requests landed
since `v1.3.0`: [#183](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/183)
(emission ordering), [#184](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/184)
(coded run stops), [#185](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/185)
(input manifests), [#186](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/186)
(collection history and binding), [#188](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/188)
(CI history), [#189](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/189)
(reference tests), and [#190](https://github.com/pineforge-4pass/pineforge-codegen-oss/pull/190)
(emitted-string regressions and edge cases). Recorded-output support from
engine request #356 is excluded. This planned release has not been tagged;
its candidate `VERSION` files still read `1.3.0` until the release workflows
set them.

- Generated C++ no longer depends on memory layout: requested-context history
resets and TA variant position ties preserve their traversal order. Refusal
### Compatibility and migration

- Codegen 1.4.0 supports only engine `v1.4.0`. Regenerate every strategy's C++
and relink with that release's generated headers and `libpineforge.a`.
The pair retains C ABI version 4, settings/capability API version 1 and the
`engine_script_run_v19` epoch; equal version numbers for those interfaces
do not make mixed engine/codegen releases supported.
A report's fingerprint records engine/codegen versions, generated C++'s
hash and the paired engine's corrected provenance values. The retained
epoch and hash domain tags do not promise equal fingerprints, state-hash
values or trades across 1.3.0 and 1.4.0.
- Emitted C++ changes, including common run-stop shims and collection history
wrappers. Do not reuse a 1.3.0 strategy library or assume source identity.
The public Python call signatures and override keys stay the same, but
`inputs` values and the Python/Pyodide success envelopes change as described
below. These are not blanket byte-identity promises for emission or envelopes.
- Every manifest row now carries `supported`. When it is `false`, hide the
input, exclude it from sweeps and optimization, and do not send a value:
the strategy uses its compiled default, and the paired engine refuses a
supplied value with `setting_unsupported`. `supported: true` alone does
not validate the value's type, bounds or options, or make a shared key safe.
A supported empty dropdown and an unsupported string input can both have
`options: []`; that list does not determine support.
- Give each input a unique title before upgrading. Calls with the same title
or fallback declaration key still produce separate manifest rows with the
same `title`; their `supported` flags do not detect the collision. Different
`group=` labels do not separate keys. On the paired 1.4.0 engine's normal
checked path, a completed run of such a script has `fingerprint: null` even
without an override. Supplying the shared key fails before execution with
`setting_rejected`, reason `ambiguous_key`. For example,
`Period="7"` returned a success report on 1.3.0, whose legacy setter applied
a shared key to all matching inputs. Legacy-path unresolved provenance rows
may still appear inside a fingerprint; that differs from the whole
fingerprint being null on the checked path.
- The paired engine's run harness now uses the checked settings API. Stored
presets and sweep values must satisfy the compiled input's type, bounds and
options. Two other examples that succeeded on 1.3.0 now fail with
`setting_rejected`: `Color="color.blue"` (`expected_integer`) and
`Choice="15"` for options `[10, 20, 30]` (`invalid_input_option`). A color's
manifest still has `type: "string"` and a Pine spelling such as `"color.red"`
as its default, but the checked native input requires the packed integer;
do not forward that manifest string as a native color value. The checked
API also validates parseable numbers, booleans, enum members and known keys.
This does not mean all invalid settings previously succeeded:
`initial_capital="abc"` and `initial_capital="-1"` already failed on 1.3.0;
they now report `expected_finite_decimal` and `value_below_minimum` reasons.
Legacy C setters retain their existing behavior.
- The paired engine adds `strategy_get_last_error_code` and
`strategy_get_last_error_args` beside the existing English error getter,
under `PINEFORGE_HAS_RUN_FAILURE_CODES_V1`. Its run harness exposes `code`
and typed `args` beside `error` on failures. Check the failure channel even
when the English message is empty; do not accept a stopped run's partial
output as success or parse the English to recover a code. These run failures
are distinct from transpile-time `CompileError` diagnostics.
`setting_rejected` is catalog class `input`; `setting_unsupported` is class
`unsupported`. A deliberate `runtime.error` is `strategy_runtime_error`,
class `strategy`. The installed engine entrypoint returns exit 4 for all
three codes: exit 4 alone does not select a billing or refund decision.
- The paired engine certifies scalar input/override types in fingerprint
provenance; unresolved values remain explicit rather than being treated as
certified native values. This is an engine report change, not another field
added to the transpile-time input manifest. See the paired engine's release
notes for report resolution reasons and counts.

### Emission and run failures

- Requested-context history resets and TA variant position ties no longer
use address or set order; they preserve traversal order. Refusal
diagnostics for conflicting scalar history and map locals are hash-seed
independent.
- Generated run stops use stable engine failure codes when the paired engine
Expand All @@ -41,48 +121,130 @@ supported as exact pairs; on the 0.x line they are independent. See the
names; substring arguments retain source order without copying the source text.
- Generated run wrappers preserve the inner failure code, and latched setter
failures and checked strategy construction retain their failure identity.

### Python and Pyodide input metadata

- The transpile-time input manifest (`transpile_full()["inputs"]` and the JSON
envelope of the Pyodide package) gains `supported`, the checked-settings
receipt's flag, on every input, and reads its defaults, choices and bounds
from the descriptor that generates the receipt, so the two agree for every
input of the public corpus and the repository's fixtures. Generated C++,
`strategyParams` and `requests` are unchanged and no key is renamed or
removed, but these values of existing keys change:
from the descriptor that generates the receipt, with the retained limitations
below. No manifest key is renamed or removed, but these values of existing
keys change:
- Source inputs (`input.source`, a plain `input(close)`): `default` is the
series name, where it was `null`, and `options` lists the nine native
sources, where it was absent.
- Enum inputs: `options` lists the `Enum.member` choices, where it was
absent.
absent; `default` keeps its `Enum.member` spelling. The compiled receipt
encodes that default as its integer member value.
- String inputs: a built-in constant publishes its runtime value
(`alert.freq_all` is `"all"`, `currency.USD` is `"USD"`; the manifest held
the Pine names), a named string constant its value, and the choices of
`input.timeframe` and `input.session` are listed. What the receipt cannot
represent (`size.small`, `position.top_right`, `na`) is `default: ""`,
`options: []` and `supported: false`.
- Typed `int`, `float`, `price`, `time` and `bool` inputs: a signed or
represent (`size.small`, `position.top_right`, `na`) has `supported: false`
and `options: []`; an unavailable default is `""`, but a representable
default survives an unsupported choice. String decoding uses the emitter
escape grammar, keeps literal Unicode and escaped backslashes, and stops
at the first NUL like the receipt's `std::string(const char*)` constructor.
NUL is emitted as a fixed three-digit octal escape even before digits;
this is consistency with current native truncation, not lossless NUL storage.
- Typed `int`, `float`, `price` and `time` inputs: a signed or
named-constant `default` is a number, where it was `null`; `min`, `max` and
`step` appear for signed and named-constant bounds (`minval=-80` was left
out); an `options=[...]` dropdown lists its numbers.
out); an `options=[...]` dropdown lists its numbers. A literal or inlined
`bool` default publishes a boolean, not a number.
- A plain `input(-5)` or `input(-2.5)` is typed `int` or `float` with its
number as `default`, where it was `string` with `null`.
number as `default`, where it was `string` with `null`. The compiled
getter and checked receipt keep their existing float type for these
signed plain inputs; this manifest correction does not change that lowering.

Values the receipt computes at run time (`LEN * 2`, `not true`,
`timestamp(year, month, ...)`, a color) stay unpublished, as before.
`timestamp(year, month, ...)`) stay unpublished: the default remains
`None`/JSON `null`, an unfoldable bound is omitted, and the whole `options`
field is omitted when any numeric choice is unfoldable. An `input.color`
retains its Pine spelling as a string default, not the receipt's packed
integer; other nonliteral plain `input()` calls remain `string` with no
published default (the source case above is the exception). A folded
`timestamp("...")` is a publishable literal. These gaps
are explicit in `tests/test_input_metadata_receipt.py::KNOWN_LIMITS`, not a
claim of arbitrary expression evaluation or language soundness.
- Documents the timeframe spellings of two surfaces, both stable: a capability
receipt records a request's timeframe as the script wrote it (`"D"`), and
request discovery and feed keys use the engine's spelling (`"1D"`).

### Collection lowering and refusals

- Bind keyword arguments of the matrix methods and the checked array methods to their own parameter slot, with the omitted optional parameters taking their defaults in place (`matrix.sort(m, order = order.descending)`, `m.submatrix(to_column = 1)`, `a.fill(7.0, index_to = 1)` no longer drop or shift a keyword), and refuse a keyword that names no parameter, or a parameter given twice, with the existing argument diagnostics. A defaulted `submatrix` evaluates its receiver once, an index-only `add_row` / `add_col` takes any numeric index, and a bound `array.concat` result (`c = a.concat(b)`, `c := array.concat(a, b)`) keeps the unsupported-function refusal. An array history that reaches a user function's or method's parameter keeps its earlier copy lowering and the earlier refusal of a callee that checks it with `na()`; a bound missing-history matrix reports the recorded na-ID runtime error.
- Compile matrix/matrix and matrix/scalar `matrix.sum` overloads with matrix-valued results, including helpers and history. Support scalar `matrix.diff`/`matrix.mult` and omitted optional arguments to matrix row/column insertion, submatrix, sort and array fill; unsupported matrix/vector multiplication and bound `array.concat` results report an existing unsupported-function diagnostic instead of failing C++ compilation.
- Preserve na array IDs when a history offset has no value, including aliases and `na()`-only bindings; array methods on those IDs report the existing runtime error instead of crashing. Bounds-check legacy collection-element reads and writes to prevent unchecked access.

The inherited follow-ups remain: keywords on UDT matrix fields, `matrix.new`
and array methods outside the checked table; `matrix.fill` range handling
(both spellings still fill the whole matrix); matrix diff/mult dimension
checks; and nullable history across callable/iteration shapes not covered by
the documented lowering. This release does not claim those cases resolved.

### Emitted-string edge cases and maintenance

- Return the prefix or ticker of the supplied symbol in `syminfo.prefix(symbol)`
and `syminfo.ticker(symbol)` instead of always reading the chart symbol; the
variable forms stay unchanged.
variable forms, no-argument calls and direct chart-ID calls keep their
chart lowering. Supplied-symbol calls split at the first colon; a prefix
is empty without a colon or with no ticker after it, and a ticker without
a colon is the full string. Non-string arguments still compile and yield
an empty string, including the retained string-`na` typing gap.
- Refuse malformed library versions with the existing import diagnostic instead
of allowing them to raise outside the JSON gate, including non-ASCII decimal
digits and ASCII versions exceeding Python's integer-conversion limit.
- Reserve generated receipt exports and version macros against script names.
- Escape embedded NUL values in generated C++ string literals, and strengthen
emitted-string regression tests with explicit emitter reach and round trips.
- CI fetches full branch history for historical reference tests (#188); two
whole-C++ comparisons use the accepted immutable scaffold (#189). These
changes affect verification only, not the compiler or execution semantics.

The hardening wording remains scoped to the previously documented emitted-string
validation and escaping; these edge-case repairs and regression checks are not
a new arbitrary-source security guarantee. No exploit payloads are included.

### Known issues with consumers

The new numeric options expose an exact-type comparison in
`pineforge-hpo` 0.11.0: a JSON integer `20` is refused with
`invalid_input_option` for an `input.float` choice `20.0`, although the
numbers are equal. For HPO 0.11.0, send each selected value with the number
type the original manifest publishes for that option: `20.0` when it lists
`20.0` (explicit JSON `20.0` or a Python float), and integer `20` when it lists
`20`. A float input written with whole-number-literal options such as
`options=[10, 20, 30]` publishes integer options; mixed lists retain mixed
number types. The input's `float` type alone does not choose the workaround.

This can affect a caller that reads a float-spelled manifest default and
sends it back through JavaScript's ordinary JSON serialization, which writes
a whole-number value as `20`, losing its float spelling. Writing `20.0` in
JavaScript before ordinary serialization does not preserve that distinction;
preserve each option's original number type at the HPO boundary. Upgrade to
HPO 0.11.1 or a later release containing the fix when available. This fix is
separate from the engine/codegen release; HPO 0.11.1 has not been released at
the time of writing.

### Recorded validation and grades

The landed #190 candidate recorded 9,254 engine-enabled pytest passes with
12 audited skips, 2,164 focused passes each under GCC and Clang without skips,
314 public corpus compilation passes, and 279 fixtures in each of the JSON
and Pyodide fixture-parity checks. Those
are the request's retained native measurements, not a new native suite run
for this documentation change. The release-pair baseline records
excellent: <!-- pf:releases[1.4.0].scoreboard.excellent|int -->7,983<!-- /pf -->;
strong: <!-- pf:releases[1.4.0].scoreboard.strong|int -->6<!-- /pf -->;
graded: <!-- pf:releases[1.4.0].scoreboard.graded|int -->7,989<!-- /pf --> probes,
with <!-- pf:releases[1.4.0].scoreboard.belowStrong|int -->0<!-- /pf --> below strong
and <!-- pf:releases[1.4.0].scoreboard.engineErrors|int -->0<!-- /pf --> engine errors.
These are recorded grading outcomes, not a statement that no internal attempt
ever failed. Baseline <!-- pf:releases[1.4.0].scoreboard.id|code -->`pineforge-parity-baseline-20261007-engine-b3192bfc`<!-- /pf -->
measures engine <!-- pf:releases[1.4.0].scoreboard.engineCommit|short-code -->`b3192bfc`<!-- /pf -->
and codegen <!-- pf:releases[1.4.0].scoreboard.codegenCommit|short-code -->`bfc4ddce`<!-- /pf -->;
these quantities render from the prepared release facts, not numeric overrides.

## 1.3.0 — 2026-10-06

Expand Down
Loading
Loading