Repository navigation
WIP: author Python expressions directly as protobuf - #449
Closed
saulshanabrook wants to merge 4 commits into
Closed
saulshanabrook wants to merge 4 commits into
saulshanabrook wants to merge 4 commits into
Conversation
Store generated messages directly behind immutable owner/index references and pack rooted closures iteratively. Pin the standalone generated-message package; preserve existing public behavior until the next migration slice. Verified 63 ownership/relocation tests, scoped Ruff and mypy, and resolver lock freshness without native builds.
Use canonical record references for runtime expressions and catalog-backed scalar APIs; pack commands directly and execute over the bytes-only extension seam. Preserve declaration closure and result ownership without the old AST execution fallback. The independently reviewed scalar checkpoint has 87 focused passes. Actual thread query remains a deliberate failing regression pending protocol support; full providers, user declarations, rules, readers and type coverage are incomplete. Preserve this intermediate artifact before the next schema epoch.
Return actual native thread counts, preserve integer-index inputs and clone/pop restoration, and classify proof request references in the canonical record inventory. Update immutable shared pins without unrelated dependency changes. Independent review and 120 focused installed-wheel tests pass. High-level module collection remains blocked by unmigrated dynamic classes; no_decomp and broad typing are incomplete, and strict Clippy reports diagnostics in unchanged native files. This is not full frontend conformance.
Keep preserved Python hooks with canonical declaration owners and derive call metadata from records. Retain keywords, defaults, recursive sort references and class lifetimes; reject unsupported decorated members before publication. Verified by 144 installed-wheel tests and independent source review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Work in progress: author Python expressions directly as generated protobuf records and execute them through the shared bytes adapter. This is a compatibility experiment, not a completed migration or a merge-ready release.
RuntimeExprholds a canonical record reference rather than the old expression dataclasses. Packing preserves declaration closure and ordered commands; decoded results retain their declaration owners. Scalar builtin wrappers consume an exported engine-owned catalog, with local.value, pretty printing and deconstruction over records. The native extension transports bytes and does not lower Python syntax or fall back to the old AST execution path.Depends on egraphs-good/egglog-experimental#80 and egraphs-good/egglog#1047. Dependency revisions are immutable; no local path overrides are required.
Supported checkpoint
Exprinitializers now author equality-sort/constructor records directly, including keyword/default arguments and exact supplied names. Preserved Python methods/properties stay local; their owner-scoped lifetime survives extraction without a global strong class registry. Unsupported decorated members reject before callable publication.Remaining work
The complete package is not migrated: constants, symbolic user methods/functions, generic classes, rules/schedules, other builtin providers, custom costs, observation/freeze/recording, richer pretty output and generated typing surfaces remain unfinished. Broad tests and type checking do not pass yet. Existing compatibility requirements remain part of the goal; unsupported operations are not accepted as permanent API removals.
Proof requests/results and creation modes are represented in the shared schema, but native proof execution is not yet exposed by this frontend.
Verification
The independently reviewed scalar/resource/constructor checkpoint passes144 focused tests against a built and installed native wheel, with no deselections or source bootstrap. The coordinator independently reran the full admitted set. Catalog bytes match the engine export; changed source modules match the wheel and installed package. Targeted typing, Python lint/format and Rust formatting pass.
The unchanged original high-level test module now reaches
constant(...)before failing collection on that unmigrated producer; an ignored interpreter-shutdown warning follows the error. Neither is claimed passing. Nondefaultno_decompremains explicitly unsupported. The earlier strict Clippy gate reports13 diagnostics in unchanged native source; it is not claimed green or rerun for this Python-only slice.Earlier full-suite results are native baselines only, not migrated coverage. No performance claim or new native compiler is included.