Repository navigation
Conversation
…Data#2061) Overloads that share a base QN (Java/C#/C++/Kotlin/Swift/Scala/ObjC) merge into one graph node today, and that node collects the CALLS of every overload. The agreed cure is one signature-qualified callable identity for all overloading languages: callable_qn := base_qn [<tparams>] (params) [cvref] This change lands the plumbing with every language still at CBM_CALLABLE_ID_NONE, so the graph is byte-identical; each language is switched on later in its own change (with its index-format bump). - internal/cbm/callable_sig.{c,h}: the per-language mode side table (NONE|TYPED|LABELED_TYPED|LABELED|ARITY; kept out of CBMLangSpec like cbm_string_dispatch_suffixes, since ~160 positional rows would otherwise all need the member under -Wmissing-field-initializers), the shared builder cbm_callable_sig[_mode] for the def and the call-scope QN, and the inverse cbm_qn_callable_base_len[_named]. Suffix contract: no '.', '::' or '->'; whitespace-normalized; comments/annotations/attributes dropped; qualified types keep their last segment; '->' spelled '=>'; variadic '[]' (Java) / '~'; C++ top-level cv dropped, arrays decay, cvref kept; <= 200 bytes, longer ones keep whole leading entries and end in '#<fnv1a64 of the full suffix>)'. The builder is iterative (cursor walk, bounded declarator loop) and self-checks against the inverse. - The inverse is anchored to the node's bare name wherever the name is known: real QNs already end in ')' without being callables (a Java field whose name is its declarator `X = f()`, make `$(VAR)` targets), and only the anchor makes the inverse exact on them. - Leaf splitters strip a suffix first (no-op today): registry by-name key (indexed by the bare name for a suffixed callable), cbm_lsp_bare_segment, cbm_pipeline_qn_class_method_tail, pxc_last_component, pxc_qn_leaf, go_index_visit_node, collect_prod_defs, sg_qn_prefix_len. - New unused fields: CBMDefinition.qn_sig_off, CBMResolvedCall.callee_sig (relocated by result compaction), CBMRegisteredFunc.sig, CBMLSPDef.sig. The lsp_surface codec is unchanged (no format bump in this change). - get_code_snippet and trace_path gain base-QN tiers (cbm_store_find_nodes_by_qn_base) that find nothing until suffixes exist. - store: cbm_store_find_nodes_by_qn_suffix built its LIKE pattern in a 512-byte buffer (a long suffix was truncated into a pattern that matched nothing) and did not escape '_'/'%' (`my_func` also matched `myXfunc`). The pattern is now heap-built, escaped, and matched with ESCAPE '\'. Proof: full sorted node+edge+lsp_surface dumps are byte-identical before and after on 7 corpora (this repo, elasticsearch libs, Exposed, dotnet System.Linq, System.Collections, coreclr/gc, a C corpus). Refs DeusData#2061 Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Enable typed, labeled Swift callable identities on the shared DeusData#2061 plumbing. Resolve calls by compatible labels, defaulted parameters, and trailing closures; retain all compatible overload edges with candidate counts. Cover registry, pipeline, and signature behavior. Refs DeusData#2061 Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Force the parallel call pass on a Swift fixture and assert both label-compatible type overloads retain CALLS edges with the ambiguity count. Refs DeusData#2061 Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
A Swift call that shares its bare name with project symbols but matches none of their overloads was handled differently by the two resolvers: the parallel path cleared the bare-name match and fell through as unresolved (keeping each resolver's empty-resolution service fallbacks), while the sequential path dropped the call outright. Skip the bare-name registry bind on the sequential path too — it would bind a wrong overload — and continue as unresolved, so count < 0 behaves exactly like any other unresolved call on each resolver. Pipeline regressions pin the global-fetch fallback (DeusData#856) on both paths: the incompatible local overload gets no wrong CALLS edge, and the URL call keeps its HTTP_CALLS edge. The route-registration fallback differs between the paths on main (sequential additionally requires a '/'-leading first argument); that pre-existing divergence is out of scope here. Refs DeusData#2061 Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
3441da4 to
5538355
Compare
Signed-off-by: DavidHLP <lysf15520112973@163.com>
|
Thank you for the detailed status update and for clearly distinguishing the actual PR head from unpublished work and unverified results. The changed-file scope includes callable identity, resolution and persisted metadata, so we need more time for a complete review together with the parent change. This remains queued for deeper review; this comment is not an approval or a request for another routine rebase. The queue is full, and we appreciate your patience and the care taken to document the remaining work. |
Swift-scope checkpoint for DeusData#2061: capture bounded trailing-closure labels, match Swift overload candidates conservatively, and cover extraction, registry, compact/spill, serial, parallel, and incremental behavior. Validation: ASan passed on the combined WIP source only; this A-only checkpoint has not been independently verified. GCC TSan, smoke, and memory gates remain failed/open. This commit is a scope checkpoint, not acceptance. Signed-off-by: David HLP <lysf15520112973@163.com>
Signed-off-by: David HLP <lysf15520112973@163.com>
Signed-off-by: David HLP <lysf15520112973@163.com>
Signed-off-by: David HLP <lysf15520112973@163.com>
Signed-off-by: David HLP <lysf15520112973@163.com>
Signed-off-by: David HLP <lysf15520112973@163.com>
Signed-off-by: David HLP <lysf15520112973@163.com>
4dbfd27 to
eea3143
Compare
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Preserve Swift identity and upstream image migration coverage. Advance the combined index format to 6 so pre-merge format-5 indexes also rebuild invented image hosts. Refs DeusData#2061. Signed-off-by: DavidHLP <lysf15520112973@163.com>
|
Thanks for the Swift identity update and for distinguishing current-head checks from earlier implementation validation. I have read your request for feedback on identity spelling and the migration boundary; those decisions remain on the maintainer side. Please hold off on further routine rebases while that review is pending. |
|
Thank you for the Swift identity measurements and the migration questions. Those decisions remain on our side; please continue to hold off on routine rebases while that review is pending. Sorry for the delay—I’ll follow up soon. |
DeusData
left a comment
There was a problem hiding this comment.
Thank you, @DavidHLP, for carrying the Swift identity on top of #2342 exactly as we proposed on #2061, for your patience while we held the spelling and migration questions on our side, and for the care in this branch. Keeping the selector uncapped and separate from the bounded identity, carrying it through serial, parallel and incremental indexing, and filtering candidates before the output limit are all the right calls. Here are our decisions on the questions you asked, and what we need before this can land. The hold on updates is lifted: please bring the branch up to current main (merging main in is fine).
We built your current head (4885230) and indexed Alamofire (bda9ed57) with it. The core fix works: on the #2061 repro, work(flag:Bool) and work(name:String) are separate nodes, onlyCallsOverloadB reaches only work(name:String), and the fake path to target() is gone. On Alamofire, 74 overload sets that were one node each are split into 404 nodes, including all 13 Session.upload overloads, and repeated runs give an identical graph.
-
Identity spelling. Please keep the static/class and async markers and the explicit =>Return. They separate real Swift overloads: Alamofire's three encode(_:any Encodable) differ only in their return type. For qualified types, please use the last segment, as in the #2342 contract, instead of / paths. Two declarations that differ only in the scope of a same-named parameter type then stay one node, as they are on main today. That case is rare, and it keeps Swift QNs consistent with the other languages. One detail we will confirm separately: associated types of a generic parameter in a where-requirement (for example where T.Element == Int). Please keep your current spelling for that one case until we follow up on it.
-
Keep the receiver's type in charge. The label block in resolve_single_call (and its twin in pass_parallel.c) runs before the registry's receiver-typed strategies and returns early, so the label shape can override the receiver's declared type. On Alamofire, swift_labels resolves 3,650 CALLS while field_type_hint resolutions fell from 884 to 1 compared with main at the time. In Tests/CachedResponseHandlerTests.swift:41, session is a Session, but session.request(.default) resolves to DataResponsePublisher.Inner.request(_:Subscribers/Demand) with candidates:1. That one pattern adds 175 edges from tests to the Combine Inner.request methods, the same kind of fake caller #2061 is about. Calls on standard-library values get bound too: output.sorted() on a [String] in URLEncodedFormEncoder.swift:1076 points at HTTPHeaders.sorted(). Please apply the label and arity filter inside the receiver-typed candidates rather than in front of them, and leave a call unresolved when the receiver is a known non-project type. A regression test with a typed receiver whose method shares its label shape with an unrelated class would pin this.
-
Chained calls with trailing closures. Some calls that main resolves are dropped, for example session.request(.default).responseDecodable(of:) { ... } (Tests/SessionTests.swift:1632) and session.streamRequest(...).responseStream { ... } (Tests/DataStreamTests.swift:582). These are common Swift idioms, so a test for each would be very welcome.
-
Language-neutral names for the new fields. CBMDefinition and CBMCall are shared by every language, and Kotlin, Scala and C# will need default-argument and trailing-lambda matching when their identity changes land. Please give the new struct fields and the persisted property keys neutral names (for example param_default_mask, param_count, call_selector and trailing_closure_*), so those changes can reuse them.
-
Index format 5. main is at index format 4 (#1913, receiver-qualified Go methods), so this change becomes format 5 with a single history entry. The format-6 step and the format-5 image migration test cover a state that only existed on this branch (main already stopped inventing image hosts in format 3), so please drop both. When you update the branch you will see conflicts in pass_calls.c (main now resolves through cbm_registry_resolve_call_scoped), pass_definitions.c and pass_parallel.c (main reworked the def-properties sizing into per-field helpers, so the selector becomes one more term there, and registration now goes through cbm_registry_add_lang_in_module), pipeline_incremental.c, pipeline.h and store.h.
-
Initializers. #2395 is about to land and makes Swift init and deinit callable definitions (Box.init becomes a Method node). Because the identity applies to every callable, your branch will give them signature QNs too, for example Box.init(name:String), and Box.deinit() under the zero-argument rule, which is what we want. Please update the #2395 tests that look up the unsuffixed Box.init and Deinitable.deinit definitions, and add tests where two init overloads stay separate nodes, a super.init call reaches the matching Base.init overload, and an external UIViewController.init(nibName:bundle:) with a label-compatible project init forms no edge. #2395 makes self.init form no edge because overloads used to share one node; we will revisit that separately once your change is in, so please leave it as it is.
-
Three smaller points from reading the code:
a. When the signature cannot be built, extract_func_def, push_method_def and compute_func_qn now return without a definition or QN. Please fall back to the plain base QN instead, so a definition is never lost.
b. cbm_service_pattern_match now strips a callable suffix for every language with cbm_qn_callable_base_len. Other languages' QNs can end in ')' for other reasons, so please use the name-anchored form or limit it to Swift.
c. cbm_registry_swift_candidates scans the whole same-name bucket and allocates on every call. With initializers in the graph, init becomes by far the largest Swift bucket, so the cost grows with initializers times calls. Please look up the named type's overloads directly when the call has a type receiver, and drop a small Swift fixture into tests/fixtures/complexity/ so the complexity suite covers it (no test code change is needed there).
Before merging we will run a Swift call-graph census on the updated branch ourselves (Alamofire, one larger Swift project and a non-Swift control), so you do not need to produce those numbers.
To be clear about what is already good here: outside the session.request cluster, a random sample of call sites whose target changed came out better with your branch than with main (3 fixed, none broken, the rest wrong either way), and most of the 372 calls that only your branch resolves are correct, for example Interceptor(adapter:retrier:) and self.state.write. So the label matching itself is a real gain; it just must not override the receiver.
Thank you again for the very careful work on this, and sorry it took us a while to come back with the decisions.
Keep receiver types ahead of overload labels, preserve chained trailing closures, and index typed overload buckets directly. Use neutral call metadata, retain base identities on signature failure, and publish index format 5. Preserve current main registration and properties sizing. Signed-off-by: DavidHLP <lysf15520112973@163.com>
…ritance Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
|
Following up on your review: I have addressed the requested contributor-side changes at
Validation on this PR head: 196 focused tests passed under ASan/UBSan with The local full clang-tidy sweep still reports pre-existing warnings reproduced on main; I have not changed its thresholds or suppressions. Other local full-test problems were reproduced against main's implementations and isolated in #2809, rather than added to this Swift PR. Those findings are now tracked in #2814 for a maintainer decision, with #2809 retained as a draft candidate. Its full-test results are not being presented as validation of this PR's head. I believe the requested code changes are complete and this is ready for your Swift call-graph census and final review. As you specified, the census of Alamofire, a larger Swift project, and a non-Swift control remains on the maintainer side. |
|
Thank you, @DavidHLP, for the very thorough update and the point-by-point reply. We checked all seven points against debefaf and each one is addressed in the source, with a test for every case, which made this easy to verify. The next step is ours: we will run the Swift call-graph census on the merged tree (Alamofire, a larger Swift project and a non-Swift control) and post the numbers here. We also plan to land #2395 first, since your branch already carries it; after that, merging main into this branch drops the diff back to the identity change itself. Nothing is needed from you in the meantime. Thank you again for the care in this branch. |
|
Thank you for the careful review and for verifying all seven points. The plan to land #2395 first and then run the census makes sense. I appreciate the time you’re putting into validating this across projects, and I look forward to seeing the results. |
|
Thank you, @DavidHLP, for this very thorough update. We checked all seven points of our review against debefaf and every one is addressed, with a test for each, which made this easy to verify. Thank you also for keeping the #2395 integration and the index-format history so tidy: your branch merged cleanly with current main (including #2395), and the format 5 step rebuilds an existing index exactly once. We ran the Swift call-graph census on Alamofire (bda9ed57), swift-nio (eabcc14f) and helm as a non-Swift control, each indexed from scratch with your branch merged into current main and with main itself:
The receiver-first resolution clearly works: in URLEncodedFormEncoderTests, encoder.encode(...) now goes to URLEncodedFormEncoder.encode instead of JSONParameterEncoder.encode, in swift-nio context.close(promise: nil) binds to ChannelHandlerContext.close(mode:promise:) through the defaulted label, and many wrong guesses such as metrics.append -> MultipartFormData.append are gone. One remaining case: when a receiver's type is known, the lookup only checks the members filed under that one type QN and its base classes. In Swift a type's members often live in an Thank you again for the care in this branch. We are close. |
Include matching extension owners in typed Swift method and initializer lookup. Preserve owner metadata for incremental indexing and cover three-file resolution in sequential, parallel, and incremental runs. Refs DeusData#2061 Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
|
Thank you, @DavidHLP, for turning this around so quickly. d33c6ac is exactly the shape we asked for: the extension members are now part of the typed owner lookup, initializers included, and the new test puts the type, its extension and the caller in three separate files and checks the sequential, parallel and incremental paths. Thank you also for the careful merge in 05c7ba7: you carried main's unresolved-call bookkeeping into both Swift label paths, which we would otherwise have had to ask for. We re-ran the same census as last time (Alamofire bda9ed57, swift-nio eabcc14f, helm as the non-Swift control, every run indexed from scratch), comparing your previous head (debefaf merged with current main) to 05c7ba7:
On the two sites from last time: session.request(.default) at SessionTests.swift:1632 now resolves to the three Endpoint overloads, and session.streamRequest(.bytes(1)) at DataStreamTests.swift:582 to its two. The chained .responseDecodable and .responseStream after them are still unresolved. All candidates of each call return the same type (DataRequest and DataStreamRequest), so if every compatible candidate agrees on the return type, the chain could continue with that type. One new problem comes from the lookup key. Extensions are matched by the type's path inside its module (swift_type_path / swift_extension, for example "Service"), so every type with that name anywhere in the repository shares its extensions:
Could you:
A small pre-existing gap explains the few wrong new edges we saw on Alamofire, and it is not caused by this change: the We are also looking at a few further points and will follow up on those separately. Thank you again for the quick and careful work; the receiver-first resolution keeps getting better. |
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Keep extension owners within visible targets and respect literal custom SwiftPM roots, nested package boundaries and manifest invalidation. Continue compatible call chains through declared return types and bounded initializer inference. Recover conditional declaration ownership and wrapped trailing closures, and preserve serial argument keywords. Add extraction, registry and serial/parallel/incremental regressions. Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
Signed-off-by: DavidHLP <lysf15520112973@163.com>
|
Thank you, @DeusData, for rerunning the census and documenting the exact call sites. The two filesystem targets in swift-nio and the generated
I also changed Alamofire’s target path from The PR head passed the default production regression checks. The local validation tree, which included the separate fixes from draft #2809, completed all 192 suites under ASan and UBSan with 10,363 passed, 0 failed, and 10 skipped. All 164 FastAPI incremental tests ran and passed. GitHub CI for the PR head passed Linux, macOS, Windows CLANG64, TSan, CodeQL, DCO, and I kept the real-repository rerun focused on the reported call sites and manifest invalidation. I haven’t rerun the complete census or the generated 60/120-directory benchmark. Thanks again for the detailed review. |
|
Thank you, @DavidHLP, for another fast and careful round, and for the clear point-by-point reply. 1a6b292 closes the three structural problems from this morning, and the conditional-compilation work goes further than we asked: it fixes the missing We re-ran the same census as this morning (Alamofire bda9ed57, swift-nio eabcc14f, helm 9b88efd6 as the non-Swift control, every run indexed from scratch), comparing 05c7ba7 to 1a6b292:
One thing went the other way: 1,976 call sites on swift-nio (103 on Alamofire) that 05c7ba7 resolved now get no edge, and for 1,362 of them (61 on Alamofire) main resolves the same target. In a sample about half of these were right before, so this is a real loss. It also shows up as test links: TESTS on swift-nio drops from 52 to 30. A few shapes we saw:
Could you:
Two smaller observations, not required for this PR: 288 Alamofire call sites such as session.request(.default) now get 6 targets (the three Endpoint overloads in TestHelpers.swift plus the three URLConvertible overloads in Session.swift), which is expected from labels alone now that those overloads have their owner. And We are also looking at a few further points and will follow up on those separately. Thank you again; the structural part is in very good shape now, and the census shows how much the conditional-compilation fix adds. |
|
Hi @DavidHLP, a quick follow-up to our comment from earlier today: to save you another round, we will make the four changes from that list ourselves on your branch (maintainer edits are enabled), and re-run the same census on the result. If you have already started on any of them, just say so here and we will build on your work instead of duplicating it. Otherwise, please hold off pushing to this branch until we post the result, or pull first, so neither of us loses commits. Thank you again for all the work on this series - the structural part is in very good shape now. |
|
Thank you, @DeusData . I haven’t started working on these fixes yet, so there are no changes from my side to coordinate. I appreciate you handling the fixes and rerunning the census. |
|
Hi @DeusData, Thank you again for taking this on. Following my earlier reply, I ran some verification in an isolated Docker environment, comparing I reproduced the generic-return and There was one difference in my run: These observations are specific to |
Summary
Refs #2061. Give Swift overloads declaration-qualified identities, preserve bare node names, and use the uncapped selector inside receiver-typed candidates. Keep static/class, async, and explicit return markers. Qualified types retain their last segment, and the associated-type spelling in
where T.Elementis retained pending maintainer follow-up.Package.swiftinvalidate source-only repair and trigger a full reindex.e0c7a1da, including fix(extraction): attribute Swift lifecycle calls #2395.Validation
AI assistance: This update was prepared with an AI agent acting on behalf of DavidHLP.
1a6b2923passed the default production regression guards. On Alamofirebda9ed57, the chain tails atTests/SessionTests.swift:1632andTests/DataStreamTests.swift:582each resolved to the expected method. On swift-nioeabcc14f, the NIOFS and _NIOFileSystem cross-target CALLS check returned zero.SourcetoSource/Coretriggeredincremental.force_full. The rebuilt and fresh graphs of that same modified checkout matched on node and edge rows, including properties. Earlier general incremental comparisons recordedSIMILAR_TOanddecorator_tagsdifferences separately.1a6b2923plus the remaining test-environment and baseline fixes from draft fix: isolate validation and handle empty initialization state #2809, retaining main's mimalloc 3.5.3. The defaultscripts/test.shpassed with GCC, ASan and UBSan, andUBSAN_OPTIONS=halt_on_error=1. It completed 192 suites with 10,363 passed, zero failed, and 10 skips. Theincrementalsuite ran all 164 tests using FastAPI0.99.1atdd4e78ca, with no skips. The fix: isolate validation and handle empty initialization state #2809 changes remain separate from this PR.1a6b2923passed Linux, macOS, Windows CLANG64, TSan, lint, CodeQL, DCO, shard completeness, andci-ok. PR workflow · DCO