Repository navigation
feat(java): no name guess for calls on a proven-external receiver - #2667
Conversation
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
DeusData
left a comment
There was a problem hiding this comment.
Thank you very much for this, @tfactor2. It is a careful and well-reasoned change, and it removes a whole class of fabricated edges that has bothered us for a long time. We measured it properly before replying, and the numbers are very good.
What we measured. We merged the PR head (0ec6967) onto current main and indexed 15 open-source Java repositories with and without it on the parallel resolver, plus 6 of them on the sequential one. We hand-judged a stratified sample of the changed edges against the source.
- Parallel, 15 repos: CALLS 749,216 → 665,992, i.e. −83,225 (−11.1 %), with 1 edge added. About 96 % of the sampled removals were wrong edges, for example:
Thread.sleep,Calendar.getInstanceandMap.getbound to same-named project methods;new Date()/new ArrayList<>()bound to apom.xmlelement;String.formatbound to a projectformat.
The remaining ~3–4 % were calls on a JDK interface type whose project implementation happened to be the guessed target (ParameterizedType.getOwnerType()→Utils.ParameterizedTypeImpl.getOwnerType). That trade-off is a broader design question on our side; nothing for you to change here.
- Sequential, 6 repos: CALLS −5,079 / +680. The additions come from the new Tier-2 registry line for small repositories and were correct in every sampled case: calls on project interfaces now reach the interface method, and constructor calls reach the constructor.
Two small requests before we merge:
1. Own package (and the file's own types) before java.lang. java.lang is an implicit on-demand import (JLS §7.3), and an on-demand import never shadows a type of the compilation unit's own package (§6.4.1, §7.5.2). The resolver checks java.lang first. Before this PR, that wrong first answer was harmless because the textual fallback still found the project class. Now it marks the receiver as external and the edge is dropped. Minimal repro (both resolver paths, with 50+ files so the shared registry is built):
// src/main/java/com/acme/protogen/Enum.java
package com.acme.protogen;
public class Enum {
public Enum(Class<?> cls, String parent) {}
}
// src/main/java/com/acme/protogen/Message.java
package com.acme.protogen;
public class Message {
public void addNestedEnum(Class<?> cls) {
Enum e = new Enum(cls, "parent"); // expected: CALLS -> com.acme.protogen.Enum constructor
}
}On main this produces addNestedEnum -> com.acme.protogen.Enum.Enum. With the PR the edge is gone, because Enum resolves to java.lang.Enum and is classified as external. The same shape occurs in Netflix/conductor (protogen.Enum). Project classes named Process, Module, Record, Override and so on are affected the same way. Could you move the own-package lookup ahead of java.lang in the type-name resolution and add a test like the one above?
2. PR description. Please restate the numbers for this PR alone. As far as we can tell, the current figures include the follow-ups you mentioned for #2382 and #2662. Please also call out the pass_lsp_cross.c line that builds the Java shared registry on the sequential path, or split it into its own commit. It is a good change, but it is a behaviour change for every repository of 50 files or fewer, and reviewers should see it as such.
Thanks again; we are looking forward to merging this.
0ec6967 to
bbbd8ba
Compare
|
Thanks for measuring. Both points are done, and the branch is rebased on 1. Own package before One more thing from your example. On 2. Description. The numbers are now for this PR only. The sequential registry line is its own commit, 8e9caf9, and the description marks it as a behaviour change for repositories of 50 files or fewer. clang-format 20 is clean ( |
When the Java resolver resolves a call's receiver to a type outside the project (a JDK package, or a package no project file declares), the call gets strategy lsp_external. Both pipeline paths (pass_calls.c and pass_parallel.c) then look the target up strictly by QN through cbm_pipeline_java_external_target and skip the registry's name guess, so no edge is created to an unrelated project method of the same name. This is the rule the Rust path follows since DeusData#2053. Positive proof only: a receiver of unknown type keeps the existing guess, and a seeded JDK type that lacks the called method stays no_method_match. Type variables (T, ID) are never taken for package-qualified types. A package counts as the project's when any registry layer holds a project type in it (the layer above the shared registry is a per-file overlay), and a path-derived spelling counts as the project's also for a file at the repository root. java_receiver_is_external memoizes its answer per package. A type of the caller's own package, or of its own file, now shadows java.lang (JLS 6.4.1): java_resolve_type_name looks the JVM package up before the java.lang auto-import. Before the gate that wrong first answer was harmless, since the name guess still found the project class; with the gate a project `Enum` or `Process` would read as external. Only the parallel path builds the Java cross registry, so on the sequential path the gate does not fire yet; the next commit adds it. Tests: java_typed_calls_parallel (external receivers get no edge; a project receiver keeps its edge, also when its type is missing from a project package or shares its simple name with a java.lang type), parallel_java_external_target_contract. Signed-off-by: Timur Nizametdinov <7229951+tfactor2@users.noreply.github.com>
pass_lsp_cross.c built the shared Java registry only on the parallel path (more than 50 files). On the sequential path every Java file was resolved against its own definitions alone, so a call on a type from another file fell back to the name guess, and the external-receiver gate of the previous commit never fired. This is a behaviour change for every repository of 50 files or fewer: calls whose receiver type is declared in another file now resolve through the LSP, as they do on the parallel path, and calls on JDK and library types lose their guessed edge. It also fixes the package-qualified parameter case measured in DeusData#2382. Tests: java_typed_calls_sequential (the parallel fixture on the sequential path, plus a parameter typed with its package-qualified name), java_typed_calls_paths_agree (both paths produce the same edges). Signed-off-by: Timur Nizametdinov <7229951+tfactor2@users.noreply.github.com>
bbbd8ba to
8e9caf9
Compare
On Windows the parallel path binds `c.getName()` (Customer comes from an import) to Filters.getName: the textual registry's name guess picks another same-named method there. This PR does not change that guess, so the test no longer pins its target: - on the parallel path, customerName only has to keep an edge; the sequential path still checks Customer.getName; - java_typed_calls_paths_agree compares only the edges the Java resolver found by type (lsp_*), not the registry's. Signed-off-by: Timur Nizametdinov <7229951+tfactor2@users.noreply.github.com>
Brings the branch up to main (a5b248f); it was 670 commits behind. One conflict, in src/pipeline/pass_calls.c: main (d4273ae) moved the LSP-target lookup of resolve_single_call into calls_emit_lsp_target and the registry fallback into calls_registry_resolution. The Java gate of this branch now lives in those helpers, unchanged in effect: - resolve_single_call computes java_external once, as before, and passes it to both helpers; - calls_emit_lsp_target looks a Java lsp_external callee up strictly by QN with no tail match, as the branch did inline; - calls_registry_resolution skips the name guess for such a call, next to the Rust and PHP vendor skips. pass_parallel.c, pass_lsp_cross.c, lsp_resolve.h, Makefile.cbm and the test registrations merged without conflicts. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The review of this change named a literal case: a project class Enum in package com.acme.protogen, and new Enum(cls, "parent") in a sibling class of that package (Netflix conductor has this shape: protogen.Enum). java.lang is an on-demand import, and an on-demand import never shadows a type of the own package (JLS 6.4.1), so the call means the project class. java_typed_calls_own_package_enum_sequential and _parallel index that pair on both pipeline paths and assert one CALLS edge from addNestedEnum to the Class node com.acme.protogen.Enum, as new does for every project type here. RED with only the type-name order of this branch reverted (java.lang before the own package): Enum reads as java.lang.Enum, the call counts as external, and both new tests find no edge. GREEN with this branch. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Thank you very much, @tfactor2, for answering both points so carefully, and for this change as a whole. Moving the own-package lookup ahead of java.lang is exactly the JLS 6.4.1 order, the Process and Record cases in java_typed_calls cover it on both paths, and splitting the sequential registry line into its own commit (8e9caf9) made the behaviour change for small repositories easy to review. Your note on We measured your current head merged onto main on two open-source Java repositories, main vs this branch:
Since maintainer edits are enabled, we pushed two commits on top of yours, so this can go to a final decision without another round trip for you:
Locally (macOS, ASan/UBSan) java_typed_calls, java_lsp, java_lsp_coverage, complexity, parallel and pipeline all pass. Please pull the branch before you push again, so these two commits are not lost. Thanks again for the patience and the careful work. This removes a whole class of wrong edges from every Java graph. |
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Thanks for your patience, @tfactor2: CI went green, but the Ruby per-file resolver (#1711) merged into main overnight and touched Makefile.cbm, src/pipeline/lsp_resolve.h and tests/test_main.c, so we merged current main into your branch once more (a plain signed-off merge commit, no conflicts; your change and our two commits are unchanged) to let CI cover the exact tree that will merge. It merges as soon as this run is green. Please pull before pushing again. |
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
Thanks for your patience, @tfactor2: the index resource-worker series (#1724) merged into main last night and touched tests/test_main.c next to your change, so we merged current main into your branch once more (a plain signed-off merge commit; your change and our commits are unchanged) to get a fresh run on the exact tree that will merge. Please pull before pushing again. |
@DeusData , what should be done on my side then? Once conflicts are resolved the PR can be merged into main, no? |
|
Merged - thank you very much, @tfactor2! A call on a receiver whose type is proven to be outside the project no longer gets a name guess onto a project method, and the own-package lookup now comes before java.lang, in JLS order. On Netflix/conductor, 24 of 25 sampled removed edges were wrong ones, so this takes a whole class of wrong edges out of every Java graph. Thanks also for your offer about linking Thanks again for the patient, careful back-and-forth on this one. |
What does this PR do?
Fixes #2660. When the Java resolver knows that a call's receiver is a JDK or library type, the call no longer gets an edge to a project method that only has the same name. Both pipeline paths look such a call up by its full name only, as Rust does since #2339.
When the type is unknown, or is missing from a project package, the old name guess stays. A project type in the caller's package or file now wins over a
java.langtype with the same name (JLS 6.4.1). So a projectEnumorProcessis not treated as a JDK type.Behaviour change in the second commit: repositories of 50 files or fewer now build the Java cross registry too, as larger ones already do. There, calls on types from other files now resolve by type, and the rule above applies.
Known gap:
List.getandList.isEmptystill get a guessed edge.Measured on a Spring service with 2,414 Java files: CALLS went from 68,305 to 66,842 (−1,463), and no edge was added. In a sample of 1,510 calls, 57 lost their edge, and all 57 were wrong. 20 of them were type-proven on
main, for exampleMap.putorList.streamlinked to a project method.Checklist
git commit -s)maintoo (9,895 passed there)lint-no-suppress,lint-memory-core,lint-formatwith clang-format 20.1.8; cppcheck not installed herejava_typed_calls(fails onmain),parallel_java_external_target_contract