Skip to content

add test for False positive with overloaded async iterators #2873 - #2939

Closed
asukaminato0721 wants to merge 2 commits into
meta:mainfrom
asukaminato0721:2873
Closed

asukaminato0721 wants to merge 2 commits into
meta:mainfrom
asukaminato0721:2873

Conversation

@asukaminato0721

@asukaminato0721 asukaminato0721 commented Mar 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes #2873

The change has two parts.

overload signatures are normalized when the implementation is an async generator, so async def overload stubs returning AsyncIterator[...] no longer stay wrapped as Coroutine[..., AsyncIterator[...]].

generator decomposition now handles unions member-by-member, which fixes the bogus yield Patch() vs FullState error from AsyncIterator[Patch] | AsyncIterator[FullState].

Test Plan

add test

@meta-cla meta-cla Bot added the cla signed label Mar 26, 2026
@asukaminato0721
asukaminato0721 marked this pull request as ready for review March 26, 2026 22:02
Copilot AI review requested due to automatic review settings March 26, 2026 22:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a false-positive overload/yield error involving async generator implementations with overload stubs returning AsyncIterator[...], and unions of async-iterator return types.

Changes:

  • Normalize overload return signatures when the implementation is an async generator (so overloads don’t remain wrapped as Coroutine[..., AsyncIterator[...]]).
  • Update generator/async-generator decomposition to handle Union types member-by-member and union the decomposed components.
  • Add a regression test covering overloaded async iterator returns and yield behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
pyrefly/lib/test/overload.rs Adds a regression test reproducing issue #2873 with overloaded async generator returning AsyncIterator[...].
pyrefly/lib/alt/unwrap.rs Decomposes generator / async generator types across union members and unions their yield/send/return components.
pyrefly/lib/alt/function.rs Normalizes overload signatures for async generator implementations before overload consistency checking.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pyrefly/lib/alt/function.rs Resolved
@github-actions github-actions Bot added size/m and removed size/m labels Mar 26, 2026
@asukaminato0721
asukaminato0721 marked this pull request as draft March 26, 2026 23:13
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@asukaminato0721
asukaminato0721 marked this pull request as ready for review March 27, 2026 01:46
@github-actions github-actions Bot added size/m and removed size/m labels Mar 27, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Based on AI review, I think this needs some more work. The mypy primer changes are encouraging but AI review found a correctness regression, and the intended fix changes overload semantics rather than merely removing a false positive.

Review notes:

Genuine coroutine returns are incorrectly erased

[function.rs:2231–2249](

fn normalize_async_generator_overload_signature(&self, sig: &Callable) -> Callable {
if let Some((_, _, ret)) = self.unwrap_coroutine(&sig.ret)
&& self.decompose_async_generator(&ret).is_some()
{
let mut sig = sig.clone();
sig.ret = ret;
sig
} else {
sig.clone()
}
}
fn normalize_async_generator_overloads(
&self,
overloads: Vec1<(TextRange, OverloadType)>,
impl_sig: &Callable,
) -> Vec1<(TextRange, OverloadType)> {
if self.decompose_async_generator(&impl_sig.ret).is_none() {
return overloads;
)

decompose_async_generator(&impl_sig.ret) checks return-type compatibility, not whether the implementation is an async generator. It succeeds for Any, so a valid synchronous coroutine factory annotated -> Any triggers normalization.

Consequently:

  • Async overloads returning AsyncIterator[T] lose their effective Coroutine wrapper.
  • Ordinary def overloads explicitly returning Coroutine[..., AsyncIterator[T]] are also rewritten.
  • Valid await factory(...) calls become “not awaitable” errors. Genuine overload inconsistencies can disappear.

I reproduced both cases against exact base/head builds. At minimum, require actual generator provenance and distinguish implicit async wrapping from explicitly declared coroutine returns. A syntactic guard also avoids allocating solver variables and running subtype checks for unrelated synchronous overloads.

Overload meanings depend on the implementation

[function.rs:418–427](

let sigs = if let Ok((impl_sig, _)) =
def_ty.toplevel_callable_signatures().exactly_one()
{
self.normalize_async_generator_overloads(sigs, impl_sig)
} else {
sigs
};
self.check_signature_consistency(&sigs, def_ty, undecorated, errors);
Type::Overload(Overload {
signatures: sigs.mapped(|(_, sig)| sig),
)

Identical declarations remain coroutine-returning in stubs/protocols but become iterator-returning alongside a yielding implementation. That conflicts with the [overload specification](https://typing.python.org/en/latest/spec/overload.html#invalid-overload-definitions).

My recommendation is to preserve the signatures and use ordinary def for the iterator-returning overload declarations. The PR’s regression snippet already passes the unchanged base with just that adjustment; mypy agrees.

Either remove the union hunk or cover it in tests

The original snippet has no yield errors on the exact base. Existing yield checking already splits top-level unions through HintRef. The unwrap.rs change needs its own failing regression demonstrating why it is necessary.

Note that splitting this into a separate PR might be a good option.

@github-actions github-actions Bot added size/m and removed size/m labels Nov 1, 2026
@asukaminato0721 asukaminato0721 changed the title fix False positive with overloaded async iterators #2873 add test for False positive with overloaded async iterators #2873 Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

@meta-codesync

meta-codesync Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

This pull request has been merged in 5f3385f.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

False positive with overloaded async iterators

4 participants