Skip to content

Commit 7302f17

Browse files
committed
perf: Stop patching AST, use functions instead
Previously we were patching AST classes, adding our own class to their base classes. This allowed us to extend these classes, by adding properties such as `kind`, `children`, etc. Now we stop patching classes, and re-define these properties as simple functions that take an AST node as argument. Their result isn't cached anymore, and we refactor them as iterators/generators. We also stop using classes for AST nodes unparsing into values / expressions when feasible. Finally, we explode the node utilities into different submodules, in the new `nodes` folder. The result is: - increased performance: for some reason, benchmarks show that performance is better without cached properties. Maybe MRO and cached values access was more costly. - increased memory efficiency: without so many cached properties, we consume a less memory. This is not super impactful because `compile` and file reads represent almost all our memory consumption. - better static typing: the AST patch would have required a mypy plugin to be understood correctly by mypy. We don't need that anymore. - easier-to-use library: we don't have to hardcode the call to `patch_ast` anywhere anymore, and users won't have to either.
1 parent bc446e4 commit 7302f17

17 files changed

Lines changed: 1813 additions & 1447 deletions

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
1616
- [`GriffeLoader.resolve_aliases(only_exported)`][griffe.loader.GriffeLoader.resolve_aliases]: Deprecated parameter was removed and replaced by `implicit` (inverse semantics)
1717
- [`GriffeLoader.resolve_aliases(only_known_modules)`][griffe.loader.GriffeLoader.resolve_aliases]: Deprecated parameter was removed and replaced by `external` (inverse semantics)
1818
- [`LinesCollection.tokens`][griffe.collections.LinesCollection]: Public object was removed (Python 3.7)
19-
- [`ASTNode.end_lineno`][griffe.agents.nodes.ASTNode]: Public object was removed (Python 3.7)
19+
- `ASTNode.end_lineno`: Public object was removed (Python 3.7)
2020
- [`griffe.agents.extensions`][griffe.agents] Deprecated module was removed and replaced by [`griffe.extensions`][]
2121

2222
### Features

src/griffe/agents/inspector.py

Lines changed: 3 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424

2525
import ast
2626
from inspect import Parameter as SignatureParameter
27-
from inspect import Signature, cleandoc, getmodule, ismodule
27+
from inspect import Signature, cleandoc
2828
from inspect import signature as getsignature
2929
from typing import TYPE_CHECKING, Any
3030

@@ -51,8 +51,6 @@
5151
from griffe.expressions import Expression, Name
5252

5353

54-
from functools import cached_property
55-
5654
empty = Signature.empty
5755

5856

@@ -99,58 +97,6 @@ def inspect(
9997
).get_module(import_paths)
10098

10199

102-
_cyclic_relationships = {
103-
("os", "nt"),
104-
("os", "posix"),
105-
("numpy.core._multiarray_umath", "numpy.core.multiarray"),
106-
("pymmcore._pymmcore_swig", "pymmcore.pymmcore_swig"),
107-
}
108-
109-
110-
def _should_create_alias(parent: ObjectNode, child: ObjectNode, current_module_path: str) -> str | None:
111-
# the whole point of the following logic is to deal with these cases:
112-
# - parent object has a module member
113-
# - if this module is not a submodule of the parent, alias it
114-
# - but break special cycles coming from builtin modules
115-
# like ast -> _ast -> ast (here we inspect _ast)
116-
# or os -> posix/nt -> os (here we inspect posix/nt)
117-
118-
child_obj = child.obj
119-
if isinstance(child_obj, cached_property):
120-
child_obj = child_obj.func
121-
122-
try:
123-
child_module = getmodule(child_obj)
124-
except Exception: # noqa: BLE001
125-
return None
126-
if not child_module:
127-
return None
128-
129-
if ismodule(parent.obj):
130-
parent_module = parent.obj
131-
else:
132-
parent_module = getmodule(parent.obj) # type: ignore[assignment]
133-
if not parent_module:
134-
return None
135-
136-
parent_module_path = getattr(parent_module.__spec__, "name", parent_module.__name__)
137-
child_module_path = getattr(child_module.__spec__, "name", child_module.__name__)
138-
parent_base_name = parent_module_path.split(".")[-1]
139-
child_base_name = child_module_path.split(".")[-1]
140-
141-
# special cases: inspect.getmodule does not return the real modules
142-
# for those, but rather the "user-facing" ones - we prevent that
143-
# and use the real parent module
144-
if (parent_module_path, child_module_path) in _cyclic_relationships or parent_base_name == f"_{child_base_name}":
145-
child_module = parent_module
146-
child_module_path = getattr(child_module.__spec__, "name", child_module.__name__)
147-
148-
is_submodule = child_module_path == current_module_path or child_module_path.startswith(current_module_path + ".")
149-
if not is_submodule:
150-
return child_module_path.lstrip("_")
151-
return None
152-
153-
154100
class Inspector:
155101
"""This class is used to instantiate an inspector.
156102
@@ -254,13 +200,8 @@ def generic_inspect(self, node: ObjectNode) -> None:
254200
before_inspector.inspect(node)
255201

256202
for child in node.children:
257-
child_module_path = _should_create_alias(node, child, self.current.module.path)
258-
if child_module_path:
259-
if child.kind is ObjectKind.MODULE:
260-
target_path = child_module_path
261-
else:
262-
child_name = getattr(child.obj, "__name__", child.name)
263-
target_path = f"{child_module_path}.{child_name}"
203+
target_path = child.alias_target_path
204+
if target_path:
264205
self.current.set_member(child.name, Alias(child.name, target_path))
265206
else:
266207
self.inspect(child)

0 commit comments

Comments
 (0)