Skip to content

feat: ship py.typed, with precise REST return types and corrected hints - #5

Closed
chipkent wants to merge 4 commits into
masterfrom
feat/typing
Closed

chipkent wants to merge 4 commits into
masterfrom
feat/typing

Conversation

@chipkent

@chipkent chipkent commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Preview only. This is a draft in the fork showing the exact text of the PR planned for massive-com/client-python. It will be closed, not merged, once the upstream PR is opened.

Fixes #<issue: F4>

Summary

Runtime behavior is unchanged. Every change is a type annotation, an @overload declaration, a docstring, a test, or the empty py.typed marker file. No runtime dependency is added. The differences a type checker sees are listed under Typing-visible differences.

Type checkers cannot use the package as published: it ships no py.typed marker, so mypy skips it. Shipping the marker alone would make things worse, because every REST method is annotated Union[Model, HTTPResponse], so ordinary code would stop type-checking:

for t in client.list_tickers():
    t.ticker            # error: "HTTPResponse" has no attribute "ticker"

client.get_previous_close_agg("AAPL")[0]   # error: not indexable

This PR makes the annotations precise and then ships the marker. It has four commits, each reviewable on its own.

Commits

1. fix: correct type hints that disagree with the API. Only hints that describe values the API never sends or returns are changed:

Hint Was Now Evidence
CryptoQuote.bid_price, ask_price, ask_size Optional[int] Optional[float] XQ sends fractional numbers, e.g. "bp": 33052.79, "as": 0.601
IndexValue.timestamp Optional[str] Optional[int] V documents t as Unix milliseconds, e.g. 1678220098130
list_tickers(cusip=...) Optional[int] Optional[Union[int, str]] The API takes a CUSIP as a string: CUSIPs are alphanumeric and leading zeros matter ("037833100"). int is still accepted; the docstring recommends a string
get_previous_close_agg() returns PreviousCloseAgg List[PreviousCloseAgg] It returns the results array; test_get_previous_close_agg already expects a list
list_treasury_yields(), list_short_interest(), list_short_volume(), vx.list_ipos() return List[...] Iterator[...] Like every other list_* method, they return the paginating generator from _paginate

Hints that are imprecise but still admit every value the API sends, such as integer timestamps hinted as float (CryptoQuote.timestamp, PreviousCloseAgg.timestamp, LastTrade.sequence_number), are deliberately left alone: narrowing them would gain little and could break callers.

2. feat: let type checkers see model constructors. @modelclass builds an __init__ that type checkers cannot see, so CryptoQuote(pair="BTC-USD", bid_price=1.5) is reported as "Unexpected keyword argument". This commit marks modelclass with dataclass_transform (PEP 681), imported only under TYPE_CHECKING (from typing on 3.11 and later, from typing_extensions on 3.10). At runtime it is a no-op decorator.

3. feat: overload REST methods on raw so callers get precise return types. Each of the 92 methods that take raw gets three @overload declarations ahead of its unchanged implementation:

@overload
def get_aggs(self, ticker: str, ..., raw: Literal[False] = False, options: ... = None) -> List[Agg]: ...
@overload
def get_aggs(self, ticker: str, ..., *, raw: Literal[True], options: ... = None) -> HTTPResponse: ...
@overload
def get_aggs(self, ticker: str, ..., raw: bool = False, options: ... = None) -> Union[List[Agg], HTTPResponse]: ...
  • Default call: returns the model type, so for t in client.list_tickers() and client.get_aggs(...)[0] check without an isinstance guard.
  • raw=True: returns HTTPResponse. In this overload raw is keyword-only: it follows parameters that have defaults, so it cannot be a required positional parameter, and giving it a default would overlap the first overload, which pyright reports as an error.
  • Any other bool (raw=flag, or raw passed positionally): the full Union, exactly as today.

The overloads are copied from each method's own signature and differ only in raw and the return type. They are generated mechanically, which is why the diff is large (about 6,800 added lines across 14 files). To review, compare one method's overloads against its signature; the rest follow the same pattern. AGENTS.md now describes the pattern for new endpoints.

4. feat: ship py.typed so the package is checked as typed.

  • Adds massive/py.typed (PEP 561). The existing packages entry in pyproject.toml already puts it in the wheel and sdist.
  • Re-exports RESTClient and WebSocketClient with import X as X, the explicit re-export form, so from massive import RESTClient passes mypy --strict. No __all__ is added, so from massive import * exports exactly the same names as before.
  • Adds test_rest/test_exports.py, which fails if massive, massive.rest, massive.rest.models, massive.websocket or massive.websocket.models stops exporting, through from <package> import *, any name it exported before this PR.
  • Adds test_rest/typing_consumer.py: ordinary user code that is type-checked but never run. It iterates list_* results and indexes get_* lists; checks that raw=True gives HTTPResponse and a plain bool gives the union; builds models with keyword arguments; pins each corrected hint with assert_type (an exact check, so Optional[int] does not pass for Optional[float]); and includes code written for earlier releases (narrowing with isinstance, passing an int CUSIP) to show it still type-checks.
  • make static checks that file with mypy --strict from inside test_rest/, where massive resolves to the installed package as it does for users, so the check also fails if py.typed goes missing.

Runtime behavior: unchanged

I compared master with this branch by importing both trees and recording everything observable:

  • Callable signatures: all 957 public functions and methods have the same parameter names, kinds and defaults. The only addition is the no-op dataclass_transform helper in massive/modelclass.py.
  • Models: all 149 have the same dataclass fields, in the same order, with the same defaults.
  • Star-imports: across all 47 modules, from <module> import * loses no names. It gains only the typing helpers the new annotations import (Literal, overload, and sys in modelclass).
  • Method bodies: with the overloads removed, every REST module parses to the same AST as before, apart from the typing import line.
  • Unit tests: every existing test passes unchanged.

Annotations are visible to runtime introspection (__annotations__, typing.get_overloads), so those differ by design. @modelclass reads only the names of its fields' hints, not their types, so changing a hint cannot change behavior.

Typing-visible differences

These affect only static type checkers.

Wider (no existing code breaks):

  • list_tickers(cusip=...) accepts str as well as int.
  • Model constructors now accept their fields as keyword or positional arguments. Before, checkers rejected any argument.
  • A plain call (no raw) now returns the model type instead of Union[Model, HTTPResponse]. Code that narrows with isinstance, or stores the result in a Union-typed variable, still checks.

Narrower (existing code may now be flagged). Each case fixes a hint that was wrong at runtime; the code it flags was already wrong, or is unusual:

  • CryptoQuote.bid_price, ask_price, ask_size are float. Reading one into an int variable is now flagged, but the value was never an int.
  • IndexValue.timestamp is int. Treating it as str is now flagged, but it was never a str.
  • get_previous_close_agg() returns a list. Storing it in a variable typed Union[PreviousCloseAgg, HTTPResponse] is now flagged. At runtime it was always a list.
  • The four list_* methods in commit 1 return an iterator. Storing the result in a variable typed List[...] is now flagged. Indexing it already failed at runtime.
  • TickerChange, TickerChangeEvent and TickerChangeResults are the only models with fields that have no default. Calling one without those fields is now flagged by commit 2. The runtime accepts the call, but reading the missing field then raises AttributeError.
  • Reusing one variable for a plain call and a raw=True call (x = c.list_trades(...), then x = c.list_trades(..., raw=True)) is now flagged by mypy, because the first assignment fixes the variable's type. Annotate the variable with the Union, or use two variables.
  • py.typed itself: mypy now checks calls into massive. Before, it skipped them, or treated them as Any under ignore_missing_imports. Code that relied on Any may now show errors.

Still flagged, as before: type checkers reject unknown keyword arguments, and more positional arguments than fields, in model constructors, while the runtime __init__ silently ignores them. Checkers rejected these calls before this PR too, since they could not see the constructor at all. Model(**some_dict) type-checks before and after. No checker-visible signature can say "these fields, plus anything else" without giving up the keyword checking that commit 2 adds, so this is left in place.

Not changed: passing raw=True positionally still gets the Union, as today. _get still returns [] when a response cannot be decoded or lacks the results key, even for methods annotated to return a single model. That is existing runtime behavior.

Test plan

  • The consumer fails without these changes. On master, make static's check reports import-untyped for every massive import. Run against the master sources with the marker present, it reports 33 errors in the consumer: missing re-exports, every union-typed call, each wrong hint, the str CUSIP, and the model constructors. The errors drop with each commit, to 0 on the last one.
  • Without py.typed: with the marker removed from the final tree, the check reports import-untyped.
  • Export check: test_exports.py passes, and fails when massive/__init__.py defines an __all__ that drops names.
  • Backward compatibility: a file of caller code that type-checks on master was run through mypy and pyright on this branch. Every common pattern still checks; only the cases listed above are flagged.
  • Built wheel: the wheel contains massive/py.typed. Installed with mypy in clean environments on Python 3.10 and 3.14, it passes mypy --strict on the consumer.
  • pyright: reports no errors on the consumer. On massive/rest, the overloads add no new pyright errors.
  • make test, make style and make static pass on Python 3.10 and 3.14.

Independent of the other PRs.

@coderabbitai

coderabbitai Bot commented Oct 24, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@chipkent
chipkent force-pushed the feat/typing branch 2 times, most recently from 7773e8d to 185c717 Compare October 24, 2026 04:22
@chipkent chipkent changed the title feat: ship py.typed and correct type hints feat: ship py.typed, with precise REST return types and corrected hints Oct 24, 2026
Several hints describe values the API never sends or returns:

- CryptoQuote.bid_price, ask_price and ask_size were int; XQ sends
  fractional numbers ("bp": 33052.79, "as": 0.601). They are float.
- IndexValue.timestamp was str; V sends integer Unix milliseconds.
- list_tickers(cusip=...) accepted only int. The API treats a CUSIP as a
  string: it is alphanumeric, and leading zeros matter ("037833100").
  The hint now accepts str as well as int, so existing callers keep
  type-checking, and the docstring recommends a string.
- get_previous_close_agg() was annotated as returning one
  PreviousCloseAgg, but it returns the "results" array as a list, as the
  existing test shows.
- list_treasury_yields(), list_short_interest(), list_short_volume() and
  list_ipos() were annotated as returning a List, but like every other
  list_* method they return a paginating iterator.

Hints that are imprecise but still admit every value the API sends, such
as integer timestamps hinted as float, are left alone: narrowing them
would gain little and could break callers that type-check today.

Only annotations and docstrings change; runtime behaviour is the same.
@modelclass builds an __init__ that accepts every field positionally or
by keyword, but type checkers could not see it: a call such as
CryptoQuote(pair="BTC-USD", bid_price=1.5) was reported as
"Unexpected keyword argument" for every field.

Mark modelclass with dataclass_transform (PEP 681) so checkers derive
the constructor from the field annotations. The decorator is imported
only under TYPE_CHECKING, from typing on 3.11+ and from
typing_extensions on 3.10, and is a no-op at runtime, so there is no new
runtime dependency and Python 3.10 keeps working.
Every REST method is annotated as returning Union[Model, HTTPResponse]
(or Union[Iterator[Model], HTTPResponse]), because raw=True returns the
undecoded urllib3 response. Once the package is checked as typed, that
union makes ordinary code fail, because the checker cannot rule out
HTTPResponse:

    for t in client.list_tickers():
        t.ticker                        # "HTTPResponse" has no attribute "ticker"
    client.get_previous_close_agg("AAPL")[0]   # not indexable

Add three overloads to each of the 92 methods that take raw:

- raw: Literal[False] = False returns the model type, so the default
  call is precise.
- *, raw: Literal[True] returns HTTPResponse. raw is keyword-only in
  this overload because it follows parameters with defaults; a
  Literal[True] with a default would overlap the first overload, which
  pyright reports.
- raw: bool = False returns the full union, so a non-literal flag, or
  raw passed positionally, type-checks exactly as before.

The overloads copy each implementation's signature and differ only in
the raw parameter and the return type. They are declarations only: the
implementations, and the signatures that _get_params inspects at
runtime, are unchanged.

AGENTS.md now describes the pattern for new endpoints.
Without the PEP 561 marker, mypy skips the installed package and reports
import-untyped, and strict users must set ignore_missing_imports, which
turns every massive type into Any.

- Add massive/py.typed. The existing packages entry in pyproject.toml
  already includes it in the wheel and sdist.
- Re-export RESTClient and WebSocketClient explicitly ("X as X"), so
  "from massive import RESTClient" passes --strict, which does not treat
  plain imports in a typed package as re-exports. This adds no __all__,
  so "from massive import *" exports exactly the names it did before.
- Add test_rest/test_exports.py, which fails if any package stops
  exporting, through "from <package> import *", a name it exported
  before this change.
- Add test_rest/typing_consumer.py, a small strict user of the public
  API that is type-checked but never run. It iterates list_* results,
  indexes get_* lists, checks that raw=True gives HTTPResponse, builds
  models with keyword arguments, pins each corrected hint with
  assert_type, and keeps a sample of code written for earlier releases
  (isinstance narrowing, an int CUSIP) to show it still type-checks.
- make static checks it with mypy --strict from inside test_rest/, so
  massive resolves to the installed package as it does for users. It
  fails if py.typed goes missing or any of the above regresses.
@chipkent

Copy link
Copy Markdown
Owner Author

Superseded by smaller PRs split by content: #14, #15, #16, #17. Closing this preview; the replacements carry the same changes, opt-in where behaviour changes.

@chipkent chipkent closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant