Skip to content

Fix 1.1 release audit findings and complete documentation follow-ups - #286

Open
jeremymanning wants to merge 228 commits into
masterfrom
fix/1.1-release-review
Open

jeremymanning wants to merge 228 commits into
masterfrom
fix/1.1-release-review

Conversation

@jeremymanning

@jeremymanning jeremymanning commented Sep 6, 2026 •

Copy link
Copy Markdown
Member

This PR fixes release-review regressions in forecasting, caching, feature preservation, fonts and plotting, and updates the tutorials and review tooling. Timestamped forecasts now use the training clock; Plotly 3D colors retain their hue under transparency; full-curve companion panels retain a moving current-time marker. All 25 tutorials have been re-executed.

Closes #284.
Closes #285.

The candidate is fa3e60e5670cabfc1cbbd282e122b9098cc6d1a3. Release publication, merging, tagging and public artifact updates remain subject to Jeremy's manual sign-off.

Findings fixed in the PR

Priority Finding Fix and evidence
High Forecast backtests reused an instance fitted on dataset 1 for dataset 2. On a sine series followed by a positive quadratic series, AutoRegressor() produced negative quadratic forecasts and MAE 2551.68, versus 0.00246 when passing the class. Deep-copy the model for each dataset; preserve caller state. Regression tests compare actual returned forecasts for class, instance, and dictionary forms.
High Scoring accepted previously fitted forecasters/imputers even when their learned state could contain the held-out values. Imputation scoring also fitted caller-owned instances. Require unfitted instances for scoring, copy them before fitting, and document the distinction from ordinary fitted-model replay. Real fitted/unfitted model tests cover both forms.
Medium Delay silently lost features when distinct pandas column labels had identical string representations (1 and '1'); a 2-column, 2-lag input produced only 2 output columns. Reject colliding labels with a renaming instruction; tests include mixed-type and duplicate labels.
Medium URL-cache temporary names used only the process ID. Threads caching the same URL collided: 82 of 100 concurrent writes failed with FileNotFoundError. Use a unique temporary file per write, clean it on error, and atomically replace the destination. Test 100 real writes across 12 threads and verify payload/metadata and cleanup.
Medium System-installed Noto Sans took precedence over the bundled Regular face, contradicting deterministic font selection and failing the existing font regression test on this machine. Register bundled faces ahead of equal-scoring system faces. A fresh interpreter with another real same-family font proves the bundled file wins.
Medium Optional dependency minimums allowed gensim 4.3 and scikit-image 0.22, predating NumPy 2 support despite the library requiring NumPy>=2. Raise floors to gensim>=4.4.0 and scikit-image>=0.23.2 in extras/dev/docs. Real minimum-version feature tests pass under NumPy 2.3.5.
Documentation Public plotting/predict/impute docstrings described shipped features as 1.2; dependency prose implied ARIMA imputation. Correct version labels and separate forecasting from imputation support.
Documentation The “convert now” forecast example still hand-wrote URL download/cache logic after the native cache landed. Use hyp.load(ARCHIVE, cache=True), regenerate and execute the tutorial. The tutorial and video are regenerated with the current code.
Tooling The browser verifier expected docs-notebooks/master, searched highlighted HTML for contiguous pip install, and demanded an autoplay call in deliberately paused Plotly animations. Validate versioned notebook links, rendered code text, loaded frames/play controls, and execute a real transition in Chromium. Allow evidence/build paths outside the checkout.

Source and regression-test map

  • Forecast/imputation ownership: hypertools/predict/backtest.py, hypertools/impute/backtest.py; tests/test_predict_backtest.py, tests/test_impute_backtest.py.
  • Cache atomicity: hypertools/io/sources.py; tests/test_load_url_cache.py.
  • Delay collisions: hypertools/manip/delay.py; tests/test_manip_delay.py.
  • Font precedence: hypertools/plot/fonts.py; tests/test_fonts_bold.py.
  • Dependency compatibility: pyproject.toml, docs/doc_requirements.txt; real minimum-version runs of tests/test_gensim_text.py and tests/test_density.py, plus packaging/optional-import checks.
  • Documentation: dispatcher docstrings, docs/optional_dependencies.rst, readme.md, CHANGELOG.md, examples/animate_forecast.py, and its executed tutorial notebook.
  • Browser verification: scripts/verify_docs_playwright.py.

Timestamp-aware forecast follow-ups

Forecast fitting accounts for time-like indexes, including irregular/out-of-order observations and different clocks across datasets. The default future interval is the median positive gap between sorted timestamps, with an explicit step override. GaussianProcess uses actual observation times; regular-grid models use a documented interpolation policy. Real-model regression checks cover unsigned and large integer timestamp arithmetic as well as forecast scoring and fitted-model reuse.

Notebook and visual review follow-ups

Finding Implemented resolution
Plotly legend/trajectory colors differ Uniform 3D RGBA alpha is transferred to native trace opacity with RGB colors, including frame updates. Opaque frames explicitly reset earlier transparency. Real Chrome pixel tests supplement color/alpha parity tests; nonuniform alpha retains its original semantics.
Colab plots disappear during Run all Each Plotly example retains a PNG preview and a standalone HTML artifact. A single final viewer opens one live iframe at a time; switching/closing disposes the previous iframe. Download payloads are prepared only on click, avoiding both cumulative live WebGL contexts and repeated multi-megabyte JavaScript payloads in saved cell outputs. Fresh Colab frontend verification remains a separate check.
Companion right marker freezes Full-curve visibility is separated from current-time marker position and hue. The example visits all 30 input rows over five seconds and labels its trailing smooth. Tests check real artists, dates, full trends, hues and backward seeking, including unequal frame/sample counts. Existing serial-clock tests remain in force.
Visual failures disappear from reports Attention tables and cross-run comparisons include visual verdicts and notes alongside automatic outcomes. Each rerun resets its visual verdict and receives a fresh execution identity.
Stale approvals/provenance Reports carry notebook, inventory and case-source hashes, execution IDs, actual package source, optional dependency versions, settings, binaries and structured warnings. Changed checks are distinguished from changed outcomes.
Animation exports falsely pass Every run uses fresh destinations. Image/movie decoders require multiple distinct frames and meaningful duration; single-frame GIF/APNG outputs fail. Export examples now have enough time and frames to demonstrate motion.
Nonportable downloads/playback Colab uses its file-download API; local Jupyter prepares a downloadable data link only on request. Tutorial MP4-producing cells embed movie bytes on Colab while retaining relative media for local Jupyter/Sphinx.
Incomplete feature/behavior coverage Tour expanded from 227 to 244 cases. Added multi-column series/date bounds, returned data/colors/metadata/panel bundles, fitted sklearn pipeline reuse, callback clocks and callable titles, CJK fonts, luminance controls, compressed/extensionless and legacy XLS input, smoothing warm-up, hosted fitted topic models and stronger hosted-data contracts. Every dispatcher family participates in registry coverage guards.
Optional-install claim lacked a real missing dependency scripts/verify_optional_install.py creates an isolated base-only environment, verifies the friendly disabled-autoinstall error, then genuinely installs the missing XLSX dependency and verifies a round trip. The working environment is never stripped of dependencies.
Tutorial installer inconsistencies All 25 tutorial installers require 1.1 or newer and preserve current local checkouts. Independent convokit/py7zr prerequisites still execute. Text tutorials demonstrate their stated on-demand policy; Kalman no longer requests the unrelated prediction extra. Successful prerequisite-only pip chatter is omitted from published outputs.
Executor skips configuration Configuration and the tagged candidate installer are separate. The executor skips only HyperTools installers, preserving other setup and work.
Installation failure evidence Colab candidate setup streams and retains its installer log; failure offers that log for download. Successful logs are included in the report ZIP. Local execution explicitly retains the selected checkout.
Native GUI case tests inline rendering Desktop interaction runs in a separate Qt process and leaves the window available. It is an explicit optional manual case, not an inline-rendering PASS.
Remote pickle trust Historical Drive/Dropbox pickle demonstrations require conspicuous, disabled-by-default trust opt-in. Built-in hosted fitted models retain their independent digest validation. No post-deserialization hash is presented as protection.
Documentation/maintenance Corrected 1D inspection text, synchronized inventory/markdown, removed generated literal branches, strengthened expected-error assertions, documented XLSX writing, corrected candidate-versus-wheel release checks, and documented behavior evidence routes. Setup/playback additions are checked against exact templates without increasing the existing example-logic size budgets.

Windows and lint follow-up

The previous head passed all eight Linux/macOS matrix jobs but failed two notebook tests on every Windows version because UTF-8 files were decoded as CP1252. Notebook generation, the embedded viewer/export helpers and tests now explicitly use UTF-8. A real Unicode Plotly HTML regression opens/closes the actual ipywidgets viewer; no mocks are used. Equivalent Colab availability imports in 14 tutorials and their template now satisfy Ruff. Repository-wide lint passes and CI provisions Ruff 0.15.20 explicitly.

Verification

Current candidate: fa3e60e5670cabfc1cbbd282e122b9098cc6d1a3.

  • Current local tour: 241 PASS, 3 SKIP, 0 FAIL/BLOCKED, all 244 cases. Reports record this exact commit and a clean tracked checkout. Source SHA-256: 10ff17a28db615994497a177dac75a58c2cb788f0a7b67fbdcbf41fb6535958b.
  • Current focused notebook/install/support/visual/tutorial checks: 364 passed, 8 skipped; full repository Ruff and git diff --check pass.
  • Opt-in end-to-end example smoke from an isolated checkout at this SHA: 6 passed, 338 deselected.
  • Actual macOS QtAgg interaction: PASS. A real window opened; mouse press/move/release rotated the plot; it closed successfully. Screenshots and event results are retained. Qt was installed only in an isolated verification environment.
  • Candidate wheel/sdist build and Twine metadata checks: PASS. Fresh wheel installation and pip check pass; public API/pipeline, three irregular-time forecasters, inferred future spacing, Matplotlib PNG and Plotly standalone HTML checks pass. Imports are from isolated site-packages, not the source checkout. Artifact digests, package versions, bundled fonts/config/licenses and exact candidate provenance are retained.
  • Prior implementation evidence: all 25 tutorials executed, full default suite 5,936 passed, 20 skipped, 2 deselected, both opt-in large-data tests passed separately, and a real missing-extra installation/round trip passed. The current equivalent tutorial import edits are covered by the focused checks above.
  • Prior browser/visual evidence: real Chrome color-pixel regressions and standalone animated HTML transitions passed. The companion movie has 30 H.264 frames over five seconds; its inspected middle frame shows both companion markers at sample 15 and the matching January 16 title. Local/standalone browser checks do not substitute for fresh Colab frontend acceptance.
  • The current executed tour is about 6 MB, retaining PNG previews rather than live Plotly MIME outputs. Automated reports preserve warnings and leave unreviewed visual cases unapproved.

Current hosted CI: run 34558379385 completed successfully: 16 successful jobs, zero failures, one expected skipped release-only gate. All 12 matrix jobs passed, including all four Windows versions. Ubuntu Python 3.12 additionally passed repository lint, pandas 3, coverage (91% total line coverage) and screenshot generation. Clean docs executed all 51 gallery examples and 323 doctests with zero failures. Dataset gate: 33 passed; live-source gate: 43 passed, 1 deselected. Fresh wheel and sdist install gates passed. PR CI does not replace master/tag release gates.

The final review packet is ready locally in notes/notebook_critical_review_2026-09-10/START_HERE_FINAL_REVIEW.md, with detailed CI evidence in ci_final_status.md. The cleared Colab notebook and executed local reference have identical cell sources. All 104 tracked package Python files match the built candidate wheel byte-for-byte. No package or test changes were needed during this final verification pass.

Local evidence: notes/colab/feature_tour_preflight_fa3e60e5/; detailed follow-up report: notes/notebook_critical_review_2026-09-10/windows_preflight_report.md. These are local review files, not published documentation. The cleared Colab upload copy is notes/colab/hypertools_1.1_candidate_fa3e60e5.ipynb.

Remaining release checks

  1. Run the pinned tour in a fresh Colab runtime. After Run all, inspect early previews, viewer switching/closing, legend/trajectory colors, companion playback/scrubbing/dates, movie playback and report/artifact downloads. Retain notebook/reports and record visual decisions.
  2. Review remaining manual cases; macOS Qt interaction is verified, while native Windows GUI and explicitly trusted remote-pickle demonstrations remain separate acceptance decisions.
  3. Obtain Jeremy's manual sign-off before merging, tagging or changing public release artifacts. From the approved final release commit, rebuild gallery/manifest and wheel/sdist, record digests, and require master and tag release-gate CI to pass. The current tag/draft/assets describe the older 96ac8b7f draft.
  4. Publish only with explicit approval, then verify the actual published wheel, public documentation and Colab links, and conda-forge follow-up.

@jeremymanning
jeremymanning marked this pull request as ready for review September 6, 2026 03:36
jeremymanning and others added 14 commits September 5, 2026 23:54
…coring loops

The ultrareview of PR #286 flagged both 'from .common import ...' lines
as loop-body imports with no circular-import reason (common.py does not
import backtest.py). Module-scope imports match every sibling module.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e, attribute the unscored warning, score ragged return_score input

Release-review findings on the 1.1 draft:
- resolve_metrics() now raises ValueError naming a metric listed twice
  (case-insensitively) instead of a TypeError deep in build_scores(),
  for both predict(holdout=) and impute(truth=).
- holdout=True with t=0 reports t=0 as the problem.
- the 'left N scored value(s) missing' warning uses external_stacklevel()
  like every other user-facing warning in these modules.
- align(return_score=True) works on ragged input that align() trims; the
  'before' score is computed on the row-trimmed input (documented).
Seven regression tests, each failing on the unfixed library.

Also: RELEASE_CHECKLIST.md rewritten for the 1.1.0 re-cut (tag exists as a
draft and must be moved; gallery namespace is republished wholesale; the
example smoke gate is a manual step), and the session note for this review.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…al observables; export HypertoolsOfflineError

- tests/predict/test_common.py: the all-identical-timestamps case is proven
  to come from the live _infer_step via the exception's own traceback and
  message, not a monkeypatched spy.
- tests/predict/test_predict_multiindex.py: grouping helpers are checked
  through their real outputs and the returned per-group models, not by
  wrapping them with observers.
- tests/test_names_display.py: notebook display timing is observed on a
  real in-process IPython.InteractiveShell with the json renderer captured
  through IPython's own capture_output; go.Figure.show is no longer patched.
- HypertoolsOfflineError is importable from hypertools and hypertools.io
  like the other three exceptions (public-API pin test updated).
- scripts/generate_baseline_screenshots.py points at the roadmap note that
  exists; trailing whitespace stripped from six tracked files so
  'git diff --check v1.0.0..HEAD' is clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…defect markers, allowlist the deliberate ax= demos, gate the rebuilt tutorials' output cells

Issue #284 claimed the DEFECT_MARKERS scan was tightened with a per-file
allowlist for deliberate ax= demos and that every tutorial's measured
output cells were recorded; the release review found the scan covered
only the six launch scripts and six launch notebooks.

- scan_for_defects(root) walks all 51 examples and every non-install
  code cell of all 25 tutorials (install cells hold the legitimate
  find_spec('hypertools') guard); a planted-marker test proves it.
- new ax= marker (hand-built axes, raw ax.plot/scatter, plt.subplots)
  with a counted DEFECT_ALLOWLIST so a new use in an allowlisted file
  still fails and a stale entry is reported.
- EXPECTED_VISIBLE_OUTPUTS gains the eight rebuilt tutorials; the
  ran-every-cell / right-cells / no-error-output tests cover all 14.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…le, colour, loader and text paths

Plot (22 findings, 53 regression tests):
- palette= lists shorter than the dataset count cycle again with no hue
  (1.0 behaviour); empty palettes raise ValueError, not StopIteration;
  per-dataset {category: color} dict entries merge by name; NaN in a
  continuous hue no longer poisons the colour range; legend_kwargs
  fontsize is honoured with font=; blend categories are RGB.
- dataset_fade= and on_frame artist mutations reach the LineCollections
  drawn under a continuous hue; loop=True accepts the documented
  2(n+1)-1 rotations list; companion= panels and {index} titles advance
  monotonically under order='serial'; window_bounds.start reflects the
  comet-head window; bad companion=/dataset_fade= values name the kwarg;
  a raising on_frame during .save() surfaces its own exception; a raising
  title= leaves no orphaned animation.
- title_wrap= applies to dynamic titles and keeps explicit newlines;
  plotly draws newline titles as line breaks and reserves top margin per
  title line and size; nested tuple labels= annotate; bare-string labels,
  bad label_anchor=, non-string title entries/callables, title_color vs
  title_kwargs colour conflicts and bad {index} formats raise clearly.

IO / tools (10 findings):
- load(offline=True) opens no connection: URLs skip the seaborn listing,
  the listing fetch has a timeout and a remembered failure, uncacheable
  sources raise HypertoolsOfflineError.
- yahoo: bars carry the exchange-local trading day (gmtoffset applied).
- synthetic datasets accept RandomState, Generator, SeedSequence and
  np.integer seeds everywhere; a reused SeedSequence is reproducible;
  n_datasets rejects non-integral values; streaming=True on a
  non-Hugging-Face source raises instead of returning everything.
- text2mat: a flat list of strings is one dataset (was [(N,d),(0,d),...]
  since 1.0), ragged nested lists work, mixed inputs raise; a dict
  semantic= spec with a gensim vectorizer warns and skips.
- text_windows accepts numpy integers.
- format_data warnings are attributed to the caller's line
  (external_stacklevel), so notebooks stop printing the library path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Manipulator.fit() and Aligner.fit() returned None, so
Smooth().fit(x).transform(y) and HyperAlign().fit(xs).transform(ys)
raised AttributeError; Imputer.fit() already returned self. Both now
return the fitted instance on every path (documented), with a chaining
test over Normalize/ZScore/Smooth/Resample/Delay, HyperAlign/Procrustes/
NullAlign and PPCA on real data.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- CHANGELOG: 'Fixed during the release review' subsection (every fix
  since the draft), predict(metrics=/per_column=/return_forecasts=),
  impute(return_imputed=), load(**source_kwargs) documented; the
  unsupported '138 examples executed' claim reworded to what the tests do.
- readme: dependency floors match pyproject exactly (pillow added);
  '1.0 API' wording updated for 1.x.
- api.rst: HypertoolsOfflineError and HypertoolsTrustError listed with
  their autosummary stubs.
- conf.py: gallery pages' furo view/edit links point at the source
  example under examples/ (auto_examples/ is gitignored, so they 404'd);
  stale chemtrails/precog comment fixed.
- tutorials.rst + market_sectors prose use model='HyperAlign' like the code.
- notebooks (sources only; re-executed separately): 'hyper' marked as a
  deprecated alias (analyze, plot); Normalize(mode='isotropic') section
  (manip); alignment illustration restored (align); legend= form for the
  hue demo (text); hyp.load('wikipedia:...') replaces the wikipedia-api
  cells (wikipedia_embeddings).
- scripts/execute_tutorial.py scrubs the executing user's home directory
  from stored outputs so notebooks stop leaking developer paths.
- tests/AGENTS.md no longer claims the hierarchy guide's doctests run in
  the suite; hypertools/io/lsl.py numpydoc underline fixed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… into the summary so numpydoc stops warning

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…gs from the 1.1 release review

- forecasters carry min_history (ARIMA from its order); fit raises a
  clear ValueError on a shorter history; the animated forecast schedule
  waits for enough revealed rows, so predict='ARIMA' (and model lists)
  no longer crash under animate= with a statsmodels IndexError.
- a datetime-like t= works inside hyp.plot; predict= collections work on
  MultiIndex frames; dated column-MultiIndex frames draw dates for every
  leaf under ndims=1; forecast_hue= is one value per dataset with a model
  collection; series-mode bundles match hyp.predict's shape.
- panels=: predict+truth in both panel_fit modes; shared mode keeps
  DataFrame index/column names and accepts 3-column frames; nested hue=/
  labels= narrow per panel; ndims>3 draws 3-D panels; save_path is
  normalised and validated up front; plotly panels use the one-shot
  display wrapper.
- ndims=1: per-column fmt lists; date-aware xlim on both backends; no
  'dataset 1' y label; a 3-D ax= with ndims<=2 raises; TimedeltaIndex is
  drawn in a readable unit.
- a trailing Smooth(center=False) that introduces NaN rows is reported
  as such with the min_periods=1 hint.
- docstrings: font= weights, HyperAnimation.drawn_extent/.save.
- CHANGELOG entries for all of the above.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d library; gate and test updates

analyze, plot, manip, align, text, wikipedia_embeddings, projectile_kalman
and conversation_trajectories re-executed with scripts/execute_tutorial.py
(home directory scrubbed from stored outputs; no /Users/ path remains in
any tutorial). manip's output-cell gate entry gains the new isotropic
Normalize cell; the nested-hue length test asserts the new message that
names the offending sub-list; tests/test_load_offline.py loses a trailing
blank line. Full suite before these two test updates: 5171 passed, 2 failed
(these two), 19 skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…xecuted notebook

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ation)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ates Windows' transient access-denied

Every Windows job on PR #286 (run 34014888620) failed on the same ten
tests: the widened native-usage scanner compared 'docs\tutorials\x'
against its forward-slash allowlist and roster, and the URL cache's
os.replace raised PermissionError (WinError 5) when twelve threads
replaced one entry at once. The scanner now reports POSIX-separated
relative paths on every host; the cache retries the rename briefly and
accepts a concurrent writer's identical file. macOS and Linux jobs were
green on the same commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (Windows paths, Windows os.replace, detached verification pipeline)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning

Copy link
Copy Markdown
Member Author

Second review pass on this branch (2026-09-06): the cloud ultrareview of the PR diff, then nine local reviewers over the full v1.0.0..HEAD diff (plot, io/tools/manip, predict/impute/align/core, packaging/CI, documentation) and item-by-item verification of #284 and #285. Every confirmed finding is fixed on the branch with a regression test; the fixes are listed in the CHANGELOG's "Fixed during the release review" subsection. The commits since the original report:

  • f116dd7 module-scope imports in the backtest loops (the ultrareview nit)
  • f1a1e09 duplicate metrics=, holdout=True, t=0 message, warning attribution, ragged return_score
  • e47968f three spy/fake-object tests rewritten against real observables; HypertoolsOfflineError exported
  • a59f2e2 native-usage scanner covers every example and tutorial, with the ax= allowlist and output-cell gates the issue described
  • ef887ca 32 plot/io/text fixes: offline=True was still opening connections, yahoo: dates east of UTC, synthetic seed types, short palette= lists (a 1.0 regression), dataset_fade= under a continuous hue, loop=True rotations, companion= under serial order, title_wrap on dynamic titles, plotly newline titles, text2mat on a flat list of strings
  • 1be634d fit() returns self on the manipulator and aligner bases
  • f0e56c9 documentation: changelog completeness and accuracy, readme floors, API exceptions, gallery edit links, tutorial prose and code
  • 384fe99 ARIMA under animate= (per-model min_history), datetime t= inside plot, predict= collections on MultiIndex frames, panels= with truth=/labels/nested hue, series-mode fmt=/xlim=/y label, TimedeltaIndex
  • 00e5a00, f775b37 eight tutorials re-executed on the reviewed library, home paths scrubbed from stored outputs
  • 9340300 Windows: POSIX paths in the scanner, retrying os.replace in the URL cache
  • 63aa9cf, d20fdde session and project notes

Local verification on the final library: full suite 5171 passed / 0 failed (after the two stale expectations were updated), ruff clean, sphinx -W over the full gallery with zero warnings, the example smoke gate 344 passed. PR CI on the branch head d20fdde: every job green (release-gate skipped by design on a branch).

After merge the release is re-cut from the merge commit; RELEASE_CHECKLIST.md now describes moving the draft tag, republishing the gallery namespace and replacing the draft release assets.

jeremymanning and others added 10 commits September 6, 2026 03:39
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The matplotlib backend draws 's--' as a smoothed line artist (which
carries the legend label) plus a markers-only artist at the raw sample
points labelled _nolegend_, so the legend handle showed only the dashes
(reported from the 1.1 feature tour, section 9.2). The line artist now
carries the marker with markevery=[]: its legend handle shows marker and
line while it still draws no markers along the interpolated vertices.
Pixel-level regression test in tests/test_plot_fmt_split_legend.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…data)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…it under return_model, quiet third-party reducer warnings

Reported from the 1.1 feature tour (section 9.8): a three-panel grid came
out 2x2 with a hole and, in a 9x3.2 in figure, each square 3-D axes
shrank to the short cell height; the cell also printed fifteen warnings.

- panels=True picks the grid from the figure's aspect ratio and prefers
  a grid with no spare cell (three panels form a row; four 2x2; six 2x3;
  five still 2x3 with one hidden cell). Explicit grids are unchanged.
- return_model=True reuses the pipeline analyze() fitted for the figure
  (the cluster stage, which runs on the reduced scores, is appended as a
  fitted step) instead of refitting every stage, so a UMAP/Isomap plot,
  and every panel grid built with it, fits and warns once.
- a seeded UMAP passes the n_jobs=1 umap forces anyway, so umap stops
  warning about a seed hypertools injected; a caller's own n_jobs= still
  reaches umap (and its warning).
- Isomap fits silence scipy's SparseEfficiencyWarning burst from
  sklearn's internal graph completion; sklearn's own connected-components
  data warning still reaches the user.
Seventeen new tests; two older panel tests updated to the new grid rule
(the hidden-spare property now covered with five panels).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pan the whole scene

- The truth curve keeps marker='o' with markevery=[] so its 'truth'
  legend entry is line + marker, no longer identical to the observed
  trace's entry (feature tour 9.11).
- New density.scene_bounds_2d: every 2-D KDE grid covers all datasets'
  padded bounds plus the unit frame square, on both backends, so a wide
  flat cloud's glow fades out instead of stopping in a hard band inside
  the frame (feature tour 9.14).
- Tests for both; projectile_kalman and stock_forecasting tutorials
  re-executed (truth= legends), which also refreshed the stock snapshot
  CSVs by one trading day as the notebook is designed to.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…+ ax=<cell>, per-panel colorbars on matplotlib

- plotly_backend.transplant_panel moves a drawn single-axes figure into a
  make_subplots cell whole: traces, the 2-D axis layout (unit frame,
  hidden ticks, DataFrame-column labels, or axis_scale='data' axes), the
  frame square and labels= annotations re-referenced to the cell, its own
  legend (plotly multiple legends) and its own colorbar beside the cell;
  3-D cells back the camera off so the cube stays inside a narrow cell.
- make_panel_grid reserves a gutter beside every cell for those legends
  and colorbars (default-sized grids widen by it; explicit size= verbatim).
- hyp.subplots(backend='plotly') returns the grid figure plus PlotlyCell
  handles that hyp.plot(..., ax=cell) draws into (title= becomes the cell
  title); several cell calls display the grid once per notebook cell.
- matplotlib: a colorbar drawn into a caller-supplied ax= (every panels=
  cell, every hyp.subplots axes) uses fig.colorbar(ax=...) instead of the
  figure-widening placement, so panels no longer stack their colorbars
  over the last panel or trip tight_layout warnings.
- Tests: tests/test_subplots_plotly.py (new), 11 more in
  tests/test_plot_panels.py. CHANGELOG entries; RELEASE_CHECKLIST.md
  brought up to date (suite size, Colab tour smoke, announce step).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…id resolution, sparse ARIMA orders, plotly panel titles/fonts/left colorbars

- panels= on matplotlib under an active plotly preference built its grid
  with subplots()' new backend='auto' default; the grid is now explicitly
  matplotlib.
- 2-D density grids pad by four kernel widths past the data instead of
  spanning the whole scene: the glow still fades out inside the grid, and
  a small cloud beside a 10,000x larger one keeps its resolution (the
  scene-wide grid sampled it to all zeros).
- ARIMA.min_history_for accepts statsmodels' sparse lag orders
  (order=([1, 3], 0, 0)), counting the highest lag, as the fitter does.
- plotly panels: titles go through the single-axes title path (newlines,
  title_wrap=, title_kwargs=) and become cell annotations; the panel's
  font= travels with its legend and becomes the grid default; a
  location='left' colorbar stays on the cell's left; drawing into a 3-D
  cell twice keeps the earlier labels= annotations.
- Tests for each; a rendered-grid test now checks ink in both cells.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ly cell title placement/replacement, per-cell fonts

- predict_new only holds the refit path to the fit-time minimum history;
  a model with an applier (ARIMA) reuses its learned parameters on any
  context the new data offers.
- plotly cells: the title annotation follows the title's own x/y/anchors
  mapped into the cell, is replaced (not stacked) when the cell is drawn
  into again, and reserves the multi-line top margin the single-axes
  path computed; the panel's font= is materialized on the cell's legend,
  title, axis titles/ticks and colorbar under explicit overrides.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
jeremymanning and others added 30 commits October 8, 2026 14:33
… piling labels

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ew page

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s its window

Recording the feature tour's GUI-native case in a real Qt window showed a
hover label cut off at the left edge: it was always placed up-left of the
point. It now faces the axes centre (test: tests/test_explore_label_placement.py,
fails on the old placement for the left edge).

The tour's GUI script called hyp.plot(..., show=False) then plt.show(), but
show=False deregisters the figure by design, so no window ever opened (the case
is skipped headless, so this went unnoticed). It now uses show=True.
scripts/record_gui_native_screencast.py drives that window with real Qt events
and grabs each frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t doesn't throw

Colab attaches its interactive-table script to every displayed DataFrame;
a second one in the same output redeclares const buttonEl and throws a
SyntaxError in the console (IMP-score and the results summary).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…writes frames.json

Closing the last window could end the Qt loop before the frame index was
written. Pointer positions let a caption pass draw the mouse, which a window
grab does not include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read the Docs writes HTML to $READTHEDOCS_OUTPUT/html, but post_build looked
for _images directly under $READTHEDOCS_OUTPUT, so the variable never matched;
RTD builds only worked through the <checkout>/_readthedocs/html path guess.
Found by building the docs the RTD way into another output directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…utputs, dark theme, phone width, console errors)

- gallery cards: the title is a visible link to the example page and the
  whole card is clickable (custom.css hid the only such link); thumbnail
  still opens the notebook on Colab; one theme-coloured tooltip, not two
- each animation is scraped once (docs/_gallery_scrapers.py): animate and
  animate_trails showed 5 videos for 2 plots, five other pages 2 for 1
- plotly figures are no longer repeated through capture_repr
- card/tooltip/link-hover colours follow furo's light and dark themes
- plotly figures refit to the column on narrow screens
- post_build strips plotly's dead CDN module import and MathJax 2 script
  from notebook pages, and RST markup from gallery tooltips
- hide sphinx_gallery config comments; filter the explore-mode backend
  warning in the gallery build only
- API page titles keep their case under the lowercase heading style

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… an ERROR box)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pages

Returns blocks of predict/impute/manip/apply_model were wrapped prose that
numpydoc parsed line by line as item names; they are now a named item with
the prose as its description. Inline literals are out of parameter-name
lines and bold spans (plot, predict, impute), and backticks no longer sit
against ~ or a trailing letter (lsl, text_windows). The slow_warning_seconds
sentinel is an instance of a private class whose repr is <default>, so the
plot signature no longer shows <object object>; identity checks unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/scan_api_markup_leaks.py reads built HTML and reports literal
backticks, :role: syntax and ** pairs in body text, numpydoc items that are
wrapped prose, and opaque signature defaults. tests/test_api_docs_markup.py
builds docs/api.rst with autodoc + numpydoc + autosummary (about 7 s) and
requires a clean scan, plus a control build proving each kind is detected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ibutes table per class

- post_build moves each tutorial's leading install cell below its title, in
  a collapsed <details>; nbsphinx rendered 15 lines of setup code above the
  heading on 23 pages. Notebooks are unchanged.
- numpydoc_show_class_members = False: the autosummary class stubs already
  list methods and attributes, so every class page showed both tables twice.
- The FrameContext and LSLStream stubs are committed as autosummary writes
  them, so a docs build no longer leaves them modified.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…etched to 400)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…notes; bring the notes up to date

The release-notes draft now links CHANGELOG.md near the top and wherever it
cites a changelog section, and covers the review-period changes it predated:
describe's mean-of-curves average and show=False figure, animated truth
reveal and forecast anchoring, default forecast group colours, data-scale
axes without gridlines, the raised dependency floors (and the corrected
scikit-image floor), and a summary of the plotly, surface, animation,
forecasting and explore fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne works on hyp.Pipeline; isotropic docstring

describe(): a configured instance (PCA(n_components=2)), a dict spec that
pins n_components, and a fitted reducer were all evaluated at that one
dimensionality for every sweep point. The sweep's dimensionality now wins:
unfitted instances are cloned per point with n_components set, pinned
kwargs are dropped, and a fitted model or an instance with no n_components
raises a ValueError saying what to pass instead.

Pipeline: __sklearn_clone__ returns an unfitted, independent copy (nested
and dispatcher pipelines included); set_params(steps=...) resolves and
names steps like the constructor. apply_model(stack=False) with a Pipeline
now returns one fitted pipeline per dataset.

Normalize(mode='isotropic'): the scale depends on orientation, so a rotated
copy generally gets a different scalar (docstring only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release red-team 2026-10-09. (1) build_scores picked attrs['best'] by lowest score regardless of coverage, so a model that left values NaN could win on the easy ones; only models with unscored == 0 are ranked now, incomplete rows are listed in attrs['incomplete'], and best/beats_baseline are None when no model (or the baseline) is complete. (2) hyp.impute(truth=, mask=) compared labelled frames by position; they are aligned to the data by label and mismatched or ambiguous labels raise. Backtests pair forecast and held-out columns by label too. Docstrings, CHANGELOG and the stock tutorial verdict cell updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lazy_import returned whatever imported. With plotly 5.24.1 beside kaleido
1.3.0 (plotly>=6.1.1 is declared) a plotly static export failed with
plotly's 'requires the kaleido package' although kaleido was installed
(release red-team review 2026-10-09).

Every requirement of the extra is now checked against the installed
metadata, once per process. A distribution below its floor is upgraded
through the same pip call while it is not imported yet; if it is already
imported, or installation is off, ImportError names the installed version,
the requirement and the command. resolve_backend asks lazy_import before
anything imports plotly, and backend='auto' falls back to matplotlib with
a warning. The low-level render check reports a plotly/kaleido mismatch
as such.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mes a stale candidate notebook

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/test_datatype_gate.py refuses direct isinstance(x, pd.DataFrame/Series)
checks outside the coercion layer; the new label-alignment code had four.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
stock_forecasting (its verdict cell now handles a None verdict), projectile_kalman,
reduce and pipelines; outputs only, plus the stock caches' rolling window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nswers

Kaleido's own timeout does not always fire: on a hosted ubuntu job the
NO_BROWSER test sat for the caller's whole 120 s and failed with
TimeoutExpired (the same test passed in seconds on 15 other jobs). The
script now ends the export at its own deadline with NO_BROWSER_EXIT, since a
browser that never answers is no usable browser. New test drives it with a
real executable that stays alive and answers nothing.

Also records the datatype-gate lesson.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1.1.0 draft was dated 2026-09-04; the release is re-cut from this commit
after the pre-publication review. Checklist step 2: notebooks already in
release form (migrator no-op, 76 of 76), README images on the v1.1.0 tag (8),
release-file gates 23 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hirtyEight loads

A macOS runner could not resolve api.kaggle.com and the test failed the
job. It now uses tests/_netskip.skip_on_transient_network: an outage skips,
a defect still fails, and HYPERTOOLS_REQUIRE_LIVE_SOURCES=1 (the
live-source-gate job) skips nothing. Checked with a real connection failure
through a dead proxy: skipped by default, failed under the gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NO_BROWSER test timed out twice on ubuntu 3.11 with nothing to go on:
TimeoutExpired discarded the script's output. The script now prints its
stage with a timestamp and dumps every thread's stack if it stalls, and the
test puts that output in the failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ubuntu 3.11 timeout was not in the export at all. The stack dump shows
the main thread in threading._shutdown: the script had already caught the
browser failure and called sys.exit, and interpreter shutdown was waiting on
choreographer's non-daemon worker threads, which a failed browser launch
leaves blocked on a queue and a pipe read. They never end on Linux with
Python 3.11. The script now flushes and leaves with os._exit once the
verdict is decided; the watchdog writes with os.write for the same reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant