Skip to content

1.1.0: audit follow-ups (gallery repeats, tutorial coverage, native usage) #284

Closed

Description

@jeremymanning

Tracking issue for the three audits run on 2026-09-05 before the 1.1.0 re-cut (repeats in the gallery, tutorial coverage of the public API, native-hypertools usage). Everything here ships in 1.1.0. Constraint: the five launch tutorials the Bluesky thread links (market_sectors, weather_decades, painting_embeddings, conversation_shape, morph_shapes_zoo) keep their pages and clips.

A. Gallery repeats and misdescriptions (examples/)

  • Delete plot_shape_morph.py (strict subset of animate_morph_zoo.py); fold the "all seven shapes" note into animate_morph_zoo; retarget animate_surface_morph.py's reference
  • Merge plot_TSNE.py + plot_UMAP.py + plot_digits.py into one example
  • Merge plot_PPCA.py into plot_missing_data.py
  • Merge animate_MDS.py into animate.py
  • Merge chemtrails.py + precog.py into one trails page (keep animate_trails_mix.py)
  • Merge plot_clusters.py/2/3 into one three-cell example
  • Fix misdescribed docstrings: plot_story_trajectories.py (execute the pipeline for real), plot_hue.py (categorical hue), animate_surface_morph.py (changelog prose), save_movie.py (hidden align=), plot_datasets_tour.py ("five vs four")
  • 'hyper' -> 'HyperAlign' in analyze.py, plot_align.py
  • Gallery ordering (within_subsection_order) and the "Story trajectories" entry in docs/tutorials.rst
  • Retitle conversation_trajectories and hugging_face_embeddings so they read as distinct from their siblings

B. Native usage rewrites

  • load_digits -> hyp.load('digits') (three examples); load_dataset -> hyp.load(...) (streaming_data, hugging_face_embeddings)
  • plot_procrustes.py: private procrustes() -> hyp.align(model='Procrustes')
  • plot_gensim_text.py: text2mat -> hyp.plot(texts, vectorizer=..., semantic=...)
  • plot_autoencoders.py: ax.scatter -> hyp.plot(..., ax=)
  • animate_surface_morph.py: drop the hand downsample; set_alpha -> alpha= (c4ed79c + morph alpha fix)
  • Remove dead sentence-transformers / pykalman import guards (six files) (examples done; tutorials in progress) now that extras install on demand
  • projectile_kalman.ipynb and stock_forecasting.ipynb hand ax.plot panels -> hyp.plot(..., reduce=None, ndims=2, ax=)
  • Library gap: isotropic (whole-array) option for Normalize (feefcbe: Normalize(mode='isotropic'), 21 tests), so the three morph files stop hand-rolling normalize_shape(); tests + docs
  • Tighten tests/test_examples_are_native.py DEFECT_MARKERS with the signatures the audit listed (with a per-file allowlist for deliberate ax= demos)

C. Coverage: new and rebuilt tutorials (each executed, in docs/tutorials.rst, install cell, gate entry)

  • hierarchy.ipynb: column-MultiIndex frame, per-sector traces + mean, names=, per-level alpha=, hierarchical predict= and return_model trace_data

  • animate_forecast.py + tutorial: animate=True, predict='Kalman', forecast_trail=True, forecast_hue/palette/fmt

  • io.ipynb: hyp.load of local files / URL / HF streaming / DataFrame, hyp.save round trips, save_path formats, HypertoolsIOError

  • pipelines.ipynb: return_model on every dispatcher, .transform, pipeline= replay, apply_model, supported_models(), bundle keys, Pipeline directly

  • manip.ipynb: hyp.manip Smooth/Resample/ZScore/Normalize and chains, animate='window' + focused=

  • Rebuild plot.ipynb: saving executed, animate=, names=, colorbar=, alpha= list, size/elev/azim, labels, font=, label_alpha=, backend='plotly', impute=, resample=, exception classes

  • Rebuild align.ipynb: Procrustes, SRM, align= inside plot/analyze, fitted aligner reuse, local image

  • Rebuild analyze.ipynb: call every stage it advertises, hypertools figures instead of seaborn heatmaps

  • Rebuild reduce.ipynb: describe curve drawn, UMAP, autoencoders, return_model

  • Extend stock_forecasting.ipynb (Chronos section) and projectile_kalman.ipynb (sklearn imputers vs Kalman)

  • Library gap found by the morph rewrite: alpha= does not reach the travelling morph cloud (matplotlib and plotly); fix + tests, then drop the set_alpha call in animate_surface_morph.py

D. Verification and release

  • Every tutorial executed with scripts/execute_tutorial.py; measured output cells recorded in the gate
  • Full suite (4118 passed), ruff, sphinx -W gallery build, Colab feature tour (both backends, fresh runtimes)
  • Gallery republished from the release commit (3a9b868); artifacts rebuilt; master pushed; - [ ] CI green; v1.1.0 moved; tag CI green; draft release updated

Found while fixing (all fixed for 1.1.0)

  • Glyph 8722 warning on caller-supplied ax= panels: tick labels under the sans-serif alias have no per-glyph fallback -> the ax= path sets the tick-label font list (217a2a6)
  • hyp.io.lsl_stream() left its inlet open -> LSLStream destroys it on close/with/GC/exit (61f3e7e)
  • PPCA divide by zero encountered in log on wide data -> slogdet (54b4b2e)
  • hyp.load(DataFrame) raised TypeError -> passthrough for loaded data (a8d073a)
  • hyp.plot(x, pipeline=<hand-built Pipeline with raw sklearn stages>) raised -> fit-or-transform replay of raw steps, Pipeline.is_fitted (3340595)
  • DEFECT_MARKERS tightened with the audit signatures (6b6bcbc); not added: weather's trailing rolling(12).mean() (hypertools Smooth is centred; visual rewrites deferred to 1.2 per maintainer)

Also fixed on the way (not in the original audit): HF vectorizers crashing with the default semantic= (#285, 1b34b1d), pipeline replay of hand-built Pipelines (3340595), LSL inlet destroy (bb07321), docs-gate literal for the market notebook (d724181), gensim example title overline, stale gallery output pages.

Activity

  1. jeremymanning commented on Sep 6, 2026

    @jeremymanning
    MemberAuthor

    Verification pass before the 1.1.0 re-cut (2026-09-06, branch fix/1.1-release-review, PR #286).

    Every checkbox in sections A, B and C was checked against the tree by running the examples and reading the notebooks, not by trusting the ticks. All hold, with these corrections now on the branch:

    • The DEFECT_MARKERS scan covered only the six launch scripts and six launch notebooks, so load_digits, load_dataset(, text2mat( and the other signatures never scanned the files this issue rewrote. It now walks all 51 examples and every non-install code cell of all 25 tutorials, has the ax= marker with the counted per-file allowlist this issue described, and a planted-marker test proves it fires. The eight rebuilt tutorials now have output-cell gate entries.
    • align.ipynb had dropped its alignment illustration instead of localising it; the image is back with a relative path.
    • The 'hyper' alias is presented as deprecated in analyze.ipynb and plot.ipynb; manip.ipynb shows Normalize(mode='isotropic').
    • Stored outputs in three notebooks leaked the author's home directory; the executor now scrubs it and the notebooks were re-executed.

    Section D: master and the v1.1.0 tag both sit at 96ac8b7 with all 17 CI jobs green and a draft GitHub release attached. The unchecked sub-item stays open on purpose: PR #286 adds fixes after the tag, so the tag, the gallery manifest and the draft assets are re-cut from the merge commit (RELEASE_CHECKLIST.md now describes that re-cut).

    The suite is at 4900+ tests. The issue text's "4118 passed" was the count at the time it was written.

    Closes on merge of #286.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions