chore: cap static analytics detail exports at top-n rows (#4924) - #4932
Draft
frano-m wants to merge 1 commit into
Draft
chore: cap static analytics detail exports at top-n rows (#4924)#4932frano-m wants to merge 1 commit into
frano-m wants to merge 1 commit into
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Caps the size of selected static analytics JSON exports to reduce page-load payload and monthly regeneration diffs, while keeping uncapped exports that the template renders in full.
Changes:
- Added an optional
max_rowsparameter toexport_df_as_json()to export only the top-N rows (with a console log when truncation happens). - Introduced
MAX_TABLE_EXPORT_ROWS = 100and applied it topageviews.json,outbound_links.json, andfilter_selected.json. - Annotated the README output listing to document which exports are capped.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| analytics/analytics_package/analytics/static_site/export.py | Adds a configurable top-N row cap and applies it to the three sliced table exports. |
| analytics/analytics_package/analytics/static_site/README.md | Documents the three capped export files in the output tree. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4924
Independent of #4930 / #4931 — this is Python-only and branches off
main.What changed
export_df_as_json()gains an optionalmax_rows; when the frame exceeds it, the top rows are kept and the truncation is logged (Capped pageviews.json at top 100 of 1810 rows) so the cap is never silent.MAX_TABLE_EXPORT_ROWS = 100is applied to the three exports the template renders as a fixed slice:pageviews.jsonslice(0, 20)outbound_links.jsonslice(0, 20)filter_selected.jsonslice(0, 30)Left uncapped on purpose: event detail tables, access requests and search queries — the template renders every row of those (behind a "Show all N rows" toggle, or with no slice at all for search), so capping them would silently drop visible data. The constant's comment records both halves of that rule, and the README's output listing now annotates the three capped files.
Measured on a 1,810-row frame: 120KB → 7KB (~95% smaller), which is also thousands of fewer diff lines per monthly regen.
Why top-N is safe here
The cap keeps whichever rows come first, so it depends on the frames being sorted. That holds by construction, not by luck: all three producers pass explicit
sort_resultsintoget_one_period_change_df, which does a stable descendingsort_valueson the current-period metric (report_elements.py—get_page_views_change,get_outbound_links_change,get_index_filter_selected_change), andfetch.py's later exclude-pages / suspicious-path filtering preserves order. The deployed JSONs are fully sorted descending.export_df_as_json's docstring states the expectation for future callers.Nothing consumes the full lists: within the template these three datasets are only ever sliced for their tables plus an emptiness check — no totals, counts or reductions are derived from them — and no other Python, notebook or TypeScript in the repo reads them. So no displayed aggregate can be skewed by the cap.
How verified
Manual local checks only — this PR adds no automated tests. The repo has no Python test harness: no
test_*.pyfiles,pytestis not a declared dependency, and no workflow runs Python tests (CI covers the TypeScript app only). Flagging that explicitly because an earlier version of this description said "unit-tested", which wrongly implied in-repo coverage.What I actually ran: an ad-hoc script in a throwaway virtualenv that imports
export_df_as_jsonand exercises it against a synthetic 1,810-row frame — capped to exactly the top 100 with order preserved and NaN changes still nulled; a below-cap frame exported untouched; nomax_rowsstill exports every row (the event-detail path);Noneand empty frames still emit[]. That script was not committed.Adding a real pytest suite for this module and wiring it into CI is worth doing, but it would introduce a Python test harness this repo doesn't have yet — happy to do that here or as a follow-up ticket, whichever is preferred.
🤖 Generated with Claude Code