Skip to content

waterdata: get_stats_data preserve geometry across continuation pages #255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thodson-usgs merged 8 commits into from
May 5, 2026
+1 −1
Merged

waterdata: get_stats_data preserve geometry across continuation pages #255

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0ad568c
waterdata: tighten stats and OGC pagination — geometry, KeyError, non…
thodson-usgs May 3, 2026
0fa758e
Unify status-check sites behind a small _raise_if_not_ok helper
thodson-usgs May 3, 2026
af00705
Harden _error_body and add geopandas continuation regression test
thodson-usgs May 4, 2026
dd5e00e
Merge remote-tracking branch 'origin/main' into fix/stats-pagination
thodson-usgs May 4, 2026
1bfea67
Address Copilot review on PR #255
thodson-usgs May 5, 2026
22c60de
Drop defensive fixes; keep only the real geometry bug
thodson-usgs May 5, 2026
949b4eb
Merge remote-tracking branch 'upstream/main' into fix/stats-pagination
thodson-usgs May 5, 2026
b7285e6
Drop trivial regression test from stats-pagination PR
thodson-usgs May 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dataretrieval/waterdata/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1102,7 +1102,7 @@ def get_stats_data(
headers=headers,
)
body = resp.json()
all_dfs.append(_handle_stats_nesting(body, geopd=False))
all_dfs.append(_handle_stats_nesting(body, geopd=GEOPANDAS))
Copy link
Copy Markdown
Collaborator Author

@thodson-usgs thodson-usgs May 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already covered: test_get_stats_data_preserves_geometry_across_pages (added in this PR) explicitly mocks two pages with point geometries, runs them through get_stats_data with GEOPANDAS=True, and asserts the result is a gpd.GeoDataFrame with df.geometry.notna().all() across all 2 sites. Also verified against the live API: with state=US:42, parameter=00060, page_size=1, the buggy code leaves 0/1890 page-2 rows with geometry while the fix preserves 1890/1890.

next_token = body["next"]
except Exception: # noqa: BLE001
error_text = _error_body(resp)
Expand Down
Loading