Skip to content

Fix get_results: preserve user-supplied WQX3.0 dataProfile#250

Merged
thodson-usgs merged 4 commits intoDOI-USGS:mainfrom
thodson-usgs:fix/wqp-data-profile-clobber
May 4, 2026
Merged

Fix get_results: preserve user-supplied WQX3.0 dataProfile#250
thodson-usgs merged 4 commits intoDOI-USGS:mainfrom
thodson-usgs:fix/wqp-data-profile-clobber

Conversation

@thodson-usgs
Copy link
Copy Markdown
Collaborator

@thodson-usgs thodson-usgs commented May 3, 2026

Summary

The validation block silently overwrote any user-supplied WQX3.0 dataProfile because the else clause attached to the outer if "dataProfile" in kwargs and ... not in result_profiles_wqx3 and fired whenever that compound condition was False — including the case where the user passed a valid profile like "narrow" or "basicPhysChem". The documented example dataProfile="narrow" therefore did not actually request the narrow profile.

This PR restructures the WQX3.0 branch so the default (fullPhysChem) is only applied when the user did not set dataProfile at all.

It also raises ValueError rather than TypeError for invalid profile values (these are bad values, not bad types) and calls the constructor with a single formatted message — the previous code passed two separate strings, producing a tuple-args exception that printed as ('msg1', 'msg2').

Test plan

  • One new regression test in tests/wqp_test.pytest_get_results_wqx3_preserves_user_dataProfile — confirms a valid user-supplied WQX3.0 profile ("narrow") is preserved end-to-end through the request URL. The accompanying TypeError -> ValueError shape fix is documented by the diff and not separately unit-tested.
  • All tests/wqp_test.py (12 tests) pass.
  • Full local test suite passes (with API_USGS_PAT set).

Related PRs

Other open PRs in this bug-review series that touch dataretrieval/wqp.py (different functions, no functional conflicts):

thodson-usgs and others added 2 commits May 3, 2026 12:58
The validation block silently overwrote any user-supplied
WQX3.0 dataProfile because the `else` clause attached to the *outer*
`if "dataProfile" in kwargs and ... not in result_profiles_wqx3` and
fired whenever that compound condition was False — including the case
where the user passed a *valid* profile like "narrow" or "basicPhysChem".
The documented example `dataProfile="narrow"` therefore did not actually
request the narrow profile.

Restructure the WQX3.0 branch so the default (`fullPhysChem`) is only
applied when the user did *not* set `dataProfile` at all. Also raise
`ValueError` rather than `TypeError` for invalid profile values (these
are bad values, not bad types) and call the constructor with a single
formatted message — the previous code passed two separate strings,
producing a tuple-args exception.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Selects valid_profiles + url + kind once per branch, then validates and
applies the WQX3.0 default in straight-line code. Replaces the
3-deep nesting in the wqx3 branch and the duplicated `dataProfile in
kwargs and ... not in profiles` check across both branches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request May 4, 2026
The simplify pass added `TypeError(msg, msg)` -> `ValueError(single_msg)`
fixes to `get_results`'s two `dataProfile` validators. PR DOI-USGS#250 ("Fix
get_results: preserve user-supplied WQX3.0 dataProfile") restructures
the same code more thoroughly and addresses the same exception-shape
bug as a side effect, so this PR should not duplicate that work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes dataretrieval.wqp.get_results() so WQX3.0 requests preserve a caller-provided dataProfile instead of always falling back to the default profile. It narrows the change to the WQP results path and adds a regression test around the request URL that gets sent.

Changes:

  • Refactors get_results() profile selection/validation so WQX3.0 only defaults to fullPhysChem when dataProfile was omitted.
  • Changes invalid dataProfile handling to raise ValueError with a single formatted message.
  • Adds a regression test confirming dataProfile="narrow" is preserved in the outbound WQX3.0 request.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
dataretrieval/wqp.py Refactors get_results() legacy/WQX3.0 profile handling and exception construction.
tests/wqp_test.py Adds a regression test covering preservation of a valid user-supplied WQX3.0 dataProfile.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dataretrieval/wqp.py Outdated
f"Valid options are {result_profiles_legacy}.",
)

if legacy:
Comment thread dataretrieval/wqp.py
Comment on lines +140 to +142
raise ValueError(
f"dataProfile {profile!r} is not a valid {kind} profile. "
f"Valid options are {valid_profiles}."
Per copilot review on PR DOI-USGS#250. Other helpers (what_sites, etc.) use
'legacy is True' to gate URL selection, so a non-bool value like 1 or
'false' would otherwise route differently in get_results vs the rest
of the module.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@thodson-usgs
Copy link
Copy Markdown
Collaborator Author

@copilot-pull-request-reviewer thanks for the review.

Accepted: if legacy is True: for parity — kept this consistent with the rest of wqp.py (e.g., what_sites line 207 uses the same idiom). Pushed in 963342d.

Declining: regression test for invalid dataProfile — earlier in this review series I trimmed the two pytest.raises(ValueError, match=...) tests for invalid profiles per maintainer guidance to omit tests that just re-state a one-line type/message change (the fix is observable from the diff). The TypeError(msg, msg) -> ValueError(single_msg) change is structural, but the test would just call the function and assert the exception type. The "preserve user-supplied profile" test (which exercises the actual silent-clobber bug) remains.

@thodson-usgs thodson-usgs marked this pull request as ready for review May 4, 2026 15:23
@thodson-usgs thodson-usgs merged commit 98b3057 into DOI-USGS:main May 4, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants