Repository navigation
check_external_links: report permanently redirected links with their new address - #1222
Merged
Merged
Conversation
…new address Closes yakew7#1196 Signed-off-by: minutechreview <141821420+minutechreview@users.noreply.github.com>
Contributor
|
@minutechreview is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
@yakew7 @ahmdkaml @propcgamer20-png - new PR touching a path you own, please review. |
Signed-off-by: minutechreview <141821420+minutechreview@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
check_external_links.pynow lists links whose redirect chain starts with a permanent 301 or 308, together with the address they end on, as a newmovedclass that is never counted as broken.Type
How it works:
_openstill usesurlopen, so the existing request path, user agents and DOI handling are unchanged. When the final URL differs from the requested one, it makes one extra request with redirects turned off to read the first hop's status. A 301 or 308 gives("moved", final_url); a 302 or 307 staysok, because the old address is still the one to cite. A 301 that ends on a 404 is stillbroken. Only links that redirected pay for the extra request.Report changes:
N moved.when there are any, and aMoved (permanent 301 or 308 redirect; update the link, not counted as broken):section listsold -> new (files)--json: a newmovedlist with the sameurl/detail/filesshape,detailbeing the new address--fail-on-brokenignores moved linksTests (local server in
tests/test_check_external_links.py):/moved(301) and/moved-permanently(308) report/okas the new address,/moved-twice(301 then 302) reports its final address,/redirect(302) and/temporary(307) stay ok,/moved-to-gone(301 to a 404) is broken, andmainprints the moved section and exits 0 with--fail-on-broken. The JSON key test now expectsmoved. CONTRIBUTING'smake external-linksline mentions it.Not in this PR: the weekly
external-links.ymlissue body still lists only broken links. It readsok/broken/unknown/ignoredby key, so the new key does not break it; showing moved links there would be a small follow-up.Checks:
pytest tests/test_check_external_links.py26 passed, 1 failed; the failure istest_a_name_that_does_not_resolve..., which fails the same way onmainon my machine (my DNS answers.invalidnames). With the old script the 4 new moved cases and the JSON key test fail; treating 302/307 as moved fails the temporary-redirect cases.ruff check faircode scripts tests,check_em_dash.pyandcheck_broken_links.pypass.Linked issue
Closes #1196