Skip to content

chore(labrinth): improve version_files performance - #7828

Open
fetchfern wants to merge 4 commits into
mainfrom
fetch/version-files
Open

fetchfern wants to merge 4 commits into
mainfrom
fetch/version-files

Conversation

@fetchfern

@fetchfern fetchfern commented Oct 5, 2026 •

Copy link
Copy Markdown
Member
  1. Changes the version files update code to, instead of loading every version of a project and doing filtering + sorting separately, do it all in postgres with a CROSS JOIN LATERAL query. Massively reduces the amount of queries + Redis commands + data loaded.
  2. Tweak the file attribution query to skip scanning attribution status per file for projects we already know aren't withheld. Most projects, especially popular ones, are almost certain to not be withheld; either they're exempt or we can easily tell they're not withheld through project_attribution_groups attribution checks. So file-level checks are almost always never necessary.

Fixes DEV-1473

fetchfern and others added 3 commits October 5, 2026 15:40
Pick the newest visible matching version per project directly in Postgres
instead of fetching every candidate version, loading all of them through
`DBVersion::get_many`, filtering visibility in Rust and loading the winners
a second time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pick the newest matching listed version per hash directly in Postgres
instead of loading every version of every project through
`DBVersion::get_many` and filtering in Rust. Visibility is still checked
on the picked version only, so a hidden or withheld newest match still
returns nothing. `version_file/{hash}/update` shares the same query, and
`update_individual` now uses the read-only pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`get_files_missing_attribution` now takes each version's project and first
finds the projects that aren't exempt and have unresolved attribution
groups. When there are none, which is the case for every mod and every
grandfathered project, the file lookup doesn't run at all, instead of
reading every file of every version passed in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Pull request changelog

App

Added

Changed

Deprecated

Removed

Fixed

Security

Website

Added

Changed

Deprecated

Removed

Fixed

Security

Hosting

Added

Changed

Deprecated

Removed

Fixed

Security

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant