Document the image processing limits - #1208
Open
ajwild wants to merge 15 commits into
Open
Conversation
Covers the new processing.timeout and processing.maxConcurrentResizes options, the raised upload timeout, and the one-thread-per-image change.
Both limits and the thread pool they share have to be set together, and none of them can be derived from the host's core count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The legacy transform runs while the result is streamed to storage, outside the maxConcurrentProcesses queue, so the thread pool is the only limit on it — which makes a large thread pool a memory risk rather than a free win. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The thread pool, not the two concurrency options, is what decides how many images are processed at once, so the sizing advice follows from both together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Link both configuration sections instead of restating how the limits interact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The per-frame limit and the frame count multiplied out to far more than either suggests, which is what the new option bounds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The per-image figure was measured at roughly 200MB. It is about 4 bytes per pixel of every frame, so a 200 megapixel animation takes 800MB, and the sizing guidance was recommending combinations that exceed a container's memory by an order of magnitude.
Still images stream, so their cost follows the variant being produced rather than the source. Only one of the two paths decodes anything: variants with use2025Behavior, uploads without it, never both.
maxTotalResolution bounds raster work to well under 20 seconds, so the hour-long processing the timeout was described as preventing is no longer reachable. SVG rendering cost is not measured by any pixel limit.
The headroom under the timeout is a function of maxTotalResolution, which is meant to be raised. Processing time scales with it linearly, so the timeout is what still holds at a high or unbounded limit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents the image processing limits added in these server PRs:
Based on #1207, which introduces the section it extends — review that one first.
mediaLibrary.images.processing.timeoutandmediaLibrary.images.processing.maxConcurrentResizesoptions.os.availableParallelism()rather thanos.cpus().lengthis the number to size against, and recommended values per container CPU limit.maxConcurrentProcessesas covering an upload end to end, which is what it does as of server PR #9865.uploadProcessingDirectorynow holds every transformed upload rather than only large animated ones, and corrects the volume sizing guidance: the worst case is 2GB rather than 300MB for themaxFileSize: 100mbthat customers commonly configure.