Skip to content

Fix slow task-log scrolling by memoizing useLogGroups' visible-items index - #73252

Open
udsy19 wants to merge 1 commit into
apache:mainfrom
udsy19:fix-tasklog-visible-items-memo
Open

udsy19 wants to merge 1 commit into
apache:mainfrom
udsy19:fix-tasklog-visible-items-memo

Conversation

@udsy19

@udsy19 udsy19 commented Sep 16, 2026

Copy link
Copy Markdown

Summary

closes: #55173

TaskLogContent re-renders on every scroll event (@tanstack/react-virtual's
scroll handler calls flushSync(rerender) synchronously whenever the visible
row range changes), and useLogGroups rebuilt its entire visibleItems /
originalToVisibleIndex / lineNumberToVisibleIndex index with an
unmemoized loop over the whole parsedLogs array on every one of those
renders — cost scaling with total log size, not with the handful of rows
actually on screen. That is the mechanism behind #55173: scrolling a large
task log gets progressively slower as the log grows, because every scroll
tick pays for a full rebuild of the whole log's visibility index, not just
the virtualized viewport.

Who reaches this: any user viewing task logs in the Airflow 3 UI
(/dags/{dag_id}/runs/{run_id}/tasks/{task_id}, Logs tab) — the standard,
documented way to read task output. Triggered by scrolling (wheel/trackpad)
a log of non-trivial size; every native scroll event on the log container
forces this rebuild.

Impact: perf-regression

Fix

Wrap the group-header index and the visible-items build in useMemo, keyed
on what they actually depend on (parsedLogs, expandedGroups,
groupParentMap), so they only recompute when the log content or the
expand/collapse state actually changes, not on every scroll-triggered
re-render.

Measured before/after

Isolated useLogGroups with renderHook, forcing a re-render whose props
are identical to the previous render (the same shape of update
react-virtual triggers on every scroll event where parsedLogs and
expandedGroups are unchanged). Measured with performance.now(), median
of 15 repeat re-renders per log size, in vitest (happy-dom).

Before: 43.571 ms / After: 0.118 ms, on the largest log size measured
(see the full size sweep below).

before (baseline, unpatched):  1,000 lines: 0.368ms  10,000 lines: 2.270ms  100,000 lines: 43.571ms
after (with this patch):       1,000 lines: 0.128ms  10,000 lines: 0.057ms  100,000 lines: 0.118ms

Before the fix, cost scales with total log size (~120x from 1k to 100k
lines) and already exceeds a 16ms/60fps frame budget at 100k — on a hot path
that runs on every scroll event, not once per log load. After the fix, cost
is flat, since the rebuild only reruns when its inputs actually change.

Negative control

Added useLogGroups.test.tsx"does not rebuild the visible-items index
on a re-render that changes neither parsedLogs nor expand state"
: renders
the hook, captures visibleItems/lineNumberToVisibleIndex, re-renders
with the identical parsedLogs reference and no state change, and asserts
referential equality (toBe) — true only if the memoization actually held.

Test fails without the fix, passes with it:

without the fix (useLogGroups.tsx reverted to main): 1 failed | 2 skipped (3)
  AssertionError: expected [ { …(2) }, …(4) ] to be [ { …(2) }, …(4) ] // Object.is equality
  (same content, different object identity — the array/map is rebuilt every render)
with the fix:                                        3 passed (3)

Was generative AI tooling used to co-author this PR?

  • Yes (please specify the tool below)
    Claude Code

Generated-by: Claude Code following the guidelines

Signed-off-by: Udaya Tejas udayatejas2004@gmail.com

react-virtual force-rerenders TaskLogContent synchronously
(flushSync) on every native scroll event whose visible range
changes. useLogGroups rebuilt its entire visibleItems /
originalToVisibleIndex / lineNumberToVisibleIndex index with an
unmemoized O(total log lines) loop on every one of those renders,
so scroll cost scaled with total log size instead of the number
of rows actually on screen -- the mechanism behind apache#55173
("Viewing large task logs get slower to scroll as they get
larger").

Wrap the group-header index and the visible-items build in
useMemo, keyed on the values they actually depend on
(parsedLogs, expandedGroups, groupParentMap) so they only
recompute when the log content or expand state actually changes.

Generated-by: Claude Code
Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com>
@boring-cyborg boring-cyborg Bot added the area:UI Related to UI/UX. For Frontend Developers. label Sep 16, 2026
@boring-cyborg

boring-cyborg Bot commented Sep 16, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Viewing large task logs get slower to scroll as they get larger.

1 participant