fix(warehouse): close MetricsData after processing to release Arrow buffers - #4335
Open
paultanay wants to merge 6 commits into
Open
fix(warehouse): close MetricsData after processing to release Arrow buffers#4335paultanay wants to merge 6 commits into
paultanay wants to merge 6 commits into
Conversation
…uffers MetricsData implements AutoCloseable and wraps an ArrowTable backed by Arrow column vectors. The persistent-data-storage consumer loop was polling MetricsData objects and passing them through writers without ever calling close(), preventing the backing Arrow buffers from being released and causing steady heap growth under load (issue apache#4136). Wrap the polled object in try-with-resources so close() is guaranteed on all exit paths—normal completion, writer exception, and interrupt. DataStorageDispatch is the terminal consumer of each object, so closing here is correct and safe.
paultanay
marked this pull request as ready for review
August 21, 2026 05:42
Aias00
approved these changes
Aug 22, 2026
Aias00
left a comment
Contributor
There was a problem hiding this comment.
Review: close MetricsData after processing to release Arrow buffers
Clean, minimal, and correct resource-leak fix.
What's good
commonDataQueue.pollMetricsDataToStorage()is now wrapped in try-with-resources, soMetricsData.close()(releasing the underlying Arrow column-vector buffers) is guaranteed on every exit path: normal completion, writer exception, and thread interrupt. This directly addresses the steady heap growth reported in #4136.DataStorageDispatch.startPersistentDataStorage()is the terminal consumer of theseMetricsDataobjects, so closing them here is safe — no downstream reader needs the Arrow buffers afterward.- The original control flow is preserved exactly: the inner
try/finallystill ensuresrealTimeDataWriter.saveData(...)runs even ifcalculateMonitorStatus/history/plugin steps throw. The null-guardedcontinueis still valid under try-with-resources (a null resource is simply not closed). MetricsData implements AutoCloseable, so the compiler-enforced contract holds.
Non-blocking note
- Standard try-with-resources behavior means if
close()itself throws, it would suppress an earlier exception from the body. That is an acceptable, rare edge for this terminal-sink path.
No blocking issues. Approve.
Contributor
Author
|
CI failures are in the setup-deps infrastructure step (fails before any compilation). Build and all 88 tests pass locally under the release profile (mvn clean package -Prelease). Looks like a runner issue. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's changed?
DataStorageDispatch.startPersistentDataStorage()pollsMetricsDataobjects from the queue and passes them through the history, plugin, and realtime writers, but never calledclose()afterward.MetricsDataimplementsAutoCloseableand wraps anArrowTablebacked by Arrow column vectors - withoutclose(), those vectors and their buffers are retained until GC decides to collect them. Under production load with many monitors this compounds into the steady heap growth reported in #4136.Wrapped the polled object in try-with-resources so
MetricsData.close()is guaranteed on all exit paths: normal completion, writer exception, and thread interrupt.DataStorageDispatchis the terminal consumer, so closing here is safe.Fixes: #4136
Checklist
hertzbeat-warehouse,hertzbeat-common-core)