[for minor release] Aggregate DataStore Accounting Requests - #8714
Open
chaen wants to merge 3 commits into
Open
Conversation
chaen
force-pushed
the
aggregate_accounting
branch
2 times, most recently
from
July 27, 2026 08:34
5937517 to
2c30acc
Compare
fstagni
changed the title
chaen
force-pushed
the
aggregate_accounting
branch
2 times, most recently
from
July 31, 2026 13:56
2421b0b to
9e6ebff
Compare
fstagni
reviewed
Aug 27, 2026
chaen
force-pushed
the
aggregate_accounting
branch
from
September 1, 2026 14:21
9e6ebff to
2bf1f18
Compare
chaen
force-pushed
the
aggregate_accounting
branch
from
September 4, 2026 09:43
b6d3b18 to
abefea9
Compare
fstagni
reviewed
Sep 4, 2026
Comment on lines
+144
to
+145
| # Number of Accounting requests to fetch to batch | ||
| # 0 to disable (default) |
Contributor
There was a problem hiding this comment.
Being pedantic...
Suggested change
| # Number of Accounting requests to fetch to batch | |
| # 0 to disable (default) | |
| # Max number of Accounting requests to fetch to batch | |
| # 0 to disable (default) |
|
|
||
| if self.ACCOUNTING_BATCH_MAX_REQUESTS: | ||
| self.log.info(f"Accounting max requests = {self.ACCOUNTING_BATCH_MAX_REQUESTS} request/cycle") | ||
| self.log.info(f"Accouting batch size = {self.ACCOUNTING_BATCH_SIZE} requests") |
Contributor
There was a problem hiding this comment.
Suggested change
| self.log.info(f"Accouting batch size = {self.ACCOUNTING_BATCH_SIZE} requests") | |
| self.log.info(f"Accounting batch size = {self.ACCOUNTING_BATCH_SIZE} requests") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the DataStore services go down, we quickly end up with a few millions ForwardDISET requests. This takes forever to execute because of the overhead of setting up a Request execution environment.
This PR adds the capability for the CleanReqDBAgent to batch together such requests.
Id also adds an extra index on the ReqDB (in the wiki already)
BEGINRELEASENOTES
*RMS
NEW: the CleanReqDBAgent can aggregate DataStore Requests together
*deployment
A new index needs to be added to the ReqDB
ENDRELEASENOTES