feat(harness): support ordered async memory writes - #2381
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
oss-maintainer
left a comment
There was a problem hiding this comment.
LGTM. Looks good.
Automated review by "github-manager-bot"
|
|
||
| private static final Logger log = LoggerFactory.getLogger(MemoryOperationScheduler.class); | ||
|
|
||
| private final ConcurrentHashMap<String, CompletableFuture<Void>> tails = |
There was a problem hiding this comment.
I think this class may require overload protection.
If LLM reponses slow, will too many async tasks in tails, and cause CPU or memory overload.
What do you think?
There was a problem hiding this comment.
Good catch. The previous per-key future chains were unbounded, so a slow memory LLM could retain operations and captured request state indefinitely. Commit 97c0111 adds a fair global bound over running + queued async memory operations (default 256, configurable with maxPendingMemoryOperations). When saturated, submitters wait for capacity, which propagates backpressure instead of dropping writes; accepted operations remain serialized per isolation key, and different keys still execute concurrently. Shutdown rejects capacity waiters after close begins and continues waiting for all already accepted work.
Added concurrent/boundary coverage for saturation backpressure, release/acceptance after capacity returns, close racing with a blocked submitter, invalid capacity, same-key ordering, and cross-key concurrency. Spotless passes; the focused scheduler/config/async-maintenance suite passes 16/16. The full core suite passes 2218 tests; three unrelated macOS temp-directory cleanup races in the harness suite passed when their two test classes were rerun (14/14).
Fixes #2334
Problem / root cause
MemoryFlushMiddlewareandMemoryMaintenanceMiddlewareappend LLM-backed work withconcatWith, so an otherwise completed agent call holds the session gate until memory extraction and consolidation finish. A naive fire-and-forget subscription would remove that latency but could reorder writes for concurrent turns and lose accepted work during shutdown.Changes
MemoryConfig.ExecutionMode.ASYNCas an opt-in while retainingBLOCKINGas the compatibility defaultHarnessAgent.close()before workspace resources closeUser impact
Latency-sensitive streaming and orchestration can configure
.executionMode(MemoryConfig.ExecutionMode.ASYNC)so the agent Flux completes without waiting for memory LLM calls. Writes for the same isolation key stay ordered. Existing applications retain blocking completion behavior unless they opt in.Tests
mvn -pl agentscope-harness -am -Djacoco.skip=true -Dtest=MemoryConfigTest,MemoryOperationSchedulerTest,MemoryMaintenanceAsyncTest -Dsurefire.failIfNoSpecifiedTests=false test— 13 passedmvn -pl agentscope-harness -DskipTests spotless:check— passed@TempDircleanup errors reproduced in the session-tree mirror path tracked by [Bug]: LocalFilesystemPersonalAssistantExampleTest flaky JUnit @TempDir cleanup race with session-tree-mirror thread #2251 / PR fix(harness): drain session-tree-mirror on HarnessAgent.close #2301, with no assertion failures from this changeCompatibility / risk
BLOCKINGremains the default. Async mode requires applications to closeHarnessAgentduring shutdown; close drains all accepted operations. The scheduler serializes work only within an isolation key, so unrelated users/sessions are not globally bottlenecked.