Skip to content

feat(harness): support ordered async memory writes - #2381

Open
March-77 wants to merge 2 commits into
agentscope-ai:mainfrom
March-77:agent/issue-2334-async-memory-writes
Open

feat(harness): support ordered async memory writes#2381
March-77 wants to merge 2 commits into
agentscope-ai:mainfrom
March-77:agent/issue-2334-async-memory-writes

Conversation

@March-77

Copy link
Copy Markdown

Fixes #2334

Problem / root cause

MemoryFlushMiddleware and MemoryMaintenanceMiddleware append LLM-backed work with concatWith, so an otherwise completed agent call holds the session gate until memory extraction and consolidation finish. A naive fire-and-forget subscription would remove that latency but could reorder writes for concurrent turns and lose accepted work during shutdown.

Changes

  • add MemoryConfig.ExecutionMode.ASYNC as an opt-in while retaining BLOCKING as the compatibility default
  • queue post-call flush and maintenance by the configured isolation key, preserving per-user/session order while allowing independent keys to run concurrently
  • snapshot the completed turn before enqueueing and offload raw session JSONL before optional LLM extraction
  • drain accepted memory operations from HarnessAgent.close() before workspace resources close
  • document the async lifecycle and configuration in English and Chinese memory guides

User impact

Latency-sensitive streaming and orchestration can configure .executionMode(MemoryConfig.ExecutionMode.ASYNC) so the agent Flux completes without waiting for memory LLM calls. Writes for the same isolation key stay ordered. Existing applications retain blocking completion behavior unless they opt in.

Tests

Compatibility / risk

BLOCKING remains the default. Async mode requires applications to close HarnessAgent during shutdown; close drains all accepted operations. The scheduler serializes work only within an isolation key, so unrelated users/sessions are not globally bottlenecked.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

@oss-maintainer oss-maintainer left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Looks good.


Automated review by "github-manager-bot"


private static final Logger log = LoggerFactory.getLogger(MemoryOperationScheduler.class);

private final ConcurrentHashMap<String, CompletableFuture<Void>> tails =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this class may require overload protection.
If LLM reponses slow, will too many async tasks in tails, and cause CPU or memory overload.
What do you think?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. The previous per-key future chains were unbounded, so a slow memory LLM could retain operations and captured request state indefinitely. Commit 97c0111 adds a fair global bound over running + queued async memory operations (default 256, configurable with maxPendingMemoryOperations). When saturated, submitters wait for capacity, which propagates backpressure instead of dropping writes; accepted operations remain serialized per isolation key, and different keys still execute concurrently. Shutdown rejects capacity waiters after close begins and continues waiting for all already accepted work.

Added concurrent/boundary coverage for saturation backpressure, release/acceptance after capacity returns, close racing with a blocked submitter, invalid capacity, same-key ordering, and cross-key concurrency. Spotless passes; the focused scheduler/config/async-maintenance suite passes 16/16. The full core suite passes 2218 tests; three unrelated macOS temp-directory cleanup races in the harness suite passed when their two test classes were rerun (14/14).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: 希望写入记忆能异步进行

3 participants