Skip to content

fix(pd): release partial request allocations on abort - #1479

Closed
sufubao wants to merge 1 commit into
ModelTC:mainfrom
sufubao:fix/pd-pending-abort-slot-cleanup
Closed

fix(pd): release partial request allocations on abort#1479
sufubao wants to merge 1 commit into
ModelTC:mainfrom
sufubao:fix/pd-pending-abort-slot-cleanup

Conversation

@sufubao

@sufubao sufubao commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • follow up fix(pd): cancel pending PD requests on abort #1476 by making pending PD request cancellation exception-safe
  • release every shared-memory request object and slot index acquired before cancellation
  • make generation-task cancellation idempotent, remove obsolete delayed abort retries, and preserve cancellation state
  • add regressions for abort-before-registration and partial slot allocation cleanup

Problem

#1476 correctly tracks and cancels generation tasks before they enter req_id_to_out_inf. However, cancellation can arrive after one or more shared-memory request indexes have been allocated while the request is still waiting for another index. The allocation then unwinds without a registered ReqStatus, so neither HttpServerManager.abort() nor the normal recycle loop can release those indexes. Repeated cancellations can permanently exhaust the finite request-slot pool.

Verification

  • PYTHONPATH=$PWD python -m pytest -q unit_tests/server/httpserver (4 passed)
  • Black check on all changed Python files
  • flake8 on all changed Python files
  • git diff --check upstream/main...HEAD

Follow up ModelTC#1476 by making pending PD cancellation exception-safe. If cancellation interrupts shared-memory request allocation before req_id_to_out_inf registration, return every acquired request object and slot index. Also make generation-task cancellation idempotent, drop obsolete delayed retries, clean pending events, and preserve CancelledError state.
@sufubao

sufubao commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Closing after rechecking the actual PD request path. Since #1439, PD master expands multi-choice requests before dispatch and sends each P/D request with n=1 and best_of=1. Therefore the partial n>1 slot-allocation scenario in this PR is not reachable from the PD abort path, and upstream #1476 is sufficient for the reported pending-abort issue. The generic allocator cleanup may still be worth evaluating separately with a real non-PD cancellation reproducer.

@sufubao sufubao closed this Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant