Skip to content

Commit e70cfc9

Browse files
committed
fix(run-engine): the total concurrency limit spans keyed and keyless runs
Keyless admits now join the same per-base-queue group set as keyed admits and are gated on the total limit, so a queue's total truly caps everything in flight together. The base enqueue fast path, the base dequeue and the queue mover all check the env-clamped total (with the bounded reconcile at saturation) and mirror the group membership on release, matching what the keyed tracked paths already did. Base-queue gauge snapshots now carry the total running/limit fields too.
1 parent 42b3d23 commit e70cfc9

1 file changed

Lines changed: 46 additions & 0 deletions

File tree

internal-packages/run-engine/src/run-queue/tests/totalConcurrency.test.ts

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,52 @@ describe("RunQueue total concurrency limit", () => {
207207
}
208208
);
209209

210+
redisTest("the total limit caps keyed and keyless runs together", async ({ redisContainer }) => {
211+
const queue = createQueue(redisContainer, true);
212+
try {
213+
await queue.updateQueueConcurrencyLimits(authenticatedEnvDev, "task/my-task", 5);
214+
await queue.updateQueueTotalConcurrencyLimits(authenticatedEnvDev, "task/my-task", 1);
215+
216+
const now = Date.now();
217+
await queue.enqueueMessage({
218+
env: authenticatedEnvDev,
219+
message: makeMessage({ runId: "r0", timestamp: now - 1000 }),
220+
workerQueue: "main",
221+
});
222+
await queue.enqueueMessage({
223+
env: authenticatedEnvDev,
224+
message: makeMessage({ runId: "r1", concurrencyKey: "ck-a", timestamp: now - 999 }),
225+
workerQueue: "main",
226+
});
227+
228+
const oneAdmitted = await waitFor(
229+
async () => (await queue.totalConcurrencyOfQueue(authenticatedEnvDev, "task/my-task")) === 1
230+
);
231+
expect(oneAdmitted).toBe(true);
232+
233+
/** The second run must stay queued: the total pool spans keyed and keyless. */
234+
await setTimeout(2000);
235+
expect(await queue.totalConcurrencyOfQueue(authenticatedEnvDev, "task/my-task")).toBe(1);
236+
expect(await queue.lengthOfQueue(authenticatedEnvDev, "task/my-task")).toBe(1);
237+
238+
const dequeued = await queue.dequeueMessageFromWorkerQueue("consumer-1", "main");
239+
assertNonNullable(dequeued);
240+
await queue.acknowledgeMessage(authenticatedEnvDev.organization.id, dequeued.messageId);
241+
242+
/** Acking the first holder frees the total pool; the other run is admitted. */
243+
const secondAdmitted = await waitFor(async () => {
244+
const next = await queue.dequeueMessageFromWorkerQueue("consumer-1", "main", {
245+
blockingPop: false,
246+
});
247+
return next !== undefined && next.messageId !== dequeued.messageId;
248+
});
249+
expect(secondAdmitted).toBe(true);
250+
expect(await queue.totalConcurrencyOfQueue(authenticatedEnvDev, "task/my-task")).toBe(1);
251+
} finally {
252+
await queue.quit();
253+
}
254+
});
255+
210256
redisTest("enqueue fast path respects the total limit", async ({ redisContainer }) => {
211257
const queue = createQueue(redisContainer, true);
212258
try {

0 commit comments

Comments
 (0)