[rosa]

Fixes #827, and replaces #825.

A concurrency semaphore counts free slots, but it doesn't know which jobs hold them, so any job that gave a slot back by key was treated as a holder. That went wrong in three ways:

• A job that ran past its duration (A job that outlived its concurrency lock releases the lock of the job that replaced it #827). The dispatcher deletes the expired semaphore and the next job creates a new one. When the first job finishes, it gives its slot back to the new semaphore, so another job starts while the new holder is still running. With a backlog, the key stays one over its limit until the backlog drains.
• A job whose class was missing when it was dispatched. It skips its limits and takes no slot, but if it later runs where its class loads, it gives one back anyway.
• A job whose class is missing when it runs (Release the concurrency lock of a job whose class is missing #825). It took a slot when it was dispatched but can't give it back, because releasing needed the class, so the jobs behind it wait until the semaphore expires.

Now, when a job gets a slot, it records the semaphore's id and its concurrency limit in its serialized Active Job data, under solid_queue_concurrency_lock, which Active Job ignores when running the job. It gives the slot back only to that semaphore. A job that skips its limits records that it holds no slot. A blocked job records its semaphore when it's released.

Jobs that got their slot before upgrading have nothing recorded, so they keep giving it back by key, as they do today. Old and new versions can run side by side during a deploy. 2.0 will drop that fallback, so ready and running jobs from before the upgrade should finish first: a slot that isn't given back only returns when its semaphore expires, and on a busy key with a limit above 1 the other jobs keep pushing that expiry out.

One case this doesn't cover, and that we accept: during a deploy, if a failed job is retried by a process still on the old version and then run by one on the new version, the old process doesn't update the recorded semaphore, so the new one gives the slot back to the wrong place. With a limit of 1, the slot comes back when the semaphore expires; with a higher limit on a busy key, the key can run one slot short for as long as it stays busy. It needs all of that to happen inside the deploy window, and it never lets jobs run over the limit.

The change is meant to ship on its own in 1.7.2, with a note in UPGRADING.

The first two tests come from the reproduction in #827 by @npadgett.

🤖 Generated with Claude Code