Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/rfc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting
- [Consumer Gate](consumer-gate.md) - Stopping and starting individual queue controllers at runtime via a consumer-side check: blocked deliveries are recorded as parked and postponed back to the queue (re-checked on redelivery), gate state as a separate extension with a file-based first implementation shared by tests and operators
- [Consumer Hold](consumer-hold.md) - Fourth delivery outcome letting a controller postpone its delivery: the message becomes a partition barrier that pauses consumption for a chosen delay, redelivers in order, and does not count as a failure toward dead-lettering
- [Change URIs](change-uri.md) - Identity of a code change: `scheme://{host[:port]}/{path}` per provider (GitHub PR, Phabricator Diff, git ref/commit) and canonical-form rules
- [Scoped Sequential Resource IDs](scoped-resource-ids.md) - Queue-scoped positive numeric IDs allocated by durable per-domain, per-kind counters, stored without queue/kind prefixes, and rendered directly in resource URL segments
- [Hooks Framework](hook-framework.md) - Implemented fire-and-forget side effects: one shared `HookEvent` contract (`api/base/hook/`) on a durable per-domain hook topic, dispatched by `platform/hook` to `platform/extension/hook`. Stovepipe `process` and `record` publish repository events; the SubmitQueue orchestrator registers the stage and does not publish events yet
- [Service-Scoped Extensions](service-scoped-extensions.md) - Implemented for SubmitQueue storage: gateway and orchestrator aggregates, schemas, and the core packages that serve one service have moved, while store contracts stay at `submitqueue/extension/storage`. Domain-level `buildrunner`, `conflict`, and `speculation` have not moved, and `changeset` still declares its own store slice

Expand Down
74 changes: 74 additions & 0 deletions doc/rfc/scoped-resource-ids.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Scoped Sequential Resource IDs

## Status

Proposed.

## Decision

A generated resource ID is the canonical decimal string for a positive value returned by a durable counter scoped to `(owner domain, queue, resource kind)`.

| Resource | Current ID | Proposed ID | Complete identity |
|---|---:|---:|---|
| SubmitQueue request | `demo-queue/42` | `"42"` | `(submitqueue, demo-queue, request, "42")` |
| SubmitQueue batch | `demo-queue/batch/7` | `"7"` | `(submitqueue, demo-queue, batch, "7")` |
| Stovepipe request | `request/monorepo/main/42` | `"42"` | `(stovepipe, monorepo/main, request, "42")` |

The decimal ID is unique only within its scope. The same value may appear in another queue, resource kind, or domain. APIs and messages therefore carry the queue separately; their typed field or message type supplies the resource kind.

Do not embed scope into the ID. Forms such as `demo-queue/42`, `demo-queue/batch/7`, `request.42`, and ARN-like resource names are not stored or accepted as IDs.

## Counter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect any entity type to be covered here? For example, we have BuildID (

type BuildID struct {
) which is an opaque string assigned by the build runner. We may want to consider whether int64 + counter are a required part of this contract, or optional for use in the entities that don't otherwise have another source for unique identifiers.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BuildIDs are always runner minted so we are just gonna treat them as is...


The counter backend persists one high-water mark per `(owner domain, queue, resource kind)`. For example:

```text
(submitqueue, demo-queue, request) -> 42
(submitqueue, demo-queue, batch) -> 7
(stovepipe, demo-queue, request) -> 11
```

Controllers allocate an ID before creating the resource; stores accept the caller-supplied ID and never generate one.

- The first ID is `1`; `0` is the unset value.
- Allocation is atomic across replicas and durable across restarts.
- Allocated values are never reused. Failed writes may leave gaps.
- Overflow fails instead of wrapping.
- Numeric order is allocation order only within the same scope.

The counter contract requires an atomic durable increment, not MySQL specifically. MySQL remains the initial implementation.

## Storage and contracts

Resource tables keep IDs as strings. Queue remains the leading key:

```text
request(queue, id VARCHAR(...), ...) PRIMARY KEY (queue, id)
batch(queue, id VARCHAR(...), ...) PRIMARY KEY (queue, id)
```

Reference columns use the same string type. Domain entities may use distinct named string types such as `RequestID` and `BatchID`; protobuf resource fields remain `string`. The counter backend may store its high-water marks as integers, and controllers convert allocated values to canonical decimal strings before creating resources.

This proposal applies only to counter-generated resources. Provider build IDs, message and hook IDs, change URIs, and content hashes keep their existing contracts.

## URLs and display

The decimal ID is used directly as one path segment:

```text
/requests/42
/batches/7
```

The route supplies the resource kind; the request context supplies the queue. Queue URL design is separate.

A UI may display `request.42`, `batch.7`, or `#42`, but those are derived labels, not identities.

## Rejected alternatives

- **Queue or kind prefixes:** duplicate explicit context, lengthen keys, require parsing, and introduce URL separators.
- **ARN-like names:** solve global lookup, which current APIs neither provide nor require.
- **UUIDs or a global counter:** provide global uniqueness at the cost of unnecessary encoding or coordination.
- **Integer resource fields:** couple the persisted and wire contracts to the current counter representation without adding identity semantics.
- **SQL auto-increment or `MAX(id) + 1`:** move allocation into one storage implementation or fail under concurrency.
- **Process-local counters:** reuse IDs after restart and collide across replicas.
Loading