Integration branch for ably-pubsub-server v4 package

.claude/skills/release/SKILL.md

@@ -17,7 +17,7 @@ Then perform these steps in order:
 1. Run `git checkout -b release/NEW_VERSION`
 2. Replace `OLD_VERSION` with `NEW_VERSION` in all of the following files:
     - `pyproject.toml` — the `version` property
-    - `ably/__init__.py` — lib_version value
+    - `ably/pubsub/version.py` — lib_version value
       3.Run `uv sync` to update `uv.lock` file
 4. Commit all files together with message: `chore: bump version to NEW_VERSION`
 3. Fetch merged PRs since the last release tag using:

.claude/skills/uts-to-python/SKILL.md

@@ -20,6 +20,9 @@ gh api 'repos/ably/specification/contents/uts/realtime/integration/<spec>.md' --
 gh api repos/ably/specification/contents/uts/docs/proxy.md --jq '.content' | base64 -d
 gh api 'repos/ably/specification/contents/uts/rest/integration/proxy/<spec>.md' --jq '.content' | base64 -d
 gh api 'repos/ably/specification/contents/uts/realtime/integration/proxy/<spec>.md' --jq '.content' | base64 -d
+gh api 'repos/ably/specification/contents/uts/objects/unit/<spec>.md' --jq '.content' | base64 -d
+gh api 'repos/ably/specification/contents/uts/objects/integration/<spec>.md' --jq '.content' | base64 -d
+gh api repos/ably/specification/contents/uts/objects/helpers/standard_test_pool.md --jq '.content' | base64 -d
 ```
 
 The realtime integration tier nests, so list a directory before fetching from it:
@@ -39,9 +42,10 @@ carries over as it stands: `channels/channel_publish_test.md` becomes
 `channels/channel_publish_test.py`. Every directory needs an `__init__.py`, as `test` is
 a package.
 
-There are two kinds of tier. `rest/unit` and `realtime/unit` serve every request from a
-mock and reach no network; `rest/integration` and `realtime/integration` run against the
-real Ably sandbox and have no mock at all. See **The integration tier** below.
+There are two kinds of tier. `rest/unit`, `realtime/unit` and `objects/unit` reach no
+network; `rest/integration`, `realtime/integration` and `objects/integration` run against
+the real Ably sandbox and have no mock at all. See **The integration tier** below, and
+**The objects tier** for LiveObjects.
 
 `test/uts/rest/unit/time_test.py` is the reference example for REST unit,
 `test/uts/realtime/unit/connection/auto_connect_test.py` for realtime unit,
@@ -451,6 +455,105 @@ async def test_rsc15l4_cloudfront_header_fallback(sandbox, proxy_session):
     assert http_responses(log)[0]['status'] == 403
 ```
 
+## The objects tier
+
+`uts/objects/unit/<name>.md` becomes `test/uts/objects/unit/<name>_test.py`, the integration
+specifications keep their `_test` suffix under `test/uts/objects/integration/`, and
+`integration/proxy/objects_faults.md` becomes
+`test/uts/objects/integration/proxy/objects_faults_test.py`. Seven unit specifications are
+pure and construct internal objects; the other eight drive `channel.object` over the mock
+websocket. Follow `internal_live_map_test.py` (pure), `path_object_mutations_test.py` (mock)
+and `objects_lifecycle_test.py` (integration). The integration package's own `conftest.py`
+provisions a separate app under the realtime tier's fixture name, `realtime_sandbox`, and
+all three of its specifications take `use_binary_protocol`. The package is
+`ably.pubsub.liveobjects`, named for the product as PDR-080 has package names do, while
+`test/uts/objects/` keeps the specification's directory name so that each test file maps
+onto its specification; the public names are exported from `ably.pubsub.server.liveobjects`.
+
+**Translate the untyped API through the typed views** (LODR-061, the RTTS partition). The
+specifications call every method on one `PathObject` or `Instance` class; ably-python
+reaches a type's methods through its view:
+
+| Pseudocode | ably-python |
+|---|---|
+| `root.get("score").value()` | `root.get('score').as_live_counter().value()` |
+| `root.get("name").value()` | `root.get('name').as_primitive().value()` |
+| `pathObject.set("k", v)` | `await path_object.as_live_map().set('k', v)`; `root` is already a `LiveMapPathObject` |
+| untyped `value() == null` | `as_primitive().value() is None` **and** `as_live_counter().value() is None` |
+| `inst.id()` | `inst.id`, a property, as is `inst.type` |
+| `{ depth: n }` | `subscribe(listener, depth=n)`, keyword-only |
+
+Mutations are `async`; reads, navigation, views and `subscribe` are not. `entries()` yields
+tuples. A path's views never raise, and a write through the wrong one raises 92007, or 92005
+for a path that does not resolve; an `Instance`'s views raise 92007. A spelling difference is
+not a deviation. Where the partition makes a read unreachable — `value()` on a
+`LiveMapInstance` — assert the `type`, `not hasattr(...)` and the 92007, and cite S-5.
+
+**The pure tier adapts to five shapes**, defined in `deviations.md` and cited by label in the
+module docstring: S-1 `apply_operation` returns a boolean, so record updates with
+`capture_updates(obj)`; S-2 the sync state machine is a standalone `RealtimeObject()`'s
+(`_on_attached(has_objects)`, `_handle_object_sync_messages(msgs, channel_serial)`,
+`_sync_state`); S-3 `evaluate(vt, timestamp_ms)`; S-4 the retained create is
+`operation.resolved_counter_create` / `resolved_map_create`; S-5 above. Members of a public
+class that LODR-061 does not name carry a leading underscore (`channel.object._objects_pool`).
+
+`test/uts/objects/helpers/standard_test_pool.py` holds what the specifications share:
+`setup_synced_channel` (and `_no_ack`), `standard_mock_websocket`, `objects_client`,
+`objects_channel_options`, `objects_connected_message`, `objects_attached_message`, the
+`build_*` builders, `json_value`, `bytes_value`, `ack_serial`, `remote_serial`,
+`below_ack_serial`, `object_message(s)`, `capture_updates`, `build_public_object_message`,
+`assert_unchanged_after_quiescence`, `provision_objects_via_rest`, and the wire constants
+(`HAS_OBJECTS`, `OBJECT_SUBSCRIBE_FLAG`, `OBJECT_PUBLISH_FLAG`, `LWW`, the actions).
+`test/uts/README.md` says what each is.
+
+### Traps found while deriving and implementing the objects tier
+
+- **An injected frame is applied on the transport's read task, not by `send_to_client`.**
+  Read state only after a `poll_until` on its effect. A negative — "did not fire", "the echo
+  was not applied" — needs a positive control sent *behind* the message under test and
+  `assert_unchanged_after_quiescence`; an exact count after `poll_until(>= n)` needs
+  `await settle()` first; and a subscription made straight after a seeding message receives
+  the seed, so poll for the seed before subscribing. The specifications often skip all three,
+  and their negatives then pass whatever the SDK does.
+- **Process a re-sync ATTACHED before starting what it should hold back.** Send it, poll until
+  `channel.object._sync_state == ObjectsSyncState.SYNCING`, then start `get()` or the write as
+  a task; started earlier, it sees SYNCED and resolves at once.
+- **A mock ATTACHED grants no modes.** The standard one carries `HAS_OBJECTS` alone, which
+  empties `channel.modes`, and RTO2 then checks the modes the channel *requested* — so request
+  them with `objects_channel_options()` or `get()` raises 40024. Grant modes as `flags` bits,
+  never as the specifications' `modes: [...]`. An injected `flags: 128` at the integration tier
+  empties them the same way.
+- **A write resolves on its ACK.** The standard mock ACKs each OBJECT with
+  `ack_serial(msgSerial, i)`; under `setup_synced_channel_no_ack`, drive the write as a task.
+  Those serials land in `appliedOnAckSerials`, so never reuse one as an inbound serial, and
+  use `remote_serial(n)` for a remote write: a bare `'99'` sorts before `POOL_SERIAL` and is
+  stale.
+- **`json` values are JSON strings on the wire** (OD2g), so compare a captured
+  `mapSet.value.json` after `json.loads`; `bytes` are base64. Actions are
+  `ObjectOperationAction`, an `IntEnum`.
+- **Numbers decode to `float`.** Read them with `value(float)`; `value(int)` raises
+  `TypeError`, since `expected` must be exactly one of `str, float, bool, bytes, list, dict`.
+  Assert booleans with `is True`, since `True == 1`.
+- **Creating a `LiveCounter` or `LiveMap` reads `/time`.** `objects_client` answers it; a
+  client built any other way needs `mock_http=time_mock_http(clock)`.
+- **A tombstoned object reads null with no GC at all**, so a GC test asserts that the object
+  left `_objects_pool`. The GC interval, 300000 ms, is read when the first ATTACHED schedules
+  the timer on the client's clock: set `channel.object._gc_interval_ms` before `get()`.
+- **The public and internal `ObjectMessage` share a name.** Reach the public one as
+  `publicmessage.ObjectMessage`, from `ably.pubsub.liveobjects`.
+- **Check that a test can fail.** Several objects specifications assert something that holds
+  with the behaviour removed. The derivation caught them by running each module against a
+  throwaway reference implementation with one fault injected; add the discriminating
+  assertion under `# UTS SPEC ERROR:` and record the fault.
+- **At the sandbox, the echo arrives before the ACK**, so RTO9a3's dedup is only reachable at
+  the mock tier, and **ACKs are paced at about one per 500 ms per connection**, so each
+  awaited write after the first can take half a second.
+- **Every objects channel gets an OBJECT_SYNC**, an empty one included, and a resumed
+  ATTACHED on an attached channel restarts the sync without a state change: wait on
+  `get()`, not on a channel state. A sync cursor can itself contain `:`.
+- **uts-proxy matches `action` as a string and names actions only up to AUTH**: OBJECT_SYNC
+  is `'20'`, OBJECT `'19'`. Its `delay` holds every later frame behind the delayed one.
+
 ## Traps that cost the most time
 
 Ordered by how much they cost, not by subject. Every one was hit for real while
@@ -728,7 +831,7 @@ Established against the real proxy and the real sandbox while deriving
   asks for a token through a client pointed at the session puts a request in front of
   the waiting rule and an extra `http_request` in the log, which breaks every
   assertion that counts requests exactly (`== 1` for RSC15l's 4xx test, `>= 2` for the
-  fallback ones). Build an inner `AblyRest(key=api_key, endpoint=SANDBOX_ENDPOINT)`,
+  fallback ones). Build an inner `create_http_client(key=api_key, endpoint=SANDBOX_ENDPOINT)`,
   request the token through that, and close it.
 - **`http_response` events carry no `path`.** They have `status` and `ruleMatched`
   only, so "the injected response fired" is read off the response events **in order**
@@ -896,8 +999,8 @@ the reasoning. The next reader will otherwise reach the same first conclusion.
 
 ```bash
 uv run --frozen --extra crypto --extra dev ruff check ably/ test/
-uv run --frozen --extra crypto --extra dev pytest test/uts/rest/unit test/uts/realtime/unit test/uts/helpers -q
-uv run --frozen --extra crypto --extra dev pytest test/uts/rest/integration test/uts/realtime/integration -q
+uv run --frozen --extra crypto --extra dev pytest test/uts/rest/unit test/uts/realtime/unit test/uts/objects/unit test/uts/helpers test/uts/objects/helpers -q
+uv run --frozen --extra crypto --extra dev pytest test/uts/rest/integration test/uts/realtime/integration test/uts/objects/integration -q
 RUN_DEVIATIONS=1 uv run --frozen --extra crypto --extra dev pytest test/uts -q
 ```
 
@@ -906,7 +1009,7 @@ environment's cutoff — and `--extra dev` carries pytest. Line length is 115. I
 `uv.lock` changes, revert only what the run touched; a wholesale
 `git checkout -- uv.lock` takes any edit of your own with it.
 
-The second command is the offline tiers, which need no network. The third is the two
+The second command is the offline tiers, which need no network. The third is the three
 integration tiers, each of which provisions a sandbox app of its own and does need the
 network; `pytest test/uts -q` runs everything together, so run the tiers separately when
 only one is in question. All three must pass.

.github/workflows/release.yml

@@ -42,27 +42,27 @@ jobs:
         with:
           name: python-package-distributions
           path: dist/
-      - name: Check that wheel and tarball contains ably/sync/
+      - name: Check that wheel and tarball contains ably/pubsub/sync/
         run: |
           # Check wheel
           WHEEL=$(ls dist/*.whl | head -n 1)
           echo "Checking wheel: $WHEEL"
-          if unzip -l "$WHEEL" | grep -q "ably/sync/"; then
-            echo "✅ Found ably/sync/ in wheel"
+          if unzip -l "$WHEEL" | grep -q "ably/pubsub/sync/"; then
+            echo "✅ Found ably/pubsub/sync/ in wheel"
           else
             unzip -l "$WHEEL" 
-            echo "❌ ably/sync/ not found in wheel"
+            echo "❌ ably/pubsub/sync/ not found in wheel"
             exit 1
           fi
 
           # Check tarball
           TARBALL=$(ls dist/*.tar.gz | head -n 1)
           echo "Checking tarball: $TARBALL"
-          if tar -tzf "$TARBALL" | grep -q "ably/sync/"; then
-            echo "✅ Found ably/sync/ in tarball"
+          if tar -tzf "$TARBALL" | grep -q "ably/pubsub/sync/"; then
+            echo "✅ Found ably/pubsub/sync/ in tarball"
           else
             tar -tzf "$TARBALL"
-            echo "❌ ably/sync/ not found in tarball"
+            echo "❌ ably/pubsub/sync/ not found in tarball"
             exit 1
           fi
 
@@ -74,7 +74,7 @@ jobs:
     runs-on: ubuntu-latest
     environment:
       name: pypi
-      url: https://pypi.org/p/ably
+      url: https://pypi.org/p/ably-pubsub-server
     permissions:
       id-token: write  # IMPORTANT: mandatory for trusted publishing
 
@@ -94,7 +94,7 @@ jobs:
       - name: Read VERSION_NAME from dist/
         id: version
         run: |
-          VERSION_NAME=$(basename dist/ably-*.tar.gz | sed -E 's/^ably-([^-]+)\.tar\.gz$/\1/')
+          VERSION_NAME=$(basename dist/ably_pubsub_server-*.tar.gz | sed -E 's/^ably_pubsub_server-([^-]+)\.tar\.gz$/\1/')
           echo "version=$VERSION_NAME" >> $GITHUB_OUTPUT
 
       - name: Compare version with tag
@@ -118,7 +118,7 @@ jobs:
 
     environment:
       name: testpypi
-      url: https://test.pypi.org/p/ably
+      url: https://test.pypi.org/p/ably-pubsub-server
 
     permissions:
       id-token: write  # IMPORTANT: mandatory for trusted publishing

.gitignore

@@ -55,5 +55,5 @@ ably/types/options.py.orig
 test/ably/restsetup.py.orig
 
 .idea/**/*
-ably/sync/**
+ably/pubsub/sync/**
 test/ably/sync/**

CONTRIBUTING.md

@@ -40,7 +40,7 @@ The release process must include the following steps:
 
 1. Ensure that all work intended for this release has landed to `main`
 2. Create a release branch named like `release/2.0.1`
-3. Add a commit to bump the version number, updating [`pyproject.toml`](./pyproject.toml) and [`ably/__init__.py`](./ably/__init__.py)
+3. Add a commit to bump the version number, updating [`pyproject.toml`](./pyproject.toml) and [`ably/pubsub/version.py`](./ably/pubsub/version.py)
 4. Run [`github_changelog_generator`](https://github.com/github-changelog-generator/github-changelog-generator) to automate the update of the [CHANGELOG](./CHANGELOG.md). This may require some manual intervention, both in terms of how the command is run and how the change log file is modified. Your mileage may vary:
    - The command you will need to run will look something like this: `github_changelog_generator -u ably -p ably-pubsub-python --since-tag v2.0.0 --output delta.md --token $GITHUB_TOKEN_WITH_REPO_ACCESS`. Generate token [here](https://github.com/settings/tokens/new?description=GitHub%20Changelog%20Generator%20token).
    - Using the command above, `--output delta.md` writes changes made after `--since-tag` to a new file

LONG_DESCRIPTION.rst

@@ -1,15 +1,22 @@
-Official Ably Bindings for Python
-==================================
+Official Ably Pub/Sub Bindings for Python Servers
+=================================================
 
-A Python client library for Ably Realtime messaging.
+A Python client library for Ably Pub/Sub realtime messaging, for use from
+servers.
 
 
 Setup
 -----
 
 You can install this package by using the pip tool and installing:
 
-    pip install ably
+    pip install ably-pubsub-server
+
+The package installs into ``ably.pubsub``. Both ``ably`` and ``ably.pubsub``
+are namespace packages shared with the other ``ably-*`` distributions, so the
+whole public API is reached through ``ably.pubsub.server``::
+
+    from ably.pubsub.server import create_http_client, create_realtime_client
 
 
 Using Ably for Python