pubsub: breaking changes to values and types returned

UPDATING.md

@@ -124,6 +124,129 @@ await pubsub_realtime_client.channels.get('my-channel').detach()
 pubsub_realtime_client.channels.release('my-channel')
 ```
 
+### `PresenceMessage.timestamp` is milliseconds since the epoch
+
+**This change is silent: comparing a timestamp with `==` against a `datetime` now returns
+`False` rather than raising.** `PresenceMessage.timestamp` is an `int` of milliseconds since
+the unix epoch, as `Message.timestamp` already is. 3.x returned a naive UTC `datetime` for a
+presence message received from Ably, and an aware one for a LEAVE the library synthesized.
+An ordering comparison such as `<` against a `datetime` raises `TypeError`. Convert the
+timestamp where you need a `datetime`:
+
+Example 3.x code:
+```python
+received_at = member.timestamp
+```
+
+Example 4.0.0 code:
+```python
+from datetime import datetime, timezone
+
+received_at = datetime.fromtimestamp(member.timestamp / 1000, tz=timezone.utc)
+```
+
+### `Stats.interval_time` is timezone-aware
+
+`Stats.interval_time` is now an aware `datetime` in UTC, where 3.x returned a naive one. An
+ordering comparison such as `<` against a naive `datetime` now raises `TypeError`, and `==`
+against one quietly returns `False`. Compare it with aware datetimes:
+
+Example 3.x code:
+```python
+if stat.interval_time >= datetime(2024, 1, 1):
+    ...
+```
+
+Example 4.0.0 code:
+```python
+if stat.interval_time >= datetime(2024, 1, 1, tzinfo=timezone.utc):
+    ...
+```
+
+### `HttpPaginatedResponse.error_code` is an `int`
+
+**This change is silent: comparing `error_code` with a string, as in `== '40101'`, now
+returns `False` rather than raising.** The `error_code` of the response that `request()`
+returns is now the `int` parsed from the `X-Ably-Errorcode` header, where 3.x returned the
+header's string. It is still `None` when the header is absent. Compare it with an `int`:
+
+Example 3.x code:
+```python
+if response.error_code == '40101':
+    ...
+```
+
+Example 4.0.0 code:
+```python
+if response.error_code == 40101:
+    ...
+```
+
+### `HttpPaginatedResponse.headers` is a read-only mapping
+
+The `headers` of the response that `request()` returns is now a read-only mapping from header
+name to value, looked up case-insensitively, where 3.x returned a list of `(name, value)`
+pairs. As before, names are lower-cased and a header sent more than once has its values
+joined with `', '`. Iterating over the mapping itself now yields only the names, so look a
+header up by name, or iterate over `.items()`:
+
+Example 3.x code:
+```python
+for name, value in response.headers:
+    ...
+```
+
+Example 4.0.0 code:
+```python
+request_id = response.headers.get('X-Request-Id')
+
+for name, value in response.headers.items():
+    ...
+```
+
+### Push removals return nothing, and results no longer expose the HTTP response
+
+`remove` and `remove_where` on `push.admin.device_registrations` and
+`push.admin.channel_subscriptions` now return `None`, where 3.x returned the underlying HTTP
+response. A removal that fails raises an `AblyException`, so there is no status code to check:
+
+Example 3.x code:
+```python
+response = await rest.push.admin.device_registrations.remove(device_id)
+assert response.status_code == 204
+```
+
+Example 4.0.0 code:
+```python
+await pubsub_http_client.push.admin.device_registrations.remove(device_id)
+```
+
+For the same reason a `PaginatedResult` no longer has a `response` attribute. The
+`HttpPaginatedResponse` that `request()` returns keeps `status_code`, `success`,
+`error_code`, `error_message` and `headers`, so read those instead of `response`.
+
+### `DeviceDetails.push` is a `DevicePushDetails`
+
+`DeviceDetails.push` is now a `DevicePushDetails`, with `recipient`, `state` and
+`error_reason` attributes, where 3.x returned the dict the server sent. `error_reason` is an
+`AblyException`. Indexing it, as in `device.push['recipient']`, now raises `TypeError`.
+Read the attributes instead. A device can still be built or saved with `push` given as a
+dict:
+
+Example 3.x code:
+```python
+token = device.push['recipient']['deviceToken']
+```
+
+Example 4.0.0 code:
+```python
+token = device.push.recipient['deviceToken']
+```
+
+`DeviceDetails` and `PushChannelSubscription` now also ignore fields they do not know when
+decoding a dict, where 3.x raised `TypeError` on any field the server added. A misspelled key
+in a dict you pass to `save()` is therefore ignored rather than rejected.
+
 ## Version 2.x to 3.0.0
 
 The 3.0.0 version of ably-python introduces several breaking changes to improve the realtime experience and align the API with the Ably specification. These include:

ably/pubsub/http/push.py

@@ -116,7 +116,7 @@ async def remove(self, device_id: str):
         - `device_id`: the id of the device
         """
         path = f'/push/deviceRegistrations/{device_id}'
-        return await self.ably.http.delete(path)
+        await self.ably.http.delete(path)
 
     async def remove_where(self, **params):
         """Deletes the registered devices identified by the given parameters.
@@ -125,7 +125,7 @@ async def remove_where(self, **params):
         - `**params`: the parameters that identify the devices to remove
         """
         path = '/push/deviceRegistrations' + format_params(params)
-        return await self.ably.http.delete(path)
+        await self.ably.http.delete(path)
 
 
 class PushChannelSubscriptions:
@@ -181,7 +181,7 @@ async def remove(self, subscription: dict):
         """
         subscription = PushChannelSubscription.factory(subscription)
         params = subscription.as_dict()
-        return await self.remove_where(**params)
+        await self.remove_where(**params)
 
     async def remove_where(self, **params):
         """Deletes the subscriptions identified by the given parameters.
@@ -190,4 +190,4 @@ async def remove_where(self, **params):
         - `**params`: the parameters that identify the subscriptions to remove
         """
         path = '/push/channelSubscriptions' + format_params(**params)
-        return await self.ably.http.delete(path)
+        await self.ably.http.delete(path)

ably/pubsub/realtime/presence.py

@@ -9,13 +9,13 @@
 
 import asyncio
 import logging
-from datetime import datetime, timezone
 from typing import TYPE_CHECKING, Any
 
 from ably.pubsub.realtime.connection import ConnectionState
 from ably.pubsub.realtime.presencemap import PresenceMap
 from ably.pubsub.types.channelstate import ChannelState, ChannelStateChange
 from ably.pubsub.types.presence import PresenceAction, PresenceMessage
+from ably.pubsub.util.clock import select_clock
 from ably.pubsub.util.eventemitter import EventEmitter
 from ably.pubsub.util.exceptions import AblyException
 
@@ -69,6 +69,7 @@ def __init__(self, channel: RealtimeChannel):
         super().__init__()
         self.channel = channel
         self.sync_complete = False
+        self._clock = select_clock(channel.ably.options)
 
         # RTP2: Main presence map keyed by memberKey (connectionId:clientId)
         self.members = PresenceMap(
@@ -583,7 +584,7 @@ def set_presence(
                     connection_id=member.connection_id,
                     data=member.data,
                     encoding=member.encoding,
-                    timestamp=datetime.now(timezone.utc)
+                    timestamp=self._clock.now_ms()
                 )
                 broadcast_messages.append(synthesized_leave)
 
@@ -717,7 +718,7 @@ def _synthesize_leaves(self, members: list[PresenceMessage]) -> None:
                 connection_id=member.connection_id,
                 data=member.data,
                 encoding=member.encoding,
-                timestamp=datetime.now(timezone.utc)
+                timestamp=self._clock.now_ms()
             )
             self._subscriptions._emit('leave', synthesized_leave)
 

ably/pubsub/request/paginatedresult.py

@@ -1,5 +1,6 @@
 import calendar
 import logging
+from collections.abc import Mapping
 from urllib.parse import urlencode
 
 from ably.pubsub.request.http import Request
@@ -42,6 +43,31 @@ def format_params(params=None, direction=None, start=None, end=None, limit=None,
     return '?' + urlencode(params) if params else ''
 
 
+class ResponseHeaders(Mapping):
+    """The headers of a response, read-only and looked up by name case-insensitively (HP8).
+
+    Names iterate lower-cased, and a header sent more than once reads as its values joined
+    with ', '.
+    """
+
+    def __init__(self, headers):
+        self.__headers = {name.lower(): value for name, value in headers.items()}
+
+    def __getitem__(self, name):
+        if not isinstance(name, str):
+            raise KeyError(name)
+        return self.__headers[name.lower()]
+
+    def __iter__(self):
+        return iter(self.__headers)
+
+    def __len__(self):
+        return len(self.__headers)
+
+    def __repr__(self):
+        return f'{type(self).__name__}({self.__headers!r})'
+
+
 class PaginatedResult:
     def __init__(self, http, items, content_type, rel_first, rel_next,
                  response_processor, response):
@@ -51,7 +77,6 @@ def __init__(self, http, items, content_type, rel_first, rel_next,
         self.__rel_first = rel_first
         self.__rel_next = rel_next
         self.__response_processor = response_processor
-        self.response = response
 
     @property
     def items(self):
@@ -113,9 +138,16 @@ async def paginated_query_with_request(cls, http, request, response_processor,
 
 
 class HttpPaginatedResponse(PaginatedResult):
+    def __init__(self, http, items, content_type, rel_first, rel_next,
+                 response_processor, response):
+        super().__init__(http, items, content_type, rel_first, rel_next,
+                         response_processor, response)
+        self.__status_code = response.status_code
+        self.__headers = ResponseHeaders(response.headers)
+
     @property
     def status_code(self):
-        return self.response.status_code
+        return self.__status_code
 
     @property
     def success(self):
@@ -124,12 +156,13 @@ def success(self):
 
     @property
     def error_code(self):
-        return self.response.headers.get('X-Ably-Errorcode')
+        error_code = self.__headers.get('X-Ably-Errorcode')
+        return int(error_code) if error_code is not None else None
 
     @property
     def error_message(self):
-        return self.response.headers.get('X-Ably-Errormessage')
+        return self.__headers.get('X-Ably-Errormessage')
 
     @property
     def headers(self):
-        return list(self.response.headers.items())
+        return self.__headers

ably/pubsub/types/channelsubscription.py

@@ -3,6 +3,8 @@
 
 class PushChannelSubscription:
 
+    _fields = ['channel', 'device_id', 'client_id', 'app_id']
+
     def __init__(self, channel, device_id=None, client_id=None, app_id=None):
         if not device_id and not client_id:
             raise ValueError('missing expected device or client id')
@@ -32,10 +34,8 @@ def app_id(self):
         return self.__app_id
 
     def as_dict(self):
-        keys = ['channel', 'device_id', 'client_id', 'app_id']
-
         obj = {}
-        for key in keys:
+        for key in self._fields:
             value = getattr(self, key)
             if value is not None:
                 key = case.snake_to_camel(key)
@@ -46,7 +46,8 @@ def as_dict(self):
     @classmethod
     def from_dict(cls, obj):
         obj = {case.camel_to_snake(key): value for key, value in obj.items()}
-        return cls(**obj)
+        # Fields this version does not model are ignored, so the server can add them
+        return cls(**{key: value for key, value in obj.items() if key in cls._fields})
 
     @classmethod
     def from_array(cls, array):