Limit pending poll notification tasks

CHANGES.md

@@ -67,6 +67,12 @@ To be released.
     successor account at the top of the profile card and muting the old
     profile's visual treatment.
 
+ -  Limited pending poll notification messages to 100 across the shared task
+    queue, including delayed wakeups and retries. Repeated schedules coalesce,
+    earlier wakeups take priority at the limit, and deferred work is recovered
+    after expiry. Delayed poll schedules no longer allocate worker timers.
+    [[#653], [#663]]
+
  -  Replaced the undocumented `SEONBI_URL` integration with the embedded
     [Gukhanmun] Node-API binding.  Set `GUKHANMUN` to a comma-separated list of
     locale patterns, such as `ko,ko-*`, to add Hangul readings to Hanja in
@@ -155,7 +161,9 @@ To be released.
 [#648]: https://github.com/fedify-dev/hollo/pull/648
 [#649]: https://github.com/fedify-dev/hollo/pull/649
 [#650]: https://github.com/fedify-dev/hollo/pull/650
+[#653]: https://github.com/fedify-dev/hollo/issues/653
 [#660]: https://github.com/fedify-dev/hollo/pull/660
+[#663]: https://github.com/fedify-dev/hollo/pull/663
 
 
 Version 0.9.22

docs/src/content/docs/install/workers.mdx

@@ -120,6 +120,40 @@ interrupted work remains recoverable. Stop older nodes before upgrading and
 run compatible task definitions on every node.
 
 
+The shared database stores at most 100 pending poll messages, including delayed
+wakeups and retries, with one wakeup per poll for new schedules. Repeated votes
+and expiry updates coalesce: an earlier expiry advances the existing wakeup;
+a later expiry is reloaded when that wakeup runs. At the limit, an earlier
+wakeup replaces the latest one. Deferred or displaced polls are recovered after
+expiry. Bursts can also defer admission when its database lock cannot be
+acquired within one second. These limits apply across nodes, rather than per
+worker. They bound stored work, not notifications or running handlers.
+
+Delayed poll wakeups use the queue's five-second polling interval rather than
+allocating a timer in each worker. Recovery still pauses at 200 shared ready
+messages. The limit of 100 matches one recovery page; it is not a throughput
+guarantee. Stop all older producers before upgrading. Legacy queued messages
+count toward the limit, but an existing backlog above 100 drains gradually;
+new admissions do not increase that backlog.
+
+The `hollo.poll-notifications.queue` logger reports capacity deferrals,
+displaced wakeups and lock timeouts at warning level, at most once per reason
+per minute per process. Inspect storage pressure with the query below.
+Enqueue counters include coalesced and deferred attempts, so they do not
+measure stored messages.
+
+~~~~ sql
+SELECT count(*) AS queued,
+       count(*) FILTER (WHERE created + delay <= CURRENT_TIMESTAMP) AS ready,
+       count(*) FILTER (WHERE created + delay > CURRENT_TIMESTAMP) AS delayed,
+       count(*) FILTER (
+         WHERE message->>'type' = 'task'
+           AND message->>'taskName' = 'hollo.poll-notification.v1'
+       ) AS pending_polls
+FROM hollo_task_message_v1;
+~~~~
+
+
 Docker Compose setup
 --------------------
 

docs/src/content/docs/ja/install/workers.mdx

@@ -110,6 +110,37 @@ Holloは6つの主要コンポーネントで構成されています:
 タスク定義を使ってください。
 
 
+共有データベースの待機中投票メッセージは、遅延と再試行を含め最大100件です。
+新しい予約は投票ごとに1件にまとめます。繰り返しの投票や期限更新を統合し、
+期限が早まると既存の予約を前倒しします。期限の延長は実行時に再取得します。
+上限では、早い予約が最も遅い予約を置き換えます。延期・置換された投票は期限後に
+復旧します。集中した負荷で入場ロックを1秒以内に取得できない場合も延期します。
+この上限は全ノードで共有し、実行中のハンドラーや通知件数ではなく保存件数を
+制限します。
+
+遅延予約はワーカーごとのタイマーではなく、キューの5秒間隔の確認を使います。
+復旧は共有キューの即時実行メッセージが200件になると引き続き停止します。
+100件は復旧1ページ分であり、処理能力の保証ではありません。更新前に旧版の
+生成ノードをすべて停止してください。旧メッセージも上限に数えますが、既に100件を
+超えるバックログは徐々に減少します。新規予約はその件数を増やしません。
+
+`hollo.poll-notifications.queue` ロガーは容量による延期、予約の置換、ロックの
+タイムアウトを warning レベルで記録します。理由ごと・プロセスごとに毎分最大1回
+です。保存負荷は以下のSQLで確認できます。enqueue カウンターは統合・延期された
+試行も含むため、保存メッセージ数ではありません。
+
+~~~~ sql
+SELECT count(*) AS queued,
+       count(*) FILTER (WHERE created + delay <= CURRENT_TIMESTAMP) AS ready,
+       count(*) FILTER (WHERE created + delay > CURRENT_TIMESTAMP) AS delayed,
+       count(*) FILTER (
+         WHERE message->>'type' = 'task'
+           AND message->>'taskName' = 'hollo.poll-notification.v1'
+       ) AS pending_polls
+FROM hollo_task_message_v1;
+~~~~
+
+
 Docker Compose設定
 ------------------
 

docs/src/content/docs/ko/install/workers.mdx

@@ -109,6 +109,38 @@ Hollo는 여섯 가지 주요 구성 요소로 이루어져 있습니다:
 업그레이드 전 기존 노드를 중지하고 모든 노드에 호환되는 태스크 정의를 배포하세요.
 
 
+공유 데이터베이스에는 지연 wakeup과 재시도를 포함해 대기 투표 메시지를 최대
+100개 저장하며, 새 예약은 투표당 하나의 wakeup으로 합칩니다. 반복 참여와 만료
+변경도 합쳐지고, 만료가 앞당겨지면 기존 wakeup을 앞당깁니다. 만료가 늦춰지면
+wakeup 실행 시 현재 만료 시각을 다시 읽습니다. 상한에서는 더 이른 wakeup이
+가장 늦은 메시지를 대체합니다. 보류되거나 밀려난 투표는 만료 후 복구됩니다. 부하가
+몰려 입장 잠금을 1초 안에 얻지 못해도 예약을 보류할 수 있습니다. 이 상한은
+워커별이 아니라 모든 노드에 걸쳐 적용되며, 실행 중 handler나 알림 수가 아닌
+저장된 작업 수를 제한합니다.
+
+지연 예약은 워커마다 타이머를 할당하는 대신 큐의 5초 주기 확인을 사용합니다.
+복구는 공유 ready 메시지가 200개이면 여전히 일시 중지합니다. 100개 상한은
+복구 한 페이지 크기이며 처리량 보장은 아닙니다. 업그레이드 전에 모든 기존
+생산자를 중지하세요. 기존 메시지도 상한에 포함되지만 이미 100개를 넘은 backlog는
+점차 줄어듭니다. 새 예약으로 그 backlog가 더 커지지는 않습니다.
+
+`hollo.poll-notifications.queue` 로거는 상한에 따른 보류, wakeup 대체와 잠금
+시간 초과를 warning 수준으로 원인별·프로세스별 매분 최대 한 번 기록합니다.
+아래 SQL로 저장 부하를 확인할 수 있습니다. enqueue 카운터에는 합쳐지거나
+보류된 시도도 포함되므로 저장 메시지 수를 나타내지 않습니다.
+
+~~~~ sql
+SELECT count(*) AS queued,
+       count(*) FILTER (WHERE created + delay <= CURRENT_TIMESTAMP) AS ready,
+       count(*) FILTER (WHERE created + delay > CURRENT_TIMESTAMP) AS delayed,
+       count(*) FILTER (
+         WHERE message->>'type' = 'task'
+           AND message->>'taskName' = 'hollo.poll-notification.v1'
+       ) AS pending_polls
+FROM hollo_task_message_v1;
+~~~~
+
+
 Docker Compose 설정
 -------------------
 

docs/src/content/docs/zh-cn/install/workers.mdx

@@ -95,6 +95,33 @@ Hollo由六个主要组件组成：
 事务结束，中断的工作可恢复。升级前停止旧节点，并在所有节点使用兼容的任务定义。
 
 
+共享数据库最多保存 100 条待处理投票消息，包括延迟唤醒和重试。新预约按每个
+投票合并为一条消息。重复投票和到期更新会合并；到期提前时，现有唤醒也提前；
+到期延后时，执行时重新读取。达到上限后，更早的唤醒替换最晚的消息。被延期或
+替换的投票在到期后恢复。突发负载导致入队锁无法在一秒内取得时也可能延期。
+上限由所有节点共享，而非按工作器计算，限制保存的任务数而非通知或运行的处理器。
+
+延迟唤醒使用队列每五秒的轮询，不为每个工作器分配计时器。共享就绪消息达到
+200 条时，恢复仍会暂停。100 条上限等于一页恢复批次，并不保证吞吐量。升级前
+停止所有旧的生产节点。旧消息也计入上限，但已超过 100 条的积压会逐渐减少；
+新预约不会增加该积压。
+
+`hollo.poll-notifications.queue` 日志以 warning 级别报告容量延期、唤醒替换和
+锁超时，每个原因、每个进程每分钟最多一次。用下面的 SQL 观察存储压力。入队计数
+包含合并及延期的尝试，不代表保存消息数。
+
+~~~~ sql
+SELECT count(*) AS queued,
+       count(*) FILTER (WHERE created + delay <= CURRENT_TIMESTAMP) AS ready,
+       count(*) FILTER (WHERE created + delay > CURRENT_TIMESTAMP) AS delayed,
+       count(*) FILTER (
+         WHERE message->>'type' = 'task'
+           AND message->>'taskName' = 'hollo.poll-notification.v1'
+       ) AS pending_polls
+FROM hollo_task_message_v1;
+~~~~
+
+
 Docker Compose设置
 ------------------
 

docs/src/content/docs/zh-tw/install/workers.mdx

@@ -95,6 +95,33 @@ Hollo 由六個主要元件組成：
 交易結束，中斷的工作可復原。升級前停止舊節點，並在所有節點使用相容的任務定義。
 
 
+共享資料庫最多儲存 100 條待處理投票訊息，包括延遲喚醒與重試。新預約按每個
+投票合併為一條訊息。重複投票與到期更新會合併；到期提前時，現有喚醒也提前；
+到期延後時，執行時重新讀取。達到上限後，更早的喚醒取代最晚的訊息。被延後或
+取代的投票在到期後復原。突發負載導致入列鎖無法在一秒內取得時也可能延後。
+上限由所有節點共享，而非按工作節點計算，限制儲存的任務數而非通知或執行的處理器。
+
+延遲喚醒使用佇列每五秒的輪詢，不為每個工作節點分配計時器。共享就緒訊息達到
+200 條時，復原仍會暫停。100 條上限等於一頁復原批次，並不保證吞吐量。升級前
+停止所有舊的生產節點。舊訊息也計入上限，但已超過 100 條的積壓會逐漸減少；
+新預約不會增加該積壓。
+
+`hollo.poll-notifications.queue` 日誌以 warning 級別報告容量延後、喚醒取代與
+鎖逾時，每個原因、每個程序每分鐘最多一次。用下面的 SQL 觀察儲存壓力。入列計數
+包含合併及延後的嘗試，不代表儲存訊息數。
+
+~~~~ sql
+SELECT count(*) AS queued,
+       count(*) FILTER (WHERE created + delay <= CURRENT_TIMESTAMP) AS ready,
+       count(*) FILTER (WHERE created + delay > CURRENT_TIMESTAMP) AS delayed,
+       count(*) FILTER (
+         WHERE message->>'type' = 'task'
+           AND message->>'taskName' = 'hollo.poll-notification.v1'
+       ) AS pending_polls
+FROM hollo_task_message_v1;
+~~~~
+
+
 Docker Compose 設定
 -------------------