Skip to content

fix(basket): use direct delivery when self-hosting - #815

Merged
izadoesdev merged 2 commits into
stagingfrom
codex/selfhost-direct-delivery
Sep 17, 2026
Merged

izadoesdev merged 2 commits into
stagingfrom
codex/selfhost-direct-delivery

Conversation

@izadoesdev

@izadoesdev izadoesdev commented Sep 16, 2026

Copy link
Copy Markdown
Member

SELFHOST=true makes Basket, Links, and Uptime operate without Kafka, even when copied environment settings contain a broker. Links reuses its acknowledged ClickHouse fallback; Uptime writes directly to ClickHouse through its existing durable delivery worker. Health checks follow the active dependencies.

Hosted Kafka behavior and protection against ambiguous delivery retries are preserved. Uptime direct delivery is at least once: failed writes retry, and a timeout can produce a duplicate. No schema changes or new services.

Validation: Basket 507, Links 27, and Uptime 58 tests; root lint and all 33 typecheck tasks; pre-push tests. Combined self-host smoke verified all three health endpoints with an invalid broker, a real synthetic Uptime write/read, and HTTP 503 from all three when ClickHouse stopped.

Depends on #806 for SELFHOST forwarding through Turbo. Complements #804 and #805. Overlaps #721 only in Links producer/tests; based directly on staging, with no code taken from that feature branch.


Summary by cubic

When SELFHOST=true, Basket, Links, and Uptime deliver directly to ClickHouse and ignore copied broker settings; health checks follow the active delivery path and hosted Kafka behavior is unchanged.

  • Basket marks Redpanda disabled instead of checking it, so it no longer degrades overall health; Uptime pings ClickHouse in self-host mode.
  • Uptime writes with async insert off and a 10s timeout; in self-host mode, failed delivery jobs stay pending past the retry limit until storage recovers instead of being dropped.
  • Links reuses its acknowledged ClickHouse fallback.
  • Delivery stays at least once, so a timeout can duplicate an Uptime write; no schema changes or new services are required.

Written for commit 70d20cb. Summary will update on new commits.

Review in cubic

@vercel

vercel Bot commented Sep 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
databuddy-status Ready Ready Preview Sep 17, 2026 7:19am UTC
2 Skipped Deployments
Project Deployment Actions Updated
dashboard Skipped Skipped Sep 17, 2026 7:19am UTC
documentation Skipped Skipped Sep 17, 2026 7:19am UTC

@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: f0faea93-e32f-477a-8d93-9341cbc5855d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@unkey-deploy

unkey-deploy Bot commented Sep 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Unkey Deploy

Name Status Preview Inspect Updated (UTC)
links (preview) Ready Visit Preview Inspect Sep 17, 2026 7:18am

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no blocking or non-blocking actionable issues remain.

Summary

This PR makes Basket, Links, and Uptime bypass Kafka when self-hosting and aligns their health checks with the active delivery dependencies. The latest changes also resolve both previous review findings.

  • Basket and Links use direct ClickHouse delivery when SELFHOST=true, even if broker configuration remains present.
  • Uptime directly inserts completed checks into ClickHouse and reports ClickHouse health in self-hosted mode.
  • Failed self-hosted Uptime deliveries are returned to BullMQ’s delayed set indefinitely until storage recovers.
  • Hosted Kafka delivery and its bounded retry policy remain unchanged.
  • The Basket test now uses a properly typed Kafka producer instead of casting through unknown.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Accepted event or completed uptime check] --> B{SELFHOST}
    B -->|true| C[Direct ClickHouse delivery]
    B -->|false| D[Kafka / Redpanda]
    D --> E[ClickHouse]
    C -->|Uptime failure| F[BullMQ delayed job]
    F -->|30-second retry| C
    C -->|success| G[Durable analytics history]
    E --> G
Loading

Reviews (2) · Last reviewed commit: "fix(uptime): retain self-hosted checks u..."

Comment thread apps/uptime/src/lib/producer.ts
Comment thread apps/basket/src/lib/producer.delivery.test.ts Outdated
@izadoesdev

Copy link
Copy Markdown
Member Author

@greptileai review this

@izadoesdev
izadoesdev merged commit 30547ae into staging Sep 17, 2026
20 checks passed
@izadoesdev
izadoesdev deleted the codex/selfhost-direct-delivery branch September 17, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant