Skip to content

Commit b79094f

Browse files
committed
refactor(sync): replace legacy examples with structured sync scenarios and update README
2 parents bb8ec30 + 00b5905 commit b79094f

10 files changed

Lines changed: 1349 additions & 24 deletions

README.md

Lines changed: 243 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,243 @@
1-
# sync
2-
Offline-first sync engine (WAL, outbox, retries).
1+
# Vix Sync
2+
3+
**Durable. Offline-first. Deterministic.**
4+
5+
Vix Sync is the core synchronization engine of Vix.cpp.
6+
7+
It guarantees that every operation:
8+
- is persisted before being sent
9+
- is never lost
10+
- is retried deterministically
11+
- converges to a consistent state
12+
13+
---
14+
15+
## Why Vix Sync exists
16+
17+
Real-world systems are not reliable:
18+
- network drops
19+
- servers restart
20+
- requests fail halfway
21+
22+
Traditional systems:
23+
- lose data
24+
- duplicate requests
25+
- become inconsistent
26+
27+
Vix Sync solves this with a simple rule:
28+
29+
> Write locally. Persist first. Sync later.
30+
31+
---
32+
33+
## Core principles
34+
35+
1. **Durability first**
36+
- Every operation is stored before any network call
37+
38+
2. **Deterministic retries**
39+
- Retry logic is reproducible and replayable
40+
41+
3. **Offline-first**
42+
- The system works without network
43+
44+
4. **Safe convergence**
45+
- Eventually, all nodes reach a consistent state
46+
47+
---
48+
49+
## Architecture
50+
51+
Vix Sync is composed of small deterministic building blocks:
52+
53+
```text
54+
Local Write
55+
56+
WAL (Write-Ahead Log)
57+
58+
Outbox (durable queue)
59+
60+
SyncWorker
61+
62+
Transport (HTTP / P2P / custom)
63+
64+
Done / Retry / Failed
65+
```
66+
67+
---
68+
69+
## Components
70+
71+
### 1. Operation
72+
73+
A durable unit of work.
74+
75+
```cpp
76+
struct Operation {
77+
std::string id;
78+
std::string kind;
79+
std::string target;
80+
std::string payload;
81+
};
82+
```
83+
84+
### 2. WAL (Write-Ahead Log)
85+
86+
Append-only log:
87+
88+
- crash-safe
89+
- replayable
90+
- ordered
91+
92+
```cpp
93+
Wal wal({ .file_path = "./.vix/wal.log" });
94+
95+
wal.append(record);
96+
wal.replay(0, handler);
97+
```
98+
99+
### 3. Outbox
100+
101+
Durable operation queue:
102+
103+
- enqueue
104+
- claim
105+
- complete
106+
- retry
107+
108+
```cpp
109+
auto id = outbox->enqueue(op, now);
110+
outbox->claim(id, now);
111+
outbox->complete(id, now);
112+
```
113+
114+
### 4. RetryPolicy
115+
116+
Deterministic exponential backoff:
117+
118+
```cpp
119+
RetryPolicy retry;
120+
retry.base_delay_ms = 500;
121+
retry.factor = 2.0;
122+
retry.max_attempts = 8;
123+
```
124+
125+
### 5. SyncWorker
126+
127+
Processes operations:
128+
129+
- fetch ready ops
130+
- send via transport
131+
- mark done / fail
132+
133+
### 6. SyncEngine
134+
135+
Orchestrates workers:
136+
137+
```cpp
138+
SyncEngine engine(cfg, outbox, probe, transport);
139+
engine.tick(now);
140+
```
141+
142+
---
143+
144+
## Examples
145+
146+
Run examples with:
147+
148+
```bash
149+
vix run examples/<file>.cpp
150+
```
151+
152+
### Basic
153+
```bash
154+
vix run examples/01_outbox_basic.cpp
155+
```
156+
157+
### Retry & failure
158+
```bash
159+
vix run examples/02_outbox_retry_and_failure.cpp
160+
```
161+
162+
### WAL
163+
```bash
164+
vix run examples/03_wal_append_and_replay.cpp
165+
```
166+
167+
### Worker
168+
```bash
169+
vix run examples/04_sync_worker_success.cpp
170+
```
171+
172+
### Permanent failure
173+
```bash
174+
vix run examples/05_sync_worker_permanent_failure.cpp
175+
```
176+
177+
### Engine requeue
178+
```bash
179+
vix run examples/06_sync_engine_requeue_inflight.cpp
180+
```
181+
182+
### End-to-end (local-first)
183+
```bash
184+
vix run examples/07_local_write_wal_outbox_engine.cpp
185+
```
186+
187+
### Offline to recovery
188+
```bash
189+
vix run examples/08_local_write_offline_then_recover.cpp
190+
```
191+
192+
---
193+
194+
## Key invariant
195+
196+
If the process crashes at any point, the system can recover fully.
197+
198+
Because:
199+
200+
- WAL stores intent
201+
- Outbox stores state
202+
- Retry is deterministic
203+
204+
---
205+
206+
## When to use Vix Sync
207+
208+
Use it when you need:
209+
210+
- offline-first systems
211+
- reliable message delivery
212+
- distributed state convergence
213+
- retry-safe APIs
214+
- edge or unstable network environments
215+
216+
---
217+
218+
## What Vix Sync is NOT
219+
220+
- not a message broker
221+
- not a queue system like Kafka
222+
- not a distributed database
223+
224+
It is:
225+
226+
> a durable sync layer for real-world unreliable systems
227+
228+
---
229+
230+
## Future
231+
232+
- P2P transport
233+
- conflict resolution
234+
- multi-device sync
235+
- edge replication
236+
237+
---
238+
239+
## License
240+
241+
MIT License
242+
© Gaspard Kirira
243+

examples/01_outbox_basic.cpp

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
/**
2+
*
3+
* @file 01_outbox_basic.cpp
4+
* @author Gaspard Kirira
5+
*
6+
* Copyright 2025, Gaspard Kirira.
7+
* All rights reserved.
8+
* https://github.com/vixcpp/vix
9+
*
10+
* Use of this source code is governed by a MIT license
11+
* that can be found in the License file.
12+
*
13+
* Vix.cpp
14+
*
15+
*/
16+
// Run:
17+
// vix run examples/sync/01_outbox_basic.cpp
18+
19+
#include <chrono>
20+
#include <filesystem>
21+
#include <iostream>
22+
#include <memory>
23+
24+
#include <vix/sync/Operation.hpp>
25+
#include <vix/sync/outbox/Outbox.hpp>
26+
#include <vix/sync/outbox/FileOutboxStore.hpp>
27+
28+
static std::int64_t now_ms()
29+
{
30+
using namespace std::chrono;
31+
return duration_cast<milliseconds>(
32+
steady_clock::now().time_since_epoch())
33+
.count();
34+
}
35+
36+
static void reset_dir(const std::filesystem::path &dir)
37+
{
38+
std::error_code ec;
39+
std::filesystem::remove_all(dir, ec);
40+
std::filesystem::create_directories(dir, ec);
41+
}
42+
43+
int main()
44+
{
45+
using namespace vix::sync;
46+
using namespace vix::sync::outbox;
47+
48+
const std::filesystem::path dir = "./build/examples/sync/basic";
49+
reset_dir(dir);
50+
51+
auto store = std::make_shared<FileOutboxStore>(FileOutboxStore::Config{
52+
.file_path = dir / "outbox.json",
53+
.pretty_json = true});
54+
55+
auto outbox = std::make_shared<Outbox>(
56+
Outbox::Config{
57+
.owner = "example-basic"},
58+
store);
59+
60+
Operation op;
61+
op.kind = "http.post";
62+
op.target = "/api/messages";
63+
op.payload = R"({"text":"hello"})";
64+
65+
const auto t0 = now_ms();
66+
const std::string id = outbox->enqueue(op, t0);
67+
68+
std::cout << "enqueued: " << id << "\n";
69+
70+
auto ready = outbox->peek_ready(t0, 10);
71+
std::cout << "ready count: " << ready.size() << "\n";
72+
73+
const bool claimed = outbox->claim(id, t0 + 1);
74+
std::cout << "claimed: " << (claimed ? "yes" : "no") << "\n";
75+
76+
const bool done = outbox->complete(id, t0 + 2);
77+
std::cout << "completed: " << (done ? "yes" : "no") << "\n";
78+
79+
auto saved = store->get(id);
80+
if (saved)
81+
{
82+
std::cout << "final status: "
83+
<< (saved->status == OperationStatus::Done ? "Done" : "Other")
84+
<< "\n";
85+
}
86+
87+
return 0;
88+
}

0 commit comments

Comments
 (0)