docs: update CLAUDE.md, README, CHANGELOG for v1.3

devwhodevs · devwhodevs · commit 1038bcc5a78c · 2026-03-26T22:27:53.000+02:00
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,23 @@
 # Changelog
 
+## v1.3.0 — HTTP/REST Transport (2026-03-26)
+
+### Added
+- **HTTP REST API** (`http.rs`) — axum-based HTTP server alongside MCP, enabled via `engraph serve --http`
+- **20 REST endpoints** mirroring all 19 MCP tools + update-metadata
+- **API key authentication** — `eg_` prefixed keys with read/write permission levels
+- **Rate limiting** — configurable per-key token bucket (requests/minute)
+- **CORS** — configurable allowed origins for web-based agents
+- **Graceful shutdown** — CancellationToken coordinates MCP + HTTP + watcher exit
+- **API key management CLI** — `engraph configure --add-api-key/--list-api-keys/--revoke-api-key`
+- **`--no-auth` mode** — local development without API keys (127.0.0.1 only)
+
+### Changed
+- `engraph serve` gains `--http`, `--port`, `--host`, `--no-auth` flags
+- Module count: 23 → 24
+- Test count: 361 → 385
+- New dependencies: axum, tower-http, tower, rand, tokio-util
+
 ## v1.2.0 — Temporal Search (2026-03-26)
 
 ### Added
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -4,7 +4,7 @@ Local knowledge graph + intelligence layer for Obsidian vaults. Rust CLI + MCP s
 
 ## Architecture
 
-Single binary with 23 modules behind a lib crate:
+Single binary with 24 modules behind a lib crate:
 
 - `config.rs` — loads `~/.engraph/config.toml` and `vault.toml`, merges CLI args, provides `data_dir()`. Includes `intelligence: Option<bool>`, `[models]` section for model overrides, `[obsidian]` section (CLI path, enabled flag), and `[agents]` section (registered AI agent names). `Config::save()` writes back to disk.
 - `chunker.rs` — smart chunking with break-point scoring algorithm. Finds optimal split points considering headings, code fences, blank lines, and thematic breaks. `split_oversized_chunks()` handles token-aware secondary splitting with overlap
@@ -23,14 +23,15 @@ Single binary with 23 modules behind a lib crate:
 - `writer.rs` — write pipeline orchestrator. 5-step pipeline: resolve tags (fuzzy match + register new), discover links (exact + fuzzy), place in folder, atomic file write (temp + rename), and index update. Supports create, append, update_metadata, move_note, archive, unarchive, edit (section-level replace/prepend/append), rewrite (full content with frontmatter preservation), edit_frontmatter (granular set/remove/add_tag/remove_tag/add_alias/remove_alias ops), and delete (soft archive or hard permanent) operations with mtime-based conflict detection and crash recovery via temp file cleanup
 - `watcher.rs` — file watcher for `engraph serve`. OS thread producer (notify-debouncer-full, 2s debounce) sends `Vec<WatchEvent>` over tokio::mpsc to async consumer task. Two-pass batch processing: mutations (index_file/remove_file/rename_file) then edge rebuild. Move detection via content hash matching. Placement correction on file moves. Centroid adjustment on file add/remove. Startup reconciliation via `run_index_shared`. `recent_writes` map coordination with MCP server to prevent double re-indexing of files written through the write pipeline
 - `serve.rs` — MCP stdio server via rmcp SDK. Exposes 19 tools: 8 read (search, read, read_section, list, vault_map, who, project, context) + 10 write (create, append, update_metadata, move_note, archive, unarchive, edit, rewrite, edit_frontmatter, delete) + 1 diagnostic (health). `edit_frontmatter` replaces `update_metadata` for granular frontmatter mutations. EngraphServer struct with Arc+Mutex wrapping for async handlers. Loads intelligence models (orchestrator + reranker) when enabled, wires into `search_with_intelligence`. Spawns file watcher on startup. CLI events table provides audit log for write operations. `recent_writes` map prevents double re-indexing of MCP-written files
+- `http.rs` — axum-based HTTP REST API server, enabled via `engraph serve --http`. 20 REST endpoints mirroring all 19 MCP tools + update-metadata. API key authentication with `eg_` prefixed keys and read/write permission levels. Per-key token bucket rate limiting (configurable requests/minute). CORS with configurable allowed origins for web-based agents. `--no-auth` mode for local development (127.0.0.1 only). Graceful shutdown via `CancellationToken` coordinating MCP + HTTP + watcher exit
 - `graph.rs` — vault graph agent. Extracts wikilink targets, expands search results by following graph connections 1-2 hops. Relevance filtering via FTS5 term check and shared tags
 - `profile.rs` — vault profile detection. Auto-detects PARA/Folders/Flat structure, vault type (Obsidian/Logseq/Plain), wikilinks, frontmatter, tags. Content-based role detection for people/daily/archive folders by content patterns (not just names). Writes/loads `vault.toml`
 - `store.rs` — SQLite persistence. Tables: `meta`, `files` (with docid, created_by), `chunks` (with vector BLOBs), `chunks_fts` (FTS5), `edges` (vault graph), `tombstones`, `tag_registry`, `folder_centroids`, `placement_corrections`, `link_skiplist` (reserved), `llm_cache` (orchestrator result cache), `cli_events` (audit log for CLI operations). `vec_chunks` virtual table (sqlite-vec) for KNN search. Dynamic embedding dimension stored in meta. `has_dimension_mismatch()` and `reset_for_reindex()` for migration. Enhanced `resolve_file()` with fuzzy Levenshtein matching as final fallback
 - `indexer.rs` — orchestrates vault walking (via `ignore` crate for `.gitignore` support), diffing, chunking, embedding, writes to store + sqlite-vec + FTS5, vault graph edge building (wikilinks + people detection), and folder centroid computation. Exposes `index_file`, `remove_file`, `rename_file` as public per-file functions. `run_index_shared` accepts external store/embedder for watcher FullRescan. Dimension migration on model change.
 - `temporal.rs` — temporal search lane. Extracts note dates from frontmatter `date:` field or `YYYY-MM-DD` filename patterns. Heuristic date parsing for natural language ("today", "yesterday", "last week", "this month", "recent", month names, ISO dates, date ranges). Smooth decay scoring for files near but outside target date range. Provides `extract_note_date()` for indexing and `score_temporal()` + `parse_date_range_heuristic()` for search
 - `search.rs` — hybrid search orchestrator. `search_with_intelligence()` runs the full pipeline: orchestrate (intent + expansions) → 5-lane RRF retrieval (semantic + FTS5 + graph + reranker + temporal) per expansion → two-pass RRF fusion. `search_internal()` is a thin wrapper without intelligence models. Adaptive lane weights per query intent including temporal (1.5 weight for time-aware queries). Results display normalized confidence percentages (0-100%) instead of raw RRF scores.
 
-`main.rs` is a thin clap CLI (async via `#[tokio::main]`). Subcommands: `index` (with progress bar), `search` (with `--explain`, loads intelligence models when enabled), `status` (shows intelligence state + date coverage stats), `clear`, `init` (intelligence onboarding prompt, detects Obsidian CLI + AI agents), `configure` (`--enable-intelligence`, `--disable-intelligence`, `--model`, `--obsidian-cli`, `--no-obsidian-cli`, `--agent`), `models`, `graph` (show/stats), `context` (read/list/vault-map/who/project/topic), `write` (create/append/update-metadata/move/edit/rewrite/edit-frontmatter/delete), `serve` (MCP stdio server with file watcher + intelligence).
+`main.rs` is a thin clap CLI (async via `#[tokio::main]`). Subcommands: `index` (with progress bar), `search` (with `--explain`, loads intelligence models when enabled), `status` (shows intelligence state + date coverage stats), `clear`, `init` (intelligence onboarding prompt, detects Obsidian CLI + AI agents), `configure` (`--enable-intelligence`, `--disable-intelligence`, `--model`, `--obsidian-cli`, `--no-obsidian-cli`, `--agent`, `--add-api-key`, `--list-api-keys`, `--revoke-api-key`), `models`, `graph` (show/stats), `context` (read/list/vault-map/who/project/topic), `write` (create/append/update-metadata/move/edit/rewrite/edit-frontmatter/delete), `serve` (MCP stdio server with file watcher + intelligence + optional `--http`/`--port`/`--host`/`--no-auth` for HTTP REST API).
 
 ## Key patterns
 
@@ -69,12 +70,17 @@ Single vault only. Re-indexing a different vault path triggers a confirmation pr
 - `ignore` (0.4) — vault walking with `.gitignore` support
 - `rusqlite` (0.32) — bundled SQLite with FTS5 support
 - `rmcp` (1.2) — MCP server SDK for stdio transport
+- `axum` — HTTP framework for REST API server
+- `tower-http` — CORS middleware for axum
+- `tower` — middleware layer utilities
+- `rand` — random API key generation
+- `tokio-util` — `CancellationToken` for graceful multi-server shutdown
 - `notify` (7.0) — cross-platform filesystem notification (FSEvents on macOS, inotify on Linux)
 - `notify-debouncer-full` (0.4) — debouncing + best-effort inode-based rename tracking
 
 ## Testing
 
-- Unit tests in each module (`cargo test --lib`) — 361 tests, no network required
+- Unit tests in each module (`cargo test --lib`) — 385 tests, no network required
 - Integration tests (`cargo test --test integration -- --ignored`) — require GGUF model download
 - Build requires CMake (for llama.cpp C++ compilation)
 
diff --git a/README.md b/README.md
@@ -18,6 +18,7 @@ Plain vector search treats your notes as isolated documents. But knowledge isn't
 
 - **5-lane hybrid search** — semantic embeddings + BM25 full-text + graph expansion + cross-encoder reranking + temporal scoring, fused via [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf). An LLM orchestrator classifies queries and adapts lane weights per intent. Time-aware queries like "what happened last week" or "March 2026 notes" activate the temporal lane automatically.
 - **MCP server for AI agents** — `engraph serve` exposes 19 tools (search, read, section-level editing, frontmatter mutations, vault health, context bundles, note creation) that Claude, Cursor, or any MCP client can call directly.
+- **HTTP REST API** — `engraph serve --http` adds an axum-based HTTP server alongside MCP with 20 REST endpoints, API key authentication, rate limiting, and CORS. Web-based agents and scripts can query your vault with simple `curl` calls.
 - **Section-level editing** — AI agents can read, replace, prepend, or append to specific sections by heading. Full note rewriting with frontmatter preservation. Granular frontmatter mutations (set/remove fields, add/remove tags and aliases).
 - **Vault health diagnostics** — detect orphan notes, broken wikilinks, stale content, and tag hygiene issues. Available as MCP tool and CLI command.
 - **Obsidian CLI integration** — auto-detects running Obsidian and delegates compatible operations. Circuit breaker (Closed/Degraded/Open) ensures graceful fallback.
@@ -51,17 +52,16 @@ Your vault (markdown files)
 │              engraph serve                   │
 │                                             │
 │  MCP Server (stdio) + File Watcher          │
+│  + HTTP REST API (--http, optional)         │
 │                                             │
 │  Search: Orchestrator → 4-lane retrieval    │
 │          → Reranker → Two-pass RRF fusion   │
 │                                             │
-│  19 tools: search, read, read_section,      │
-│  edit, rewrite, edit_frontmatter, delete,   │
-│  health, context, who, project, create...   │
+│  19 MCP tools + 20 REST endpoints           │
 └─────────────────────────────────────────────┘
         │
         ▼
-  Claude / Cursor / any MCP client
+  Claude / Cursor / any MCP client / curl / web agents
 ```
 
 1. **Index** — walks your vault, chunks markdown by headings, embeds with a local GGUF model via llama.cpp (Metal GPU on macOS), stores everything in SQLite with FTS5 + sqlite-vec + a wikilink graph
@@ -129,6 +129,32 @@ engraph serve
 
 Now Claude can search your vault, read notes, build context bundles, and create new notes — all through structured tool calls.
 
+**Enable HTTP REST API:**
+
+```bash
+# Start MCP + HTTP server on port 3030
+engraph serve --http
+
+# Custom port and host
+engraph serve --http --port 8080 --host 0.0.0.0
+
+# Local development without API keys (127.0.0.1 only)
+engraph serve --http --no-auth
+```
+
+**API key management:**
+
+```bash
+# Add a new API key (read or write permission)
+engraph configure --add-api-key
+
+# List existing keys
+engraph configure --list-api-keys
+
+# Revoke a key
+engraph configure --revoke-api-key eg_abc123...
+```
+
 **Enable intelligence (optional, ~1.3GB download):**
 
 ```bash
@@ -234,6 +260,81 @@ engraph context health
 
 Returns orphan notes (no links in or out), broken wikilinks, stale notes, and tag hygiene issues.
 
+## HTTP REST API
+
+`engraph serve --http` adds a full REST API alongside the MCP server, exposing the same capabilities over HTTP for web agents, scripts, and integrations.
+
+**20 endpoints:**
+
+| Method | Endpoint | Permission | Description |
+|--------|----------|------------|-------------|
+| GET | `/api/health-check` | read | Server health check |
+| POST | `/api/search` | read | Hybrid search (semantic + FTS5 + graph + reranker + temporal) |
+| GET | `/api/read/{file}` | read | Read full note content + metadata |
+| GET | `/api/read-section` | read | Read a specific section by heading |
+| GET | `/api/list` | read | List notes with optional tag/folder/created_by filters |
+| GET | `/api/vault-map` | read | Vault structure overview (folders, tags, recent files) |
+| GET | `/api/who/{name}` | read | Person context bundle |
+| GET | `/api/project/{name}` | read | Project context bundle |
+| POST | `/api/context` | read | Rich topic context with token budget |
+| GET | `/api/health` | read | Vault health diagnostics |
+| POST | `/api/create` | write | Create a new note |
+| POST | `/api/append` | write | Append content to existing note |
+| POST | `/api/edit` | write | Section-level editing (replace/prepend/append) |
+| POST | `/api/rewrite` | write | Full note rewrite (preserves frontmatter) |
+| POST | `/api/edit-frontmatter` | write | Granular frontmatter mutations |
+| POST | `/api/move` | write | Move note to different folder |
+| POST | `/api/archive` | write | Soft-delete (archive) a note |
+| POST | `/api/unarchive` | write | Restore archived note |
+| POST | `/api/update-metadata` | write | Update note metadata |
+| POST | `/api/delete` | write | Delete note (soft or hard) |
+
+**Authentication:**
+
+All requests require an API key via the `Authorization` header:
+
+```bash
+curl -H "Authorization: Bearer eg_abc123..." http://localhost:3030/api/vault-map
+```
+
+Keys have either `read` or `write` permission. Write keys can access all endpoints; read keys are restricted to read-only endpoints. Use `--no-auth` for local development without keys (127.0.0.1 only).
+
+**curl examples:**
+
+```bash
+# Search
+curl -X POST http://localhost:3030/api/search \
+  -H "Authorization: Bearer eg_..." \
+  -H "Content-Type: application/json" \
+  -d '{"query": "authentication architecture", "top_n": 5}'
+
+# Read a note
+curl http://localhost:3030/api/read/01-Projects/API-Design.md \
+  -H "Authorization: Bearer eg_..."
+
+# Create a note
+curl -X POST http://localhost:3030/api/create \
+  -H "Authorization: Bearer eg_..." \
+  -H "Content-Type: application/json" \
+  -d '{"content": "# Meeting Notes\n\nDiscussed auth timeline.", "tags": ["meeting", "auth"]}'
+```
+
+**Rate limiting:** Configurable per-key token bucket (requests per minute). Defaults to 60 req/min. Returns `429 Too Many Requests` when exceeded.
+
+**CORS:** Configurable allowed origins in `config.toml` under `[http]`. Defaults to allow all origins for local development.
+
+```toml
+[http]
+port = 3030
+host = "127.0.0.1"
+cors_origins = ["http://localhost:3000", "https://myapp.example.com"]
+rate_limit = 60
+
+[[http.api_keys]]
+key = "eg_..."
+permission = "write"
+```
+
 ## Use cases
 
 **AI-assisted knowledge work** — Give Claude or Cursor deep access to your personal knowledge base. Instead of copy-pasting context, the agent searches, reads, and cross-references your notes directly.
@@ -251,7 +352,7 @@ Returns orphan notes (no links in or out), broken wikilinks, stale notes, and ta
 | Search method | 5-lane RRF (semantic + BM25 + graph + reranker + temporal) | Vector similarity only | Keyword only |
 | Query understanding | LLM orchestrator classifies intent, adapts weights | None | None |
 | Understands note links | Yes (wikilink graph traversal) | No | Limited (backlinks panel) |
-| AI agent access | MCP server (19 tools) | Custom API needed | No |
+| AI agent access | MCP server (19 tools) + HTTP REST API (20 endpoints) | Custom API needed | No |
 | Write capability | Create/edit/rewrite/delete with smart filing | No | Manual |
 | Vault health | Orphans, broken links, stale notes, tag hygiene | No | Limited |
 | Real-time sync | File watcher, 2s debounce | Manual re-index | N/A |
@@ -269,6 +370,7 @@ engraph is not a replacement for Obsidian — it's the intelligence layer that s
 - llama.cpp inference via Rust bindings (GGUF models, Metal GPU on macOS, CUDA on Linux)
 - Intelligence opt-in: heuristic fallback when disabled, LLM-powered when enabled
 - MCP server with 19 tools (8 read, 10 write, 1 diagnostic) via stdio
+- HTTP REST API with 20 endpoints, API key auth (`eg_` prefix), rate limiting, CORS — enabled via `engraph serve --http`
 - Section-level reading and editing: target specific headings with replace/prepend/append modes
 - Full note rewriting with automatic frontmatter preservation
 - Granular frontmatter mutations: set/remove fields, add/remove tags and aliases
@@ -283,7 +385,7 @@ engraph is not a replacement for Obsidian — it's the intelligence layer that s
 - Enhanced file resolution with fuzzy Levenshtein matching fallback
 - Content-based folder role detection (people, daily, archive) by content patterns
 - Configurable model overrides for multilingual support
-- 361 unit tests, CI on macOS + Ubuntu
+- 385 unit tests, CI on macOS + Ubuntu
 
 ## Roadmap
 
@@ -293,7 +395,7 @@ engraph is not a replacement for Obsidian — it's the intelligence layer that s
 - [x] ~~Vault health monitor — orphan notes, broken links, stale content, tag hygiene~~ (v1.1)
 - [x] ~~Obsidian CLI integration — auto-detect and delegate with circuit breaker~~ (v1.1)
 - [x] ~~Temporal search — find notes by time period, date-aware queries~~ (v1.2)
-- [ ] HTTP/REST API — complement MCP with a standard web API (v1.3)
+- [x] ~~HTTP/REST API — complement MCP with a standard web API~~ (v1.3)
 - [ ] Multi-vault — search across multiple vaults (v1.4)
 
 ## Configuration
@@ -328,7 +430,7 @@ All data stored in `~/.engraph/` — single SQLite database (~10MB typical), GGU
 ## Development
 
 ```bash
-cargo test --lib          # 361 unit tests, no network (requires CMake for llama.cpp)
+cargo test --lib          # 385 unit tests, no network (requires CMake for llama.cpp)
 cargo clippy -- -D warnings
 cargo fmt --check
 
@@ -340,7 +442,7 @@ cargo test --test integration -- --ignored
 
 Contributions welcome. Please open an issue first to discuss what you'd like to change.
 
-The codebase is 23 Rust modules behind a lib crate. `CLAUDE.md` in the repo root has detailed architecture documentation for AI-assisted development.
+The codebase is 24 Rust modules behind a lib crate. `CLAUDE.md` in the repo root has detailed architecture documentation for AI-assisted development.
 
 ## License
 
