From a4913e90acf3a7166d93e5897ebd9897161dae0f Mon Sep 17 00:00:00 2001 From: Jason Mulligan Date: Thu, 11 Jun 2026 21:46:20 -0400 Subject: [PATCH 1/3] docs: update memory layer documentation with GC, ephemeral, and daily reflection --- README.md | 12 ++++++++++-- docs/OVERVIEW.md | 33 ++++++++++++++++++++++++--------- 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 27464e5d..5299c836 100644 --- a/README.md +++ b/README.md @@ -374,10 +374,16 @@ Built-in tools are registered only when their required permissions are enabled f `madz` operates on a dual-layer memory architecture that evolves naturally over time: **Canonical Memories** -Set explicitly by the user, these form the enduring foundation of the system. Stored as individual Markdown files in `memory/context/`, each carries `createdDate` and `updatedDate` metadata in YAML frontmatter. At the start of every session, canonical memories are loaded and appended to the system prompt, ensuring core context, preferences, and personal details remain consistent across interactions. +Set explicitly by the user, these form the enduring foundation of the system. Stored as individual Markdown files in `memory/context/`, each carries `createdDate` and `updatedDate` metadata in YAML frontmatter. At the start of every session, canonical memories are loaded and appended to the system prompt, ensuring core context, preferences, and personal details remain consistent across interactions. Includes profile, clarifications, reflections, and temporal captures. **Ephemeral Memories** -Captured autonomously by the harness during operation, these record patterns, milestones, emotional tones, and recurring themes. Stored temporarily with automatic expiration, they act as a living lens — subtly influencing how `madz` approaches future tasks, adapts its tone, and anticipates needs. They are not hardcoded; they evolve organically as the relationship deepens. +Captured autonomously by the harness during operation, these record patterns, milestones, emotional tones, and recurring themes. Stored temporarily with automatic expiration via `expiresAt` frontmatter, they act as a living lens — subtly influencing how `madz` approaches future tasks, adapts its tone, and anticipates needs. They are not hardcoded; they evolve organically as the relationship deepens. Cleaned automatically by `expireEphemeralMemories()`. + +**V8 Garbage Collection** +When Node.js is started with `--expose-gc`, `madz` manages V8 garbage collection via an idle timer (`memory.gc.idleTimeoutMs`, default 5 min). GC triggers after inactivity, respecting a rate limit (`memory.gc.maxGcPerHour`, default 4). Manually triggerable via the `:gc` TUI command or `:gc status`. + +**Daily Reflection** +On first profile save (onboarding completion), `madz` automatically installs a daily cron job (`0 2 * * *`) that runs `/reflection` via `--chat` mode. The job definition is persisted as `memory/schedules/reflection-daily.json` and registered in the system crontab under the `madz-schedules` block. Together, these layers create a system that remembers what matters while naturally adapting to how you work. When you update or delete a canonical memory, follow it with `:new` so the current session reflects the change immediately. @@ -395,6 +401,8 @@ Optional `@opentelemetry/sdk-node` integration. Configurable exporter (console, Recurring job definitions in `config.yaml`. Supports both in-process scheduling and delegation to the system crontab (`mode: "system"`). Each invocation inherits the current session's memory context and sandbox permissions. Max-concurrency control prevents run overlap. +On first onboarding completion, `madz` automatically installs a `reflection-daily` cron job (`0 2 * * *`) into the system crontab. Job definitions are persisted as JSON in `memory/schedules/` and managed under the `madz-schedules` block. + ## Directory Structure ``` diff --git a/docs/OVERVIEW.md b/docs/OVERVIEW.md index f974bf9b..25b47dbc 100644 --- a/docs/OVERVIEW.md +++ b/docs/OVERVIEW.md @@ -94,18 +94,33 @@ The agent runs: reason → call tool(s) → reason again → answer. Tool array ## Memory -`src/memory/` — persistent Markdown storage with YAML frontmatter. +`src/memory/` — persistent Markdown storage with YAML frontmatter, dual-layer architecture (canonical + ephemeral), V8 GC management, and automated daily reflection scheduling. | File | Purpose | |------|---------| -| `writer.js` | `writeMemoryFile()` — writes timestamped `.md` files | -| `reader.js` | `parseFrontmatter()`, `readMemoryFile()` | -| `context.js` | `loadContext()` — scans context files, loads profile, returns combined string | -| `retention.js` | `cleanRetainedMemory()` / `enforceMaxEntries()` — expiry cleanup | -| `loadMemories.js` | `loadMemories()` — loads entries sorted by `updatedDate` → `formatMemoriesForPrompt()` | -| `profile.js` | User profile CRUD: `loadProfile()`, `saveProfile()`, onboarding state machine | - -At session start, memories and profile are appended to the system prompt under "The following are important memories for the user:". +| `writer.js` | `writeMemoryFile()` — writes timestamped `.md` files with YAML frontmatter, auto-slugifies titles | +| `reader.js` | `parseFrontmatter()` — YAML frontmatter parsing via `js-yaml`; `readMemoryFile()` — loads and parses a single memory file | +| `context.js` | `loadContext()` — scans context directory for `.md` files, loads profile, returns combined string sorted by `timestamp` frontmatter | +| `retention.js` | `cleanRetainedMemory()` — removes files older than `retentionDays` (default 90); `enforceMaxEntries()` — caps directory at `maxEntries` (default 1000) by oldest mtime | +| `loadMemories.js` | `loadMemories()` — loads all entries sorted by `updatedDate` descending; `formatMemoriesForPrompt()` — formats entries with category labels (`USER PROFILE`, `USER CLARIFICATIONS`, `WORKING REFLECTION`, `TEMPORAL CAPTURE`); `parseEntryFile()` — parses a single entry's frontmatter + body | +| `profile.js` | User profile CRUD: `loadProfile()`, `saveProfile()`, `hasProfile()`, `formatProfileContext()`, `sanitizeProfileData()`. Defines 12 attributes (name, dob, relationship, pets, hobbies, expertise, favorite bands/books/tv/movies, location, notes) with onboarding state machine (`INIT → ATTRACTOR → COLLECT → SAVE → TRANSCEND`) and control pattern matching (`skip`, `cancel`, `exit`) | +| `expireEphemeral.js` | `expireEphemeralMemories()` — scans context directory, removes `.md` files with `ephemeral: true` + expired `expiresAt`; `isExpired()` — checks `expiresAt` against current time; `readEphemeralFile()` — extracts ephemeral metadata from frontmatter | +| `gc.js` | V8 garbage collection manager: `gc()` — triggers `global.gc()` with rate limiting (default 4 calls/hour, sliding window); `initGC()` — creates idle-timer controller with `onActivity()` reset and `stop()`; `isAvailable()` — checks `--expose-gc`; `getGcCalls()` / `_resetGcCalls()` — call tracking for testing | +| `prompts.js` | `loadSystemPrompt()` — loads `prompts/SYSTEM_PROMPT.md`, strips YAML frontmatter if present | + +**Dual-Layer Architecture:** + +- **Canonical Memories** — Long-term, user-defined context stored as individual `.md` files in `memory/context/`. Each carries `createdDate` and `updatedDate` in YAML frontmatter. Loaded at session start and appended to the system prompt. Includes profile, clarifications, reflections, and temporal captures. + +- **Ephemeral Memories** — Autonomously captured moments (victories, frustrations, insights) with automatic expiration via `expiresAt` frontmatter field. Cleaned by `expireEphemeralMemories()` on a scheduled basis. These create a living lens that subtly influences tone and awareness over time. + +**GC Integration:** + +V8 garbage collection is managed by `initGC()` which creates an idle-timer controller. The timer fires `gc()` after `memory.gc.idleTimeoutMs` (default 300000ms / 5 min) of inactivity, respecting `memory.gc.maxGcPerHour` (default 4) via a sliding window. Manually triggerable via the `:gc` TUI command and `:gc status`. GC is available when Node.js is started with `--expose-gc`. + +**Daily Reflection Scheduler:** + +`src/scheduler/autoSchedule.js` — `setupAutoSchedule()` returns a callback invoked after `saveProfile()` succeeds during onboarding. It automatically installs a `reflection-daily` cron job (`0 2 * * *`) into the system crontab and persists the job definition as `memory/schedules/reflection-daily.json`. The job invokes `node index.js --chat "/reflection"` at 2 AM daily. --- From dcc597ef400424f046991637c20c7e631fc0f1c2 Mon Sep 17 00:00:00 2001 From: Jason Mulligan Date: Thu, 11 Jun 2026 21:48:12 -0400 Subject: [PATCH 2/3] docs: remove V8 GC from memory system documentation --- README.md | 3 --- docs/OVERVIEW.md | 4 ---- 2 files changed, 7 deletions(-) diff --git a/README.md b/README.md index 5299c836..b24dd57b 100644 --- a/README.md +++ b/README.md @@ -379,9 +379,6 @@ Set explicitly by the user, these form the enduring foundation of the system. St **Ephemeral Memories** Captured autonomously by the harness during operation, these record patterns, milestones, emotional tones, and recurring themes. Stored temporarily with automatic expiration via `expiresAt` frontmatter, they act as a living lens — subtly influencing how `madz` approaches future tasks, adapts its tone, and anticipates needs. They are not hardcoded; they evolve organically as the relationship deepens. Cleaned automatically by `expireEphemeralMemories()`. -**V8 Garbage Collection** -When Node.js is started with `--expose-gc`, `madz` manages V8 garbage collection via an idle timer (`memory.gc.idleTimeoutMs`, default 5 min). GC triggers after inactivity, respecting a rate limit (`memory.gc.maxGcPerHour`, default 4). Manually triggerable via the `:gc` TUI command or `:gc status`. - **Daily Reflection** On first profile save (onboarding completion), `madz` automatically installs a daily cron job (`0 2 * * *`) that runs `/reflection` via `--chat` mode. The job definition is persisted as `memory/schedules/reflection-daily.json` and registered in the system crontab under the `madz-schedules` block. diff --git a/docs/OVERVIEW.md b/docs/OVERVIEW.md index 25b47dbc..69d14058 100644 --- a/docs/OVERVIEW.md +++ b/docs/OVERVIEW.md @@ -114,10 +114,6 @@ The agent runs: reason → call tool(s) → reason again → answer. Tool array - **Ephemeral Memories** — Autonomously captured moments (victories, frustrations, insights) with automatic expiration via `expiresAt` frontmatter field. Cleaned by `expireEphemeralMemories()` on a scheduled basis. These create a living lens that subtly influences tone and awareness over time. -**GC Integration:** - -V8 garbage collection is managed by `initGC()` which creates an idle-timer controller. The timer fires `gc()` after `memory.gc.idleTimeoutMs` (default 300000ms / 5 min) of inactivity, respecting `memory.gc.maxGcPerHour` (default 4) via a sliding window. Manually triggerable via the `:gc` TUI command and `:gc status`. GC is available when Node.js is started with `--expose-gc`. - **Daily Reflection Scheduler:** `src/scheduler/autoSchedule.js` — `setupAutoSchedule()` returns a callback invoked after `saveProfile()` succeeds during onboarding. It automatically installs a `reflection-daily` cron job (`0 2 * * *`) into the system crontab and persists the job definition as `memory/schedules/reflection-daily.json`. The job invokes `node index.js --chat "/reflection"` at 2 AM daily. From 795975150018d87209ceab04310e102711ec0bae Mon Sep 17 00:00:00 2001 From: Jason Mulligan Date: Thu, 11 Jun 2026 21:49:04 -0400 Subject: [PATCH 3/3] docs: update to triple-layer memory architecture (canonical + ephemeral + reflection) --- README.md | 8 ++++---- docs/OVERVIEW.md | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index b24dd57b..00e098b1 100644 --- a/README.md +++ b/README.md @@ -371,16 +371,16 @@ Built-in tools are registered only when their required permissions are enabled f ### Memory System -`madz` operates on a dual-layer memory architecture that evolves naturally over time: +`madz` operates on a triple-layer memory architecture that evolves naturally over time: **Canonical Memories** -Set explicitly by the user, these form the enduring foundation of the system. Stored as individual Markdown files in `memory/context/`, each carries `createdDate` and `updatedDate` metadata in YAML frontmatter. At the start of every session, canonical memories are loaded and appended to the system prompt, ensuring core context, preferences, and personal details remain consistent across interactions. Includes profile, clarifications, reflections, and temporal captures. +Set explicitly by the user, these form the enduring foundation of the system. Stored as individual Markdown files in `memory/context/`, each carries `createdDate` and `updatedDate` metadata in YAML frontmatter. At the start of every session, canonical memories are loaded and appended to the system prompt, ensuring core context, preferences, and personal details remain consistent across interactions. Includes profile, clarifications, and temporal captures. **Ephemeral Memories** Captured autonomously by the harness during operation, these record patterns, milestones, emotional tones, and recurring themes. Stored temporarily with automatic expiration via `expiresAt` frontmatter, they act as a living lens — subtly influencing how `madz` approaches future tasks, adapts its tone, and anticipates needs. They are not hardcoded; they evolve organically as the relationship deepens. Cleaned automatically by `expireEphemeralMemories()`. -**Daily Reflection** -On first profile save (onboarding completion), `madz` automatically installs a daily cron job (`0 2 * * *`) that runs `/reflection` via `--chat` mode. The job definition is persisted as `memory/schedules/reflection-daily.json` and registered in the system crontab under the `madz-schedules` block. +**Reflections** +A daily cron job (`0 2 * * *`) installed automatically on first onboarding completion. Runs `/reflection` via `--chat` mode, generating a narrative reflection summary from recent session history. The job definition is persisted as `memory/schedules/reflection-daily.json` and registered in the system crontab under the `madz-schedules` block. Reflections are stored as canonical memories in `memory/context/` with `createdDate` and `updatedDate` metadata. Together, these layers create a system that remembers what matters while naturally adapting to how you work. When you update or delete a canonical memory, follow it with `:new` so the current session reflects the change immediately. diff --git a/docs/OVERVIEW.md b/docs/OVERVIEW.md index 69d14058..2274e651 100644 --- a/docs/OVERVIEW.md +++ b/docs/OVERVIEW.md @@ -94,7 +94,7 @@ The agent runs: reason → call tool(s) → reason again → answer. Tool array ## Memory -`src/memory/` — persistent Markdown storage with YAML frontmatter, dual-layer architecture (canonical + ephemeral), V8 GC management, and automated daily reflection scheduling. +`src/memory/` — persistent Markdown storage with YAML frontmatter, triple-layer architecture (canonical + ephemeral + reflection), and automated daily reflection scheduling. | File | Purpose | |------|---------| @@ -108,13 +108,13 @@ The agent runs: reason → call tool(s) → reason again → answer. Tool array | `gc.js` | V8 garbage collection manager: `gc()` — triggers `global.gc()` with rate limiting (default 4 calls/hour, sliding window); `initGC()` — creates idle-timer controller with `onActivity()` reset and `stop()`; `isAvailable()` — checks `--expose-gc`; `getGcCalls()` / `_resetGcCalls()` — call tracking for testing | | `prompts.js` | `loadSystemPrompt()` — loads `prompts/SYSTEM_PROMPT.md`, strips YAML frontmatter if present | -**Dual-Layer Architecture:** +**Triple-Layer Architecture:** - **Canonical Memories** — Long-term, user-defined context stored as individual `.md` files in `memory/context/`. Each carries `createdDate` and `updatedDate` in YAML frontmatter. Loaded at session start and appended to the system prompt. Includes profile, clarifications, reflections, and temporal captures. - **Ephemeral Memories** — Autonomously captured moments (victories, frustrations, insights) with automatic expiration via `expiresAt` frontmatter field. Cleaned by `expireEphemeralMemories()` on a scheduled basis. These create a living lens that subtly influences tone and awareness over time. -**Daily Reflection Scheduler:** +- **Reflections** — Generated daily by a cron job (`0 2 * * *`) that runs `/reflection` via `--chat` mode. Reflections are stored as canonical memories in `memory/context/` with `createdDate` and `updatedDate` metadata. The cron job is auto-installed on first onboarding completion, persisted as `memory/schedules/reflection-daily.json`, and registered in the system crontab under the `madz-schedules` block. `src/scheduler/autoSchedule.js` — `setupAutoSchedule()` returns a callback invoked after `saveProfile()` succeeds during onboarding. It automatically installs a `reflection-daily` cron job (`0 2 * * *`) into the system crontab and persists the job definition as `memory/schedules/reflection-daily.json`. The job invokes `node index.js --chat "/reflection"` at 2 AM daily.