diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d33425..215a3a4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,18 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +## [5.3.1] - 2026-11-14 + +### Fixed + +- **Preview shows the names a Move gives when the folder already has a file of that name.** When two + files of one run got the same new name (the second numbered `_1`) and the folder already held + `report.pdf`, Preview showed both files going to `report_1.pdf` while the move gave one of them + `report_2.pdf`. With `report_1.pdf` taken as well, a file was named `report_1_1.pdf`. The move now + skips the names other files of the run are headed for and counts on from the original name + (`report_2.pdf`, `report_3.pdf`), and Preview shows the same names. A run resumed after a crash + numbers files the same way. No file was overwritten or lost. + ## [5.3.0] - 2026-11-13 ### Fixed @@ -1681,7 +1693,8 @@ notes. Note: versions before 5.0.0 were released from predecessor repositories whose tags are not part of this repository's history, so they are listed without links. -[Unreleased]: https://github.com/iLevyTate/StratoSortStack/compare/v5.3.0...HEAD +[Unreleased]: https://github.com/iLevyTate/StratoSortStack/compare/v5.3.1...HEAD +[5.3.1]: https://github.com/iLevyTate/StratoSortStack/compare/v5.3.0...v5.3.1 [5.3.0]: https://github.com/iLevyTate/StratoSortStack/compare/v5.2.0...v5.3.0 [5.2.0]: https://github.com/iLevyTate/StratoSortStack/compare/v5.1.2...v5.2.0 [5.1.2]: https://github.com/iLevyTate/StratoSortStack/compare/v5.1.1...v5.1.2 diff --git a/CLAUDE.md b/CLAUDE.md index 8000c0e..97153e8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -103,7 +103,7 @@ tree. `ci`) enforces a monotonic floor and blocks new .js under `COVERED_DIRS` (`src/shared`, `src/preload`, `src/main/{ipc,services,analysis,errors,utils,workers,transports,core}`, `src/renderer/{services,hooks,phases,store,utils,components}` today (every `.js` under `src/` is - now opted in — 506/506 as of 2026-11-13, and `GLOBAL_DIRECTIVE_FLOOR` matches at 506) — see + now opted in — 507/507 as of 2026-11-14, and `GLOBAL_DIRECTIVE_FLOOR` matches at 507) — see `scripts/verify-checkjs-ratchet.js` for the authoritative list) from landing without the directive. When adding a file to a covered dir: put `// @ts-check` on line 1 and fix the errors before commit. When opting in a new file elsewhere: bump `GLOBAL_DIRECTIVE_FLOOR` in diff --git a/SECURITY.md b/SECURITY.md index ffed278..9b9d167 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -158,4 +158,4 @@ Contributors who report valid security issues will be acknowledged here (with pe --- -_Last updated: 2026-11-13 — applies to StratoSort Stack 5.3.0._ +_Last updated: 2026-11-14 — applies to StratoSort Stack 5.3.1._ diff --git a/docs/BETA_TESTER_GUIDE.md b/docs/BETA_TESTER_GUIDE.md index 002f74f..9049502 100644 --- a/docs/BETA_TESTER_GUIDE.md +++ b/docs/BETA_TESTER_GUIDE.md @@ -44,7 +44,7 @@ No sign-up is needed: download the public release and send feedback through ## 1. Download one installer 1. Open **[GitHub Releases](https://github.com/iLevyTate/StratoSortStack/releases)**. -2. Pick the newest release and find the assets for your OS (current product version is **5.3.0**). +2. Pick the newest release and find the assets for your OS (current product version is **5.3.1**). 3. Download **exactly one app installer/package** for your OS (do not download checksum or `.yml` files unless asked). @@ -60,13 +60,13 @@ No sign-up is needed: download the public release and send feedback through | Your computer | Typical filename pattern | | :----------------------------------------- | :-------------------------------------------- | -| **Windows** (x64, preferred) | `StratoSortStack-Setup-5.3.0.exe` | -| **Windows** (x64, no-install zip) | `StratoSortStack-5.3.0-win-x64.zip` | -| **Windows on Arm** (arm64) | `StratoSortStack-Setup-5.3.0-arm64.exe` | -| **Mac** (Apple Silicon: M1 / M2 / M3 / M4) | `StratoSortStack-5.3.0-mac-arm64.dmg` | -| **Mac** (Intel) | `StratoSortStack-5.3.0-mac-x64.dmg` | -| **Linux** (x64) | `StratoSortStack-5.3.0-linux-x86_64.AppImage` | -| **Linux** (arm64) | `StratoSortStack-5.3.0-linux-arm64.AppImage` | +| **Windows** (x64, preferred) | `StratoSortStack-Setup-5.3.1.exe` | +| **Windows** (x64, no-install zip) | `StratoSortStack-5.3.1-win-x64.zip` | +| **Windows on Arm** (arm64) | `StratoSortStack-Setup-5.3.1-arm64.exe` | +| **Mac** (Apple Silicon: M1 / M2 / M3 / M4) | `StratoSortStack-5.3.1-mac-arm64.dmg` | +| **Mac** (Intel) | `StratoSortStack-5.3.1-mac-x64.dmg` | +| **Linux** (x64) | `StratoSortStack-5.3.1-linux-x86_64.AppImage` | +| **Linux** (arm64) | `StratoSortStack-5.3.1-linux-arm64.AppImage` | Always take the **newest** installer for your OS, not one from an older release. @@ -283,7 +283,7 @@ Do not attach documents or logs that contain private information; GitHub issues **Actual:** Empty results and a warning about model mismatch. -**Environment:** Windows 11, StratoSort Stack 5.3.0 (installer), 32 GB RAM, RTX 3060. +**Environment:** Windows 11, StratoSort Stack 5.3.1 (installer), 32 GB RAM, RTX 3060. **Attachment:** Exported logs from Settings → Diagnostics → Export logs. @@ -291,7 +291,7 @@ Do not attach documents or logs that contain private information; GitHub issues ## Other ways to help (no code) -- Confirm someone else’s bug on **your** OS and version (“Reproduced on macOS 15 / app 5.3.0”). +- Confirm someone else’s bug on **your** OS and version (“Reproduced on macOS 15 / app 5.3.1”). - Test each new release and note **regressions** (things that used to work). - Suggest UX improvements with **screenshots** and short before/after notes. diff --git a/docs/INSTALL_GUIDE.md b/docs/INSTALL_GUIDE.md index 8b8bdbc..868b90d 100644 --- a/docs/INSTALL_GUIDE.md +++ b/docs/INSTALL_GUIDE.md @@ -42,7 +42,7 @@ download the single file that matches your operating system. | **Linux** x64 (if artifact is attached) | **`StratoSortStack-X.X.X-linux-x86_64.AppImage`** | | **Linux** arm64 (if artifact is attached) | **`StratoSortStack-X.X.X-linux-arm64.AppImage`** | -Replace **X.X.X** with the version number shown on the Releases page (e.g. `5.3.0`). The Windows +Replace **X.X.X** with the version number shown on the Releases page (e.g. `5.3.1`). The Windows arm64 build ships without bundled FFmpeg, so audio/video analysis there needs FFmpeg on your `PATH`. ### Mac -- Apple Silicon or Intel? diff --git a/package-lock.json b/package-lock.json index 4437680..76370b1 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "stratosort-stack", - "version": "5.3.0", + "version": "5.3.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "stratosort-stack", - "version": "5.3.0", + "version": "5.3.1", "hasInstallScript": true, "license": "PolyForm-Noncommercial-1.0.0", "dependencies": { diff --git a/package.json b/package.json index 1e2eb5b..de0583a 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "stratosort-stack", "productName": "StratoSort Stack", - "version": "5.3.0", + "version": "5.3.1", "description": "StratoSort Stack - Local AI Document Organizer. Privacy-focused document organization using in-process AI (node-llama-cpp + Orama). No required external services: inference, embeddings, and search run on-device.", "keywords": [ "electron", diff --git a/scripts/verify-checkjs-ratchet.js b/scripts/verify-checkjs-ratchet.js index 36704f6..25c7c07 100644 --- a/scripts/verify-checkjs-ratchet.js +++ b/scripts/verify-checkjs-ratchet.js @@ -43,7 +43,7 @@ const COVERED_DIRS = [ 'src/renderer/utils', 'src/renderer/components' ]; -const GLOBAL_DIRECTIVE_FLOOR = 506; +const GLOBAL_DIRECTIVE_FLOOR = 507; // ──────────────────────────────────────────────────────────────────────── const DIRECTIVE_SCAN_LINES = 5; diff --git a/src/main/ipc/files/batchOrganizeHandler.js b/src/main/ipc/files/batchOrganizeHandler.js index db77488..c4d4428 100644 --- a/src/main/ipc/files/batchOrganizeHandler.js +++ b/src/main/ipc/files/batchOrganizeHandler.js @@ -26,6 +26,7 @@ const systemAnalytics = require('../../core/systemAnalytics'); const { acquireBatchLock, renewBatchLock, releaseBatchLock } = require('./batchLockManager'); const { validateBatchOperation, MAX_BATCH_SIZE } = require('./batchValidator'); const { executeRollback } = require('./batchRollback'); +const { formatNumberedCollisionCandidate, numberingBaseOf } = require('./collisionNames'); const { sendOperationProgress, sendChunkedResults, @@ -64,6 +65,9 @@ const { * app suggested before the user confirmed; a different landing folder is a correction * @property {string} [analysisLabel] - The label the analysis gave the file, which a * correction is recorded under when the history holds no different one + * @property {boolean} [renumbered] - Organize numbered this destination apart + * from another file of the batch headed for the same name (`_N` added); a + * taken name is then numbered from the name before that (numberingBaseOf) */ /** @@ -114,24 +118,35 @@ const YIELD_EVERY_N_OPS = 10; // Yield to event loop every N operations const PREVIEW_MAX_NUMERIC_ATTEMPTS = 50; /** - * Format the Nth collision-suffixed candidate for a destination path. - * - * IMPORTANT: This is the single source of truth for the suffix shape so - * the dry-run preview and the real move path stay byte-identical. They - * had drifted post-2.0.9: the executor used `${base}_${N}${ext}` while - * the preview used `${base} (${N})${ext}`, so the Preview button - * displayed a destination that did not match what the move would produce. - * - * @param {string} originalDestination - The originally requested destination. - * @param {number} counter - 1-based collision counter. - * @returns {string} Suffixed candidate path. + * Comparison key for a batch destination, the same before and after path + * validation rewrites an operation's paths to their normalized form. + * @param {string} destination + * @returns {string} */ -function formatNumberedCollisionCandidate(originalDestination, counter) { - const ext = path.extname(originalDestination); - // When ext is empty, -ext.length is -0 which equals 0, causing slice(0,0) - // to return empty string — guard explicitly for the no-extension case. - const baseName = ext.length > 0 ? originalDestination.slice(0, -ext.length) : originalDestination; - return `${baseName}_${counter}${ext}`; +function batchDestinationKey(destination) { + return _canonicalizeForOverlapCheck(path.resolve(destination)); +} + +/** + * The next numbered form of `base` after `counter` that no other file of the + * batch is headed for. A name another operation will take is skipped, so a + * file that finds its name taken on disk does not take the name Organize + * numbered another file of the batch to (and Preview shows what the move + * does). + * @param {string} base + * @param {number} counter - last counter tried (0 before the first) + * @param {Set | null | undefined} reserved - batchDestinationKey of + * every name the batch is headed for, plus any it has claimed + * @returns {{ candidate: string, counter: number }} + */ +function nextFreeNumberedCandidate(base, counter, reserved) { + let next = counter + 1; + let candidate = formatNumberedCollisionCandidate(base, next); + while (reserved && next < MAX_NUMERIC_RETRIES && reserved.has(batchDestinationKey(candidate))) { + next += 1; + candidate = formatNumberedCollisionCandidate(base, next); + } + return { candidate, counter: next }; } /** @@ -230,6 +245,49 @@ function _detectPathOverlap(ops) { return null; } +/** + * The ops that must wait for an earlier op of the batch: those numbered from + * the same name (numberingBaseOf) when that name or one of their numbered + * names is already taken on disk. Each op whose name is taken then counts up + * the same numbers; run concurrently, which file gets which number depends on + * timing, and only batch order gives each the number Preview showed. Only + * such a group runs one at a time; every other file keeps the batch's + * concurrency, and when none of a group's names is taken no op of it walks. + * @param {Array<{source:string,destination:string,renumbered?:boolean,status?:string}>} ops + * @returns {Promise>} op index -> index of the op it waits for + */ +async function _numberingOrder(ops) { + /** @type {Map} */ + const waitsFor = new Map(); + if (!Array.isArray(ops) || ops.length < 2) return waitsFor; + /** @type {Map} */ + const groups = new Map(); + ops.forEach((op, index) => { + if (!op || op.status === 'done' || typeof op.destination !== 'string') return; + const base = numberingBaseOf(op.destination, op.renumbered); + const key = _canonicalizeForOverlapCheck(base); + if (!key) return; + const group = groups.get(key) || { base, indices: [] }; + group.indices.push(index); + groups.set(key, group); + }); + for (const { base, indices } of groups.values()) { + if (indices.length < 2) continue; + let taken = false; + for (const candidate of [base, ...indices.map((index) => ops[index].destination)]) { + if (await pathExists(candidate)) { + taken = true; + break; + } + } + if (!taken) continue; + for (let k = 1; k < indices.length; k += 1) { + waitsFor.set(indices[k], indices[k - 1]); + } + } + return waitsFor; +} + /** * Concurrency limiter (p-limit pattern). Returns a function `limit(fn)` that * queues `fn` behind `concurrency` slots. @@ -621,6 +679,15 @@ async function handleBatchOrganize(params) { } ); } + // Files numbered from one name that is taken on disk move one after + // another, in batch order, so each gets the name Preview showed. + const numberingWaits = pathOverlap ? new Map() : await _numberingOrder(batch.operations); + if (numberingWaits.size > 0) { + log.info( + '[FILE-OPS] Files numbered from one taken name move in batch order, as Preview numbered them', + { batchId, waiting: numberingWaits.size } + ); + } const batchOperationTotal = batch.operations.length; const limit = pLimit(effectiveConcurrency); // Files this batch has still to move. The duplicate check must not @@ -630,6 +697,13 @@ async function handleBatchOrganize(params) { const pendingSources = batch.operations .filter((op) => op.status !== 'done' && typeof op.source === 'string') .map((op) => op.source); + // Every name the batch is headed for: a file whose name is taken on + // disk is numbered past these, as Preview numbers it. + const reservedDestinations = new Set( + batch.operations + .filter((op) => typeof op.destination === 'string' && op.destination) + .map((op) => batchDestinationKey(op.destination)) + ); // Abort reasons the USER deliberately triggered (renderer buttons). // Anything else on the signal — 'ipc-timeout', 'shutdown' (abortAll), @@ -871,7 +945,8 @@ async function handleBatchOrganize(params) { log, computeFileChecksum, signal, - pendingSources + pendingSources, + reservedDestinations ); try { const outcome = await Promise.race([movePromise, abandoned]); @@ -1188,7 +1263,20 @@ async function handleBatchOrganize(params) { } }; - const promises = batch.operations.map((_, i) => limit(() => processOperation(i))); + // An op that waits for an earlier one queues for a slot only once + // that op has settled, so waiting never holds a slot other files need. + /** @type {Promise[]} */ + const promises = []; + batch.operations.forEach((_, i) => { + const waitFor = numberingWaits.get(i); + promises[i] = + waitFor === undefined + ? limit(() => processOperation(i)) + : promises[waitFor].then( + () => limit(() => processOperation(i)), + () => limit(() => processOperation(i)) + ); + }); const opResults = await Promise.allSettled(promises); // `processOperation` owns a try/catch that translates expected failures @@ -1741,8 +1829,17 @@ function reconcileRolledBackResults(rollbackOutcome, completedOperations, rollba * @param {string[]} [pendingSources] - Every source the caller's batch has * still to move (the same array for each call), so the duplicate check does * not count one of them as a copy already at the destination. + * @param {Set} [reservedDestinations] - batchDestinationKey of every + * destination of the caller's batch; a taken name is numbered past them. */ -async function performFileMove(op, log, checksumFn, abortSignal = null, pendingSources) { +async function performFileMove( + op, + log, + checksumFn, + abortSignal = null, + pendingSources, + reservedDestinations +) { // This prevents creating numbered copies of files that already exist. const duplicateResult = await handleDuplicateMove({ sourcePath: op.source, @@ -1802,16 +1899,23 @@ async function performFileMove(op, log, checksumFn, abortSignal = null, pendingS let counter = 0; let uniqueDestination = op.destination; - const ext = path.extname(op.destination); + // A taken name is numbered from the name before Organize numbered it, if + // it did (report_1.pdf taken -> report_2.pdf, not report_1_1.pdf). + const numberingBase = numberingBaseOf(op.destination, op.renumbered); + const ext = path.extname(numberingBase); // When ext is empty, -ext.length is -0 which equals 0, causing slice(0,0) to return empty string - const baseName = ext.length > 0 ? op.destination.slice(0, -ext.length) : op.destination; + const baseName = ext.length > 0 ? numberingBase.slice(0, -ext.length) : numberingBase; let operationComplete = false; const advanceToNextCandidate = () => { - counter++; // Single source of truth so preview and execute produce identical - // suffixed paths; see formatNumberedCollisionCandidate at top of file. - uniqueDestination = formatNumberedCollisionCandidate(op.destination, counter); + // suffixed paths (formatNumberedCollisionCandidate, collisionNames.js); both + // skip the names other files of the batch are headed for. + ({ candidate: uniqueDestination, counter } = nextFreeNumberedCandidate( + numberingBase, + counter, + reservedDestinations + )); const maxPathLength = getEffectiveMaxPathLength(); if (uniqueDestination.length > maxPathLength) { const err = /** @type {NodeJS.ErrnoException} */ ( @@ -2542,7 +2646,16 @@ async function previewBatchOrganize({ operation, log }) { // The numeric-suffix collision resolver is normally driven by repeated // fs.rename + EEXIST retries. For preview we walk the same algorithm - // using fs.access — same shape of result, no side effects. + // using fs.access — same shape of result, no side effects. The move skips + // every name the batch is headed for, and a numbered name an earlier file + // took is on disk by the time a later one walks (ops counting from one + // name taken on disk run in batch order, _numberingOrder), so both go in + // `reserved`. + const reserved = new Set( + ops + .filter((op) => typeof op?.destination === 'string' && op.destination) + .map((op) => batchDestinationKey(op.destination)) + ); for (const op of ops) { if (!op?.source || !op?.destination) { skipped.push({ @@ -2583,13 +2696,20 @@ async function previewBatchOrganize({ operation, log }) { if (!alreadyAtDestination && (await pathExistsSilent(op.destination))) { collision = true; conflicts += 1; - const ext = path.extname(op.destination); - const baseName = ext.length > 0 ? op.destination.slice(0, -ext.length) : op.destination; + const numberingBase = numberingBaseOf(op.destination, op.renumbered); + const ext = path.extname(numberingBase); + const baseName = ext.length > 0 ? numberingBase.slice(0, -ext.length) : numberingBase; let resolved = false; - for (let i = 1; i <= PREVIEW_MAX_NUMERIC_ATTEMPTS; i += 1) { - const candidate = formatNumberedCollisionCandidate(op.destination, i); - if (!(await pathExistsSilent(candidate))) { - finalDestination = candidate; + let counter = 0; + // Bounded by names looked up on disk, not by the number: skipping the + // batch's own names does not count against the preview's budget. + for (let probes = 0; probes < PREVIEW_MAX_NUMERIC_ATTEMPTS; probes += 1) { + const next = nextFreeNumberedCandidate(numberingBase, counter, reserved); + counter = next.counter; + if (counter >= MAX_NUMERIC_RETRIES) break; + if (!(await pathExistsSilent(next.candidate))) { + finalDestination = next.candidate; + reserved.add(batchDestinationKey(next.candidate)); resolved = true; break; } @@ -2633,6 +2753,7 @@ module.exports = { computeFileChecksum, performFileMove, formatNumberedCollisionCandidate, + numberingBaseOf, recordUndoAndUpdateDatabase, isCriticalFileError, MAX_BATCH_SIZE, diff --git a/src/main/ipc/files/collisionNames.js b/src/main/ipc/files/collisionNames.js new file mode 100644 index 0000000..0817a55 --- /dev/null +++ b/src/main/ipc/files/collisionNames.js @@ -0,0 +1,53 @@ +// @ts-check +/** + * The names a move gives a file whose name is taken. Shared by the batch + * handler's move and dry-run preview and by the resume of an interrupted + * batch, so all three number a file the same way. + */ +const path = require('path'); + +/** + * Format the Nth collision-suffixed candidate for a destination path. + * + * IMPORTANT: This is the single source of truth for the suffix shape so + * the dry-run preview and the real move path stay byte-identical. They + * had drifted post-2.0.9: the executor used `${base}_${N}${ext}` while + * the preview used `${base} (${N})${ext}`, so the Preview button + * displayed a destination that did not match what the move would produce. + * + * @param {string} originalDestination - The originally requested destination. + * @param {number} counter - 1-based collision counter. + * @returns {string} Suffixed candidate path. + */ +function formatNumberedCollisionCandidate(originalDestination, counter) { + const ext = path.extname(originalDestination); + // When ext is empty, -ext.length is -0 which equals 0, causing slice(0,0) + // to return empty string — guard explicitly for the no-extension case. + const baseName = ext.length > 0 ? originalDestination.slice(0, -ext.length) : originalDestination; + return `${baseName}_${counter}${ext}`; +} + +/** + * The name a taken destination is numbered from. Organize numbers two files + * of one batch headed for the same name apart before it sends them + * (report.pdf, report_1.pdf) and marks the numbered one `renumbered`. When + * report_1.pdf is then taken on disk, counting goes on from report + * (report_2.pdf) instead of numbering the number (report_1_1.pdf). Derived + * from the destination itself, so it is always in the same folder. + * @param {string} destination + * @param {boolean} [renumbered] + * @returns {string} + */ +function numberingBaseOf(destination, renumbered) { + if (renumbered !== true || typeof destination !== 'string') return destination; + const ext = path.extname(destination); + const stem = ext.length > 0 ? destination.slice(0, -ext.length) : destination; + const number = stem.match(/_[1-9]\d*$/); + if (!number || number.index === undefined) return destination; + const baseStem = stem.slice(0, number.index); + // A name that was only the number (`_1.pdf`) has nothing to count from. + if (!baseStem || /[\\/]$/.test(baseStem)) return destination; + return `${baseStem}${ext}`; +} + +module.exports = { formatNumberedCollisionCandidate, numberingBaseOf }; diff --git a/src/main/ipc/validationSchemas.js b/src/main/ipc/validationSchemas.js index 46a392c..4290920 100644 --- a/src/main/ipc/validationSchemas.js +++ b/src/main/ipc/validationSchemas.js @@ -803,7 +803,10 @@ if (!z) { // The label the analysis gave the file; a correction is recorded under // it. Listed for the same reason as `suggestion`. Organize leaves off a // label longer than this rather than have the whole batch refused. - analysisLabel: z.string().max(255).optional() + analysisLabel: z.string().max(255).optional(), + // Organize numbered this destination apart from another file of the + // batch (`_N` added), so a taken name is numbered from the name before. + renumbered: z.boolean().optional() }); const fileOperationSchema = z diff --git a/src/main/services/OrganizeResumeService.js b/src/main/services/OrganizeResumeService.js index 39b4815..bdc2df5 100644 --- a/src/main/services/OrganizeResumeService.js +++ b/src/main/services/OrganizeResumeService.js @@ -10,6 +10,7 @@ const { validateFileOperationPath } = require('../../shared/pathSanitization'); const { acquireBatchLock, releaseBatchLock } = require('../ipc/files/batchLockManager'); const { deferGhostCleanup, resumeGhostCleanup } = require('../../shared/ghostCleanupGuard'); const { migrateLegacyPersistedBatch } = require('./ProcessingStateService'); +const { numberingBaseOf } = require('../ipc/files/collisionNames'); /** * Resume runs replay operations from a persisted journal, so the source and @@ -124,6 +125,16 @@ async function findMovedSiblings(plannedDestination, recorded, exclude = new Set return matches; } +/** + * The name a journaled op's taken destination is numbered from + * (numberingBaseOf). Always in the destination's folder. + * @param {{ destination: string, renumbered?: boolean }} op + * @returns {string} + */ +function numberingBaseFor(op) { + return numberingBaseOf(op.destination, op.renumbered); +} + /** * Comparison key for a path: resolved, and case-folded on Windows. * @param {string} filePath @@ -383,7 +394,7 @@ async function resumeBatch( const siblingMatches = recorded && !(destinationExists && (await matchesRecordedFile(op.destination, recorded))) ? await findMovedSiblings( - op.destination, + numberingBaseFor(op), recorded, new Set( batch.operations @@ -516,12 +527,27 @@ async function resumeBatch( if (destinationExists) { let counter = 1; let uniqueDestination; - const ext = path.extname(finalDestination); + // Numbered as the live batch numbers it: from the name before an + // Organize `_N`, past every name another file of the batch is headed for. + const numberingBase = numberingBaseFor({ ...op, destination: finalDestination }); + const otherDestinations = new Set( + batch.operations + .filter( + (/** @type {any} */ other, /** @type {number} */ j) => + j !== i && typeof other?.destination === 'string' + ) + .map((/** @type {any} */ other) => pathKey(other.destination)) + ); + const ext = path.extname(numberingBase); const baseName = - ext && ext.length > 0 ? finalDestination.slice(0, -ext.length) : finalDestination; + ext && ext.length > 0 ? numberingBase.slice(0, -ext.length) : numberingBase; while (counter <= 1000) { - const base = baseName || finalDestination; + const base = baseName || numberingBase; uniqueDestination = `${base}_${counter}${ext}`; + if (otherDestinations.has(pathKey(uniqueDestination))) { + counter += 1; + continue; + } try { await fs.access(uniqueDestination); counter += 1; diff --git a/src/renderer/phases/organize/useOrganization.js b/src/renderer/phases/organize/useOrganization.js index 5d82888..d6b2d4d 100644 --- a/src/renderer/phases/organize/useOrganization.js +++ b/src/renderer/phases/organize/useOrganization.js @@ -668,9 +668,11 @@ export function findInPlaceMoves({ files, smartFolders = [], defaultLocation, .. /** * The Nth numbered form of a destination, in the shape main gives a name - * that is taken (formatNumberedCollisionCandidate in batchOrganizeHandler.js): + * that is taken (formatNumberedCollisionCandidate in collisionNames.js): * `_N` before the last extension, so report.pdf becomes report_1.pdf. A name - * numbered here is the one the move uses, and the one Preview shows. + * numbered here is the one the move uses unless it is taken on disk; main + * then numbers from report again, past every name of the batch, and Preview + * shows the name it will use. * @param {string} destination * @param {number} counter - 1-based * @returns {string} @@ -772,8 +774,15 @@ function buildOperations({ suggestionsByPath }); const operations = planned.map((entry) => { - const { file, normalized, categoryChanged, originalCategory, finalCategory, suggestion } = - entry; + const { + file, + normalized, + categoryChanged, + originalCategory, + finalCategory, + suggestion, + renumbered + } = entry; const analysisLabel = analysisLabelOf(file); if (categoryChanged) { @@ -789,7 +798,9 @@ function buildOperations({ source: file.path, destination: normalized, ...(suggestion ? { suggestion } : {}), - ...(analysisLabel ? { analysisLabel } : {}) + ...(analysisLabel ? { analysisLabel } : {}), + // Main numbers a name taken on disk from the name before this `_N`. + ...(renumbered ? { renumbered: true } : {}) }; }); const renumberedSources = new Set( diff --git a/test/OrganizeResumeService.collisionRecovery.test.js b/test/OrganizeResumeService.collisionRecovery.test.js index 42f270d..9f1f50b 100644 --- a/test/OrganizeResumeService.collisionRecovery.test.js +++ b/test/OrganizeResumeService.collisionRecovery.test.js @@ -175,4 +175,76 @@ describe('OrganizeResumeService: crash after a collision-suffixed move', () => { destination: moved }); }); + + describe('a file Organize numbered apart from another of the batch', () => { + test('is found under the number counted from the name before its `_1`', async () => { + // The live batch numbers a taken r_1.txt from r (r_2.txt), not r_1_1.txt. + const source = path.join(testDir, 'inbox', 'b.txt'); + const planned = path.join(testDir, 'sorted', 'r_1.txt'); + const moved = path.join(testDir, 'sorted', 'r_2.txt'); + await fs.writeFile(planned, 'a different, longer occupant'); + await fs.writeFile(moved, 'moved'); + const movedStat = await fs.stat(moved); + processingState.getIncompleteOrganizeBatches.mockReturnValue([ + { + id: 'batch1', + operations: [ + { + source, + destination: planned, + renumbered: true, + status: 'in_progress', + sourceSize: movedStat.size, + sourceMtime: movedStat.mtimeMs + } + ] + } + ]); + + await resumeIncompleteBatches({ processingState }, logger, getMainWindow); + + expect(processingState.markOrganizeOpError).not.toHaveBeenCalled(); + expect(processingState.markOrganizeOpDone).toHaveBeenCalledWith('batch1', 0, { + destination: moved + }); + }); + + test('is moved under the number the live batch would give it', async () => { + await fs.mkdir(path.join(testDir, 'inbox'), { recursive: true }); + const sorted = (/** @type {string} */ name) => path.join(testDir, 'sorted', name); + const inbox = (/** @type {string} */ name) => path.join(testDir, 'inbox', name); + await fs.writeFile(sorted('r.txt'), 'OLD'); + await fs.writeFile(sorted('r_1.txt'), 'OLD1'); + await fs.writeFile(inbox('a.txt'), 'AAA'); + await fs.writeFile(inbox('b.txt'), 'BBB'); + processingState.getIncompleteOrganizeBatches.mockReturnValue([ + { + id: 'batch1', + operations: [ + { source: inbox('a.txt'), destination: sorted('r.txt'), status: 'pending' }, + { + source: inbox('b.txt'), + destination: sorted('r_1.txt'), + renumbered: true, + status: 'pending' + } + ] + } + ]); + + await resumeIncompleteBatches({ processingState }, logger, getMainWindow); + + expect(processingState.markOrganizeOpError).not.toHaveBeenCalled(); + // r_1 is the other file's name, so the first goes past it; the second + // counts on from r, not from its own number. + expect(processingState.markOrganizeOpDone).toHaveBeenCalledWith('batch1', 0, { + destination: sorted('r_2.txt') + }); + expect(processingState.markOrganizeOpDone).toHaveBeenCalledWith('batch1', 1, { + destination: sorted('r_3.txt') + }); + expect(await fs.readFile(sorted('r_2.txt'), 'utf8')).toBe('AAA'); + expect(await fs.readFile(sorted('r_3.txt'), 'utf8')).toBe('BBB'); + }); + }); }); diff --git a/test/batchOrganize.batchNumbering.realfs.test.js b/test/batchOrganize.batchNumbering.realfs.test.js new file mode 100644 index 0000000..44e91b0 --- /dev/null +++ b/test/batchOrganize.batchNumbering.realfs.test.js @@ -0,0 +1,247 @@ +/** + * @jest-environment node + */ +/** + * Organize numbers two files of one batch headed for the same name apart + * before it sends them (report.pdf, report_1.pdf). When a file of that name + * is already in the folder, the move numbers the file whose name is taken. + * Runs the real batch handler and dry-run preview against a temp dir. + * + * The failure this pins: the move did not know the batch's own numbered + * names, so with report.pdf in the folder Preview showed both files going to + * report_1.pdf while the move gave them report_1.pdf and report_2.pdf the + * other way round, and a taken report_1.pdf became report_1_1.pdf. + */ + +jest.unmock('fs'); +jest.unmock('fs/promises'); + +jest.mock('../src/main/services/TrashService', () => ({ + TrashService: class { + /** @param {string} filePath */ + async trashOne(filePath) { + const fsp = require('fs').promises; + const pathMod = require('path'); + const trashDir = pathMod.join(pathMod.dirname(pathMod.dirname(filePath)), '_TRASH'); + await fsp.mkdir(trashDir, { recursive: true }); + await fsp.rename(filePath, pathMod.join(trashDir, pathMod.basename(filePath))); + return { success: true }; + } + } +})); + +jest.mock('../src/main/ipc/files/embeddingSync', () => ({ + removeEmbeddingsForPathBestEffort: jest.fn(), + syncEmbeddingForMove: jest.fn(), + verifyEmbeddingAfterMove: jest.fn().mockResolvedValue({ valid: true, issues: [] }) +})); + +jest.mock('../src/main/ipc/files/batchLockManager', () => ({ + acquireBatchLock: jest.fn().mockResolvedValue(true), + renewBatchLock: jest.fn().mockReturnValue(true), + releaseBatchLock: jest.fn() +})); + +jest.mock('../src/main/ipc/files/batchProgressReporter', () => ({ + sendFileOperationBatchComplete: jest.fn(), + sendOperationProgress: jest.fn(), + sendChunkedResults: jest.fn().mockResolvedValue({ sent: true }) +})); + +jest.mock('../src/main/services/ServiceContainer', () => ({ + container: { has: jest.fn(() => false), resolve: jest.fn(), tryResolve: jest.fn(() => null) }, + ServiceIds: { FILE_PATH_COORDINATOR: 'filePathCoordinator', LEARNING_FEEDBACK: 'lf' } +})); + +jest.mock('../src/main/ipc/files/destinationPolicy', () => ({ + authorizeDestinations: jest.fn().mockResolvedValue(new Map()), + authorizeDestination: jest.fn().mockResolvedValue({ allowed: true }), + decisionForDestination: jest.fn(() => ({ allowed: true })), + buildStrictFileOpsAllowlist: jest.fn(() => ({ enabled: false, allowlist: [] })) +})); + +jest.mock('../src/shared/pathSanitization', () => ({ + validateFileOperationPath: jest.fn(async (p) => ({ + valid: true, + normalizedPath: require('path').resolve(p) + })), + normalizePathForIndex: jest.fn((p) => String(p).replace(/\\/g, '/')) +})); + +jest.mock('../src/main/services/FolderMatchingService', () => ({ + matchCategoryToFolder: jest.fn(() => null) +})); + +jest.mock('../src/main/ipc/semantic', () => ({ + getSearchServiceInstance: jest.fn(() => null) +})); + +const fs = require('fs'); +const fsp = fs.promises; +const path = require('path'); +const realOs = jest.requireActual('os'); + +const { + handleBatchOrganize, + previewBatchOrganize +} = require('../src/main/ipc/files/batchOrganizeHandler'); + +const silentLog = { info() {}, warn() {}, error() {}, debug() {} }; + +/** @type {string[]} */ +const tempRoots = []; + +afterAll(async () => { + for (const root of tempRoots) { + await fsp.rm(root, { recursive: true, force: true }); + } +}); + +/** + * @param {string} filePath + * @param {string} content + */ +async function writeFile(filePath, content) { + await fsp.mkdir(path.dirname(filePath), { recursive: true }); + await fsp.writeFile(filePath, content); +} + +/** + * Put `existing` (name -> content) in a Sorted folder and `incoming` in an + * Inbox, then preview and run the batch Organize sends for them. + * @param {Record} existing + * @param {{ name: string, content: string, to: string, renumbered?: boolean }[]} incoming + */ +async function runBatch(existing, incoming, logger = silentLog) { + const root = await fsp.mkdtemp(path.join(realOs.tmpdir(), 'ss-batch-numbering-')); + tempRoots.push(root); + const sorted = path.join(root, 'Sorted'); + await fsp.mkdir(sorted, { recursive: true }); + for (const [name, content] of Object.entries(existing)) { + await writeFile(path.join(sorted, name), content); + } + const operations = []; + for (const file of incoming) { + await writeFile(path.join(root, 'Inbox', file.name), file.content); + operations.push({ + type: 'move', + source: path.join(root, 'Inbox', file.name), + destination: path.join(sorted, file.to), + ...(file.renumbered ? { renumbered: true } : {}) + }); + } + const plan = await previewBatchOrganize({ operation: { operations }, log: silentLog }); + const result = await handleBatchOrganize({ + operation: { operations }, + logger, + getServiceIntegration: () => null, + getMainWindow: () => null + }); + /** @type {Record} */ + const onDisk = {}; + for (const name of (await fsp.readdir(sorted)).sort()) { + onDisk[name] = await fsp.readFile(path.join(sorted, name), 'utf8'); + } + /** @param {{ source: string, destination: string }} row */ + const landing = (row) => `${path.basename(row.source)} -> ${path.basename(row.destination)}`; + return { + preview: plan.wouldMove.map(landing), + moved: result.results.filter((row) => row.success).map(landing), + result, + onDisk + }; +} + +// What Organize sends for two exports that get the same new name: the first +// keeps it, the second is numbered `_1` and marked renumbered. +const twoReports = [ + { name: 'a.pdf', content: 'AAA', to: 'r.pdf' }, + { name: 'b.pdf', content: 'BBB', to: 'r_1.pdf', renumbered: true } +]; + +/** + * @param {Awaited>} run + * @param {string[]} expected + */ +function expectPreviewMatchesMove(run, expected) { + expect(run.result.success).toBe(true); + expect([...run.moved].sort()).toEqual([...expected].sort()); + expect([...run.preview].sort()).toEqual([...expected].sort()); + // Nothing is numbered twice. + expect(Object.keys(run.onDisk).filter((name) => /_\d+_\d+\./.test(name))).toEqual([]); +} + +describe('a batch Organize numbered apart, into a folder that already has the name', () => { + test('the file whose name is taken skips the number another file is headed for', async () => { + const run = await runBatch({ 'r.pdf': 'OLD' }, twoReports); + + expectPreviewMatchesMove(run, ['a.pdf -> r_2.pdf', 'b.pdf -> r_1.pdf']); + expect(run.onDisk).toEqual({ 'r.pdf': 'OLD', 'r_1.pdf': 'BBB', 'r_2.pdf': 'AAA' }); + }); + + test('a numbered name taken on disk counts on from the name, not from the number', async () => { + const run = await runBatch({ 'r.pdf': 'OLD', 'r_1.pdf': 'OLD1' }, twoReports); + + expectPreviewMatchesMove(run, ['a.pdf -> r_2.pdf', 'b.pdf -> r_3.pdf']); + expect(run.onDisk).toEqual({ + 'r.pdf': 'OLD', + 'r_1.pdf': 'OLD1', + 'r_2.pdf': 'AAA', + 'r_3.pdf': 'BBB' + }); + }); + + test('only the numbered name taken: the first file keeps its name', async () => { + const run = await runBatch({ 'r_1.pdf': 'OLD1' }, twoReports); + + expectPreviewMatchesMove(run, ['a.pdf -> r.pdf', 'b.pdf -> r_2.pdf']); + expect(run.onDisk).toEqual({ 'r.pdf': 'AAA', 'r_1.pdf': 'OLD1', 'r_2.pdf': 'BBB' }); + }); + + test('three files, two of their names taken: every file lands where Preview said', async () => { + const run = await runBatch({ 'r.pdf': 'OLD', 'r_2.pdf': 'OLD2' }, [ + ...twoReports, + { name: 'c.pdf', content: 'CCC', to: 'r_2.pdf', renumbered: true } + ]); + + expectPreviewMatchesMove(run, ['a.pdf -> r_3.pdf', 'b.pdf -> r_1.pdf', 'c.pdf -> r_4.pdf']); + expect(run.onDisk).toEqual({ + 'r.pdf': 'OLD', + 'r_1.pdf': 'BBB', + 'r_2.pdf': 'OLD2', + 'r_3.pdf': 'AAA', + 'r_4.pdf': 'CCC' + }); + }); +}); + +test('a name of the file’s own that ends in a number is numbered as before', async () => { + // Not marked renumbered: `_1` is part of the file's name, not Organize's. + const run = await runBatch({ 'notes_1.pdf': 'OLD' }, [ + { name: 'n.pdf', content: 'NNN', to: 'notes_1.pdf' } + ]); + + expect(run.moved).toEqual(['n.pdf -> notes_1_1.pdf']); + expect(run.preview).toEqual(['n.pdf -> notes_1_1.pdf']); + expect(run.onDisk).toEqual({ 'notes_1.pdf': 'OLD', 'notes_1_1.pdf': 'NNN' }); +}); + +describe('files numbered from one name run in batch order', () => { + const orderNote = /move in batch order/; + const spyLog = () => ({ ...silentLog, info: jest.fn() }); + /** @param {{ info: jest.Mock }} log */ + const ranInOrder = (log) => log.info.mock.calls.some(([message]) => orderNote.test(message)); + + test('only when one of their names is already taken', async () => { + const taken = spyLog(); + await runBatch({ 'r.pdf': 'OLD' }, twoReports, taken); + expect(ranInOrder(taken)).toBe(true); + }); + + test('not when every name is free, so they move alongside the rest of the batch', async () => { + const free = spyLog(); + const run = await runBatch({}, twoReports, free); + expect(ranInOrder(free)).toBe(false); + expect(run.onDisk).toEqual({ 'r.pdf': 'AAA', 'r_1.pdf': 'BBB' }); + }); +}); diff --git a/test/useOrganization.runFiles.test.js b/test/useOrganization.runFiles.test.js index e125446..24f12b3 100644 --- a/test/useOrganization.runFiles.test.js +++ b/test/useOrganization.runFiles.test.js @@ -300,7 +300,8 @@ describe('useOrganization: the per-file record of a run', () => { expect(options.addNotification.mock.calls.map((call) => call[0])).not.toEqual( expect.arrayContaining([expect.stringMatching(/Cannot organize/)]) ); - expect(options.executeAction.mock.calls[0][0].operations).toEqual([ + const sent = options.executeAction.mock.calls[0][0].operations; + expect(sent).toEqual([ expect.objectContaining({ source: exports[0].path, destination: '/Documents/Uncategorized/search-performance-2026-09-03.xlsx', @@ -309,9 +310,12 @@ describe('useOrganization: the per-file record of a run', () => { expect.objectContaining({ source: exports[1].path, destination: '/Documents/Uncategorized/search-performance-2026-09-03_1.xlsx', - analysisLabel: 'GSC exports' + analysisLabel: 'GSC exports', + // Main numbers a name taken on disk from the name before this `_1`. + renumbered: true }) ]); + expect(sent[0]).not.toHaveProperty('renumbered'); }); describe('a batch main refused before any file', () => { diff --git a/test/validationSchemas.test.js b/test/validationSchemas.test.js index bafcfb5..aaaa3ca 100644 --- a/test/validationSchemas.test.js +++ b/test/validationSchemas.test.js @@ -362,6 +362,20 @@ describe('validationSchemas (Zod path)', () => { expect(result.data.operations[0].analysisLabel).toBe('Finance'); }); + // And for the numbered-apart mark: without it main numbers a taken + // report_1.pdf as report_1_1.pdf. + test('preserves the per-op renumbered mark and refuses a non-boolean one', () => { + const parse = (renumbered) => + schemas.fileOperation.safeParse({ + type: 'batch_organize', + operations: [{ source: '/a/r.pdf', destination: '/b/r_1.pdf', renumbered }] + }); + const kept = parse(true); + expect(kept.success).toBe(true); + expect(kept.data.operations[0].renumbered).toBe(true); + expect(parse('yes').success).toBe(false); + }); + test('rejects a suggestion with a relative path', () => { const result = schemas.fileOperation.safeParse({ type: 'batch_organize',