Commit 55d91bf
feat: smart chunking with break-point scoring algorithm
Port qmd's scored break-point chunking to Rust. Replaces heading-only
splitting with a scoring system that finds optimal break points near
the token target. Code fence protection prevents splitting inside
code blocks. 15% overlap for context continuity.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent f2e8a15 commit 55d91bf
1 file changed
Lines changed: 486 additions & 67 deletions
0 commit comments