Contact Information
talk about issue
MaxKB Version
v1.10.x ~ v2.x (all versions affected)
Problem Description
When splitting Markdown/PDF documents by titles, if a heading's text also happens to appear as a substring inside the body content (either earlier in the same section or in a later section), the splitter cuts the paragraph at the wrong position, causing part of the real body text to be lost from the resulting paragraph.
This happens because SplitModel.parse_to_tree and get_level_block locate the start/end of each block using text.index(start_content, cursor) / text.index(next_content, ...) — i.e., they search for the first occurrence of the title string in the text, rather than using the actual character offset returned by the regex match that originally found the title. If the title string coincidentally recurs inside the body, index() locks onto the wrong occurrence, mis-defining the block boundaries.
实际发现就是此时，标题与正文内容重叠时，导致分段截断错误，正文丢失、混乱！
Steps to Reproduce
- Create a Markdown (or PDF) document where a heading's exact text also appears verbatim inside the body of a preceding or following section, e.g.:
# Chapter 1
This chapter talks about Chapter 1 and Chapter 2 in advance, giving context before diving in.
More real content for chapter 1 continues here...
# Chapter 2
Actual content of Chapter 2... some foo bar you know this is not CARE Chapter 1 just title Chapter 2 xxx
2. Upload this document to a MaxKB knowledge base and let it auto-split by title (default pattern-based split).
3. Inspect the resulting paragraph for "Chapter 1" in the paragraph list.
### The expected correct result
The paragraph for "Chapter 1" should contain the full body text: "This chapter talks about Chapter 2 in advance, giving context before diving in. More real content for chapter 1 continues here..."
Actual Result
Because get_level_block finds "Chapter 2" via text.index(next_content, start_index + 1), and the substring "Chapter 2" first appears inside the Chapter 1 body (in "talks about Chapter 2 in advance"), the block is truncated there. The paragraph content ends up being only "This chapter talks about " and the rest of Chapter 1's real content ("giving context before diving in. More real content for chapter 1 continues here...") is silently dropped from the paragraph, moved incorrectly, or lost.
### Related log output
```shell
There is no directly associated log data. This constitutes a quality bug that manifests silently.
Additional Information
Suggested Fix Direction
Instead of re-locating title strings via text.index/text.find after they've already been captured by regex, retain and reuse the actual match.start()/match.end() offsets from re_findall/parse_level when slicing blocks, so that boundary detection is based on position rather than content re-matching.
Contact Information
talk about issue
MaxKB Version
v1.10.x ~ v2.x (all versions affected)
Problem Description
When splitting Markdown/PDF documents by titles, if a heading's text also happens to appear as a substring inside the body content (either earlier in the same section or in a later section), the splitter cuts the paragraph at the wrong position, causing part of the real body text to be lost from the resulting paragraph.
This happens because
SplitModel.parse_to_treeandget_level_blocklocate the start/end of each block usingtext.index(start_content, cursor)/text.index(next_content, ...)— i.e., they search for the first occurrence of the title string in the text, rather than using the actual character offset returned by the regex match that originally found the title. If the title string coincidentally recurs inside the body,index()locks onto the wrong occurrence, mis-defining the block boundaries.Steps to Reproduce
Additional Information
Suggested Fix Direction
Instead of re-locating title strings via text.index/text.find after they've already been captured by regex, retain and reuse the actual match.start()/match.end() offsets from re_findall/parse_level when slicing blocks, so that boundary detection is based on position rather than content re-matching.