Repository navigation
Fix quadratic-time backtracking in attr_list attribute regex - #1656
Conversation
`AttrListTreeprocessor.BASE_RE` matched the attribute content with
`([^\}\n ][^\n]*)` and then allowed trailing spaces before the closing
brace with a separate `[ ]*`. Because `[^\n]*` already matches spaces,
the two groups both cover the same run of trailing spaces. When a block
ends with an unclosed attribute list, the closing brace is never found,
so for a run of n spaces the engine retries the match from each of the
n+1 ways to split the spaces between the two groups, which is O(n^2).
The trailing `[ ]*` can only ever match zero characters in a successful
match (the greedy `[^\n]*` consumes the spaces first), so dropping it
leaves every match and capture group unchanged while removing the
ambiguity. A two-line block such as `text\n{a` followed by 200k spaces
drops from ~10s to ~20ms, and the cost is now linear in the input size.
|
Please review our AI Policy. Specifically, the following:
You description reads like it was written by AI. Also, as the PR template was deleted, I am assuming you had AI autonomously submit the PR. This presents 3 problems:
That said, it is possible that this was not AI generated. If you can honestly confirm that, then we will proceed. Otherwise, this will be closed. PS to other maintainers: this is going to be my standard response to AI submitted PRs/issues from now on. Feel free to respond in kind if you see a new PR first. |
|
Hi @waylan, I apologize for this, I used AI for the wordings and stuff and pasted it without noticing. If the policy means this has to be closed, I understand. I’m happy to resubmit with the template filled in and the AI use disclosed, or to open an issue instead so a maintainer can make the fix. |
|
It is fine of you provide thevinfo requested in the template. |
|
I've filled in the AI disclosure section of the template in the description. Claude was used for the description wording and the docstring follow-up. |
waylan
added
approved
2 participants
Description
AttrListTreeprocessor.BASE_REin theattr_listextension is\{\:?[ ]*([^\}\n ][^\n]*)[ ]*\}. The capture group[^\n]*already eats spaces, and the[ ]*after it eats spaces too, so a run of trailing spaces can be split between the two pieces in n+1 ways. That's harmless when a}is found, but when a block ends with an unclosed{followed by lots of spaces there is no}, and the engine tries every split before giving up. The time grows quadratically with the number of spaces.I noticed this while reading the regex, and confirmed it with a timing run against
markdown.markdown(text, extensions=['attr_list'])on'text\n{a' + ' ' * n:Doubling the spaces roughly quadruples the time before the change, and roughly doubles it after.
The fix is to drop the trailing
[ ]*. It can never match anything in a successful match anyway: the greedy[^\n]*already consumes the trailing spaces and only gives back what\}needs, so the[ ]*is always empty. Removing it leaves every match and capture group the same (I checked old vs new patterns against the existing attr_list test cases, valid attribute lists, and a few thousand random brace/space/tab strings) and takes away the ambiguity, so the failing case is now linear. This affectsHEADER_RE,BLOCK_REandINLINE_RE, which are all built fromBASE_RE.It's the same sort of problem as the
ReferenceProcessorfix in #798. Added a regression test with a 50k-space input and a changelog entry.AI Assistance Disclosure
Claude (via Claude Code) was used to help draft this PR description and to make the follow-up docstring change. I reviewed the change and can explain it.
Checklist