Skip to content

fix(markdown): show a text with unclosed spans or many lists as plain - #472

Merged
ixuz merged 2 commits into
mainfrom
fix/markdown-code-span-html-lists
Nov 1, 2026
Merged

ixuz merged 2 commits into
mainfrom
fix/markdown-code-span-html-lists

Conversation

@ixuz

@ixuz ixuz commented Nov 1, 2026

Copy link
Copy Markdown
Member

What and why

The Markdown parser takes time quadratic in two more constructs that the cost guard did not count. The first is an opener that never closes: a backtick run, <!, <? or a link title. The parser reads such an opener to the end of its paragraph, so a paragraph with many of them is slow to read. The second is a text that opens many separate lists. A text within MAX_CHARS could hold enough of either to hold up the main thread each time it was first read: on a card, on a course page or question, and in the Studio's release check. A release can hold many such texts.

The guard in packages/markdown/src/parse.ts now counts both:

  • New limit, MAX_LOOKAHEAD (500,000 characters). It caps how far the parser may read ahead for the end of a span or title, added up over the text. Each opener counts only to its closer: a backtick run to the next run of its length, <!-- to -->, <? to ?>, a link's title to its closing quote or paren, or to the next blank line when none comes. Closed spans, as in a long list or table of commands, therefore count about once, and only openers that never close add up to the cap. The count itself takes time linear in the text and stops once past the cap.
  • New limit, MAX_LISTS (200). It caps the lists a text opens. A list's next items, and the lines its items hold, open none. An item that starts empty ends at a blank line right after it, because an item can start with at most one blank line. A marker that cannot interrupt a paragraph (an empty item, or an ordered item that does not start at 1) opens no list while a paragraph is open.
  • The list count errs high, on purpose. A marker counts wherever it is, in fenced code or an HTML block too, and a lazy line leaves the list. Following micromark's block rules exactly to skip code blocks turned out to be easy to get wrong, so the guard does not try. The cost: a card whose fenced code holds more than about 200 list-like lines (a long diff), or more than 200 list items whose lines wrap without indentation, shows as plain text. No text in the library comes near either.
  • Past either limit. A text past either limit is shown as plain text, the same as past the guard's other limits. The pathological tests pin each shape at the cap and past it, and other tests check that a tight list, a table and a long paragraph full of closed code spans still read as Markdown.
  • Repository pages. parseDocument lifts MAX_LOOKAHEAD together with MAX_CHARS, because the count grows with a page's length. It refuses a page past MAX_LISTS.

One existing test changed: "a\n- b\n".repeat(MAX_UNDERLINES + 1) now goes past MAX_LISTS first, so the MAX_UNDERLINES test uses "a\n- b\n- c\n" instead.

docs/markdown.md lists both limits and says how they are counted.

Refs: GHSA-25hm-hjrf-w8fh

Checklist

  • One issue. This pull request addresses one issue, linked above. (the advisory referenced above)
  • One step. This pull request does not finish its issue, so it says Refs:, not Closes:. Leave this unchecked otherwise.
  • Done, as AGENTS.md defines it: npm run check passes (lint, Markdown and spelling included), npm run lint:reuse too if a file was added, and npm run crosscheck too if ns/ or decks/ changed. (lint:reuse and crosscheck n/a: no file was added, and ns/ and decks/ are unchanged)
  • Tests. New and changed code has its tests, and every package stays at 100% coverage (docs/testing.md).
  • Docs. The docs say what changed, and a new page in docs/ is listed in docs/docs.ttl. (no new page)
  • Languages. A new or changed message is in every locale: en, sv, ko, de, es and fr (packages/ui/src/i18n/). (n/a: no message added or changed)
  • Vocabulary and shapes. Any change to ns/ was agreed in an issue first; no published ns/shapes/*/vN.ttl or decks/*/vN.ttl was edited; generated files were regenerated, not edited. (n/a: ns/, decks/ and generated files are unchanged)
  • Breaking? Nothing in the public API breaks, or a commit says so (! and a BREAKING CHANGE: footer). npm run check:breaking sees a line of api/ that goes, but not a feature removed, nor a reader that reads less than before. (check:breaking passes, and every deck in the library still passes library:check)
  • Pod data. Any change to what the app writes to a pod comes with its migration (docs/migrations.md). (n/a: nothing written to a pod changes)
  • Policy. Any change to an external rule the code follows (SM-2, the plural rules, a specification) is cited. (no external rule changes; the counts follow how CommonMark 0.31.2 closes spans and starts lists)
  • Licenses. New content is MIT, or a deck states its own license; a new file has its SPDX header (npm run lint:reuse), or a new deck its line in REUSE.toml; third-party content is credited. (n/a: no file, deck or third-party content added)
  • Architecture. Imports follow docs/boundaries.md and the layers of docs/architecture.md.
  • Run. I ran the app (npm run dev) and tried the change, and considered the user journeys.
  • Authorship. I read and understand every line of this pull request, and answer for it, whatever tools helped write it (CONTRIBUTING.md).
  • Commits. Each commit is one logical change, in Conventional Commits form (type(scope): subject, ! and a BREAKING CHANGE: footer if it breaks something), with an imperative subject and a body that explains why (CONTRIBUTING.md).

The parser takes time quadratic in two more constructs that the cost
guard did not count. A backtick run, `<!` or `<?` that never closes, and
a link title that never closes, is read to the end of its paragraph, so
a paragraph of many such openers, escaped code spans, unclosed inline
HTML comments, processing instructions, declarations and link titles
among them, is slow to read. So is a text that opens many separate
lists, each ended by a paragraph, a thematic break or another marker, at
the top level or in one quote. A text within MAX_CHARS could hold enough
of either to hold up the main thread each time it was first read, on a
card, a course page or question, and in the Studio's release check, and
a release can hold many such texts.

The guard now counts both. MAX_LOOKAHEAD, 500,000, caps the characters
the parser may read ahead for a span's or title's end, added up over
the text: each opener counts to its closer, a backtick run to the next
run of its length, `<!--` to `-->`, a link's title to its quote or
paren, or to the next blank line when none comes. So closed spans, as
in a long list or table of commands, count about once over, and only
openers that never close add up to the cap. The count reads the text
only as far as it counts and stops past the cap. MAX_LISTS, 200, caps
the lists a text opens: a list's next items and the lines its items
hold open none. An item that starts empty ends at a blank line right
after it, as an item can start with at most one blank line, so the line
after leaves its list. A marker that cannot interrupt a paragraph, an
empty item or an ordered one that starts other than at 1, opens none
while a paragraph is surely open. A marker counts wherever it is, in
fenced code or an HTML block too, and a lazy line leaves the list, so
the count errs high, and a fence holding a long diff can pass it. A
text past either is shown as plain text, as past the guard's other
limits; the pathological tests pin both shapes, at the cap and past it.

No text in the library's decks or courses comes near either limit, and
none is newly shown as plain. parseDocument lifts MAX_LOOKAHEAD with
MAX_CHARS, as the count grows with a page's length, and refuses a page
past MAX_LISTS. The Markdown docs list both limits.

Refs: GHSA-25hm-hjrf-w8fh
@ixuz ixuz added the bug Something isn't working label Nov 1, 2026
@ixuz
ixuz enabled auto-merge November 1, 2026 04:29
Comment thread packages/markdown/src/parse.ts Fixed
CodeQL's js/bad-tag-filter flagged the `-->` pattern the cost guard
reads inline HTML comments with, as it does not take `--!>`. The guard
is no HTML filter: it counts how far the parser reads ahead, and the
parser closes a comment at `-->` only, as CommonMark does, so a count
that stopped at `--!>` would count low and let the quadratic input
back in. The closer is now found as a `--` before a `>`, which counts
the same characters in the same linear time, and a comment says why
`--!>` does not count.

Refs: #472
@ixuz
ixuz added this pull request to the merge queue Nov 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 1, 2026
@ixuz
ixuz added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit f4e3625 Nov 1, 2026
50 checks passed
@ixuz
ixuz deleted the fix/markdown-code-span-html-lists branch November 1, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants