fix: protect code cell options from formatting (#655)

* fix(format): strip cell option directives before formatting embedded code

Quarto code cells carry metadata on leading comment lines (`#| label:
foo`, `#| echo: false`, `//| label: bar`, ...) that must survive
verbatim so Quarto can parse them on the next render. Previously,
`formatBlock` handed the entire cell to the language formatter (Black,
autopep8, styler, ...), which treats these lines as ordinary comments
and may reflow, reorder, or rewrite them.

Hide the option lines from the formatter entirely by slicing them off
`blockLines` before calling `virtualDocForCode`, and shift the returned
edits back into the real code range via `block.range.start.line + 1 +
optionLines`. Blocks that are entirely option directives short-circuit
to `undefined` (no formatter invocation, no out-of-range toast).

This replaces the filter-after approach that dropped any edit whose
start line landed in the option region, which quietly swallowed valid
multi-line edits and block-wide edits starting at virtual-doc line 0.

* test(format): cover cell option protection with deterministic fixtures

Add a `Code Block Formatting` suite that registers a fake Python
formatter from a pure `string -> string` function, so each case can
prove exactly what did and did not get rewritten. The fake formatters
mangle `=` and `+` into spaced forms, rewrite `#| ` to `# PIPE`, and
normalise comment spacing.

Cover six scenarios: a single directive followed by code, multiple
directives preserved in original order, a block with no directives,
a directives-only block (no-op), multiline code with badly-formatted
standalone and inline comments, and a hostile formatter run against a
mixed block to prove the directives never reach the formatter at all.

* docs: note cell option formatting fix in CHANGELOG

* fix(format): protect option directives via canonical regex and per-block formatting

Close three gaps discovered in adversarial review of the initial fix:

1. The `languageOptionComment` helper used a private map keyed by short
   language ids (e.g. `python`, `r`, `js`). It didn't know about
   `typescript`, so `//| ...` directives in `ts` cells were never
   stripped and still reached the formatter. Use `language.comment` from
   editor-core instead, which is the canonical, language-wide mapping.

2. The option-line detection used `startsWith("<comment>| ")`, which
   only matched the canonical form. Quarto's own cell-option parser
   (`cell/options.ts`) accepts `^<comment>\s*\| ?`, so variants like
   `#|label`, `# | label`, and `#|  label` are legal. Switch to the
   same canonical regex so every variant is protected.

3. `embeddedDocumentFormattingProvider` bypassed `formatBlock` entirely
   for languages with `canFormatDocument !== false` (R, Julia, TS, ...),
   handing the whole virtualised document to the formatter. Option
   lines inside every block leaked through. Route every target block
   through `formatBlock` instead, so the same strip-before-format path
   protects both cell-format and document-format invocations.

* test(format): cover whitespace variants, typescript, and document formatting

Add three fixtures and three regression tests for the gaps fixed in the
previous commit:

- `format-python-option-variants.qmd` exercises every whitespace
  variant the Quarto cell-option parser accepts (`#|label`,
  `# | label`, `#|  label`). The accompanying test runs an aggressive
  formatter that rewrites any `#\s*\|` line to `# MANGLED`; with the
  canonical-regex strip in place, no directive ever reaches the
  formatter, so `# MANGLED` must not appear.

- `format-typescript.qmd` covers the `//|` directive form. Before
  switching to `language.comment` the `ts` language had no entry in
  the private comment map and the directive was unprotected; this
  test would have silently allowed the hostile rewrite.

- `format-r-multiple-blocks.qmd` drives the document-formatting path
  (formerly bypassed `formatBlock` for languages with `canFormatDocument
  !== false`). The test invokes `embeddedDocumentFormattingProvider`
  directly since the LSP middleware isn't wired up in the vscode-test
  host, and asserts that every block's directives survive while the
  code itself is reformatted.

The two new helpers `hostileRFormatter` and `hostileTypeScriptFormatter`
use the same canonical regex pattern to decide what to mangle, so they
are realistic stand-ins for any formatter that treats `#|`/`//|` lines
as ordinary comments.

* refactor(format): reuse canonical option pattern and harden doc-format path

Address adversarial review findings on the option-protection patch:

- Reuse `optionCommentPattern` from `cell/options.ts` instead of
  re-encoding the regex in `format.ts`. The protection path can no
  longer drift from Quarto's own cell-option parser, and the local
  `escapeRegExp` helper is gone.
- Make document formatting all-or-nothing: aggregate per-block bails
  into a single message and abort the whole operation rather than
  apply a partial format. Per-block toasts are silenced via a new
  `silentOutOfRange` flag on `formatBlock` so the doc path no longer
  spams a message per failing block.
- Thread `defaultOptions` through `embeddedDocumentRangeFormattingProvider`
  and `FormatCellCommand`, both of which previously fell back to a
  hardcoded `tabSize: 4 / insertSpaces: true` regardless of the user's
  editor settings.
- Mark `languageOptionComment` as private in `option.ts` since it has
  no remaining external consumers.

* test(format): exercise real edit-apply path and multi-edit aggregation

- Replace the direct provider call in the document-formatting test
  with a real `vscode.executeFormatDocumentProvider` invocation. The
  test now registers `embeddedDocumentFormattingProvider` against the
  `quarto` language and lets VS Code's command machinery validate
  ordering and overlap. Pairwise non-overlap is asserted explicitly so
  a future regression in offset arithmetic surfaces here.
- Add a test that exercises a formatter returning one `TextEdit` per
  non-empty line, covering the multi-edit aggregation path that all
  previous hostile formatters skipped.
- Wrap both new tests in try/finally so a failed assertion can never
  leak a registered formatter into a later suite.
- Drop the misleading comment about inject lines from
  `mangleHashPipeLines` and remove an unnecessary `wait(450)` from
  the document-formatting test.

* fix(format): harden edge cases from adversarial review

- Treat empty formatter results as no-op (not out-of-range bail) in both
  formatBlock and FormatCellCommand, preventing spurious error toasts.
- Use trim() in the options-only guard so whitespace-only tails after
  option directives are correctly treated as empty.
- Wrap testFormatter disposal in try/finally to prevent leaked formatter
  registrations on test failure.
- Document why block-comment languages are not affected by the option
  stripping logic.

---------

Co-authored-by: Elliot <key.draw@gmail.com>

Author: mcanouil

apps/vscode/CHANGELOG.md

@@ -5,6 +5,7 @@
 - Added support for Positron's statement execution feature that reports the approximate line number of the parse error (<https://github.com/quarto-dev/quarto/pull/919>).
 - Fixed a bug where `Quarto: Format Cell` would notify you that no formatter was available for code cells that were already formatted (<https://github.com/quarto-dev/quarto/pull/933>).
 - No longer claim `.typ` files. Typst syntax highlighting in Quarto documents is unaffected, but standalone Typst files are now left to dedicated extensions like Tinymist (<https://github.com/quarto-dev/quarto/pull/943>).
+- Preserve Quarto code cell option directives (e.g. `#| label: foo`) when formatting embedded code. The directives are now stripped from the virtual document before being handed to the language formatter, so formatters such as Black, autopep8, and styler can no longer reflow or rewrite them (<https://github.com/quarto-dev/quarto/pull/655>).
 - Fixed a bug where closing the Quarto Preview terminal via the trash icon did not clean up intermediate `.quarto_ipynb` files (<https://github.com/quarto-dev/quarto/pull/947>).
 
 ## 1.130.0 (Release on 2026-02-18)

apps/vscode/src/providers/cell/options.ts

@@ -91,7 +91,7 @@ function langCommentChars(lang: string): string[] {
     return chars;
   }
 }
-function optionCommentPattern(comment: string) {
+export function optionCommentPattern(comment: string) {
   return new RegExp("^" + escapeRegExp(comment) + "\\s*\\| ?");
 }
 

apps/vscode/src/providers/format.ts

@@ -35,14 +35,15 @@ import { TokenCodeBlock, TokenMath, codeForExecutableLanguageBlock, languageBloc
 import { Command } from "../core/command";
 import { isQuartoDoc } from "../core/doc";
 import { MarkdownEngine } from "../markdown/engine";
+import { optionCommentPattern } from "./cell/options";
 import { EmbeddedLanguage, languageCanFormatDocument } from "../vdoc/languages";
 import {
+  isBlockOfLanguage,
   languageFromBlock,
   mainLanguage,
   unadjustedRange,
   VirtualDoc,
   virtualDocForCode,
-  virtualDocForLanguage,
   withVirtualDocUri,
 } from "../vdoc/vdoc";
 
@@ -90,22 +91,42 @@ export function embeddedDocumentFormattingProvider(engine: MarkdownEngine) {
       return [];
     }
 
-    if (languageCanFormatDocument(language)) {
-      // Full document formatting support
-      const vdoc = virtualDocForLanguage(document, tokens, language);
-      return executeFormatDocumentProvider(
-        vdoc,
-        document,
-        formattingOptions(document.uri, vdoc.language, options)
+    // If the selected language supports whole-document formatting, format
+    // every block of it; otherwise, format only the cell containing the
+    // cursor. Either way, each block is routed through `formatBlock` so
+    // that Quarto option directives are protected by the same
+    // strip-before-format path and can't leak to the language formatter.
+    const targetBlocks: (TokenMath | TokenCodeBlock)[] = languageCanFormatDocument(language)
+      ? (tokens.filter(isBlockOfLanguage(language)) as (TokenMath | TokenCodeBlock)[])
+      : block
+        ? [block]
+        : [];
+
+    // Document formatting is all-or-nothing: if any block fails the
+    // out-of-range guard, abort the whole operation rather than apply a
+    // partial format. We pass `silentOutOfRange: true` so per-block
+    // failures don't toast individually; a single aggregated message is
+    // shown below.
+    const allEdits: TextEdit[] = [];
+    let outOfRangeBlockFailures = 0;
+    for (const target of targetBlocks) {
+      const edits = await formatBlock(document, target, options, true);
+      if (edits === undefined) {
+        continue;
+      }
+      if (edits.length === 0) {
+        outOfRangeBlockFailures++;
+        continue;
+      }
+      allEdits.push(...edits);
+    }
+    if (outOfRangeBlockFailures > 0) {
+      window.showInformationMessage(
+        `Formatting edits were out of range in ${outOfRangeBlockFailures} code cell${outOfRangeBlockFailures === 1 ? "" : "s"}; document was not modified.`
       );
-    } else if (block) {
-      // Just format the selected block if there is one
-      const edits = await formatBlock(document, block);
-      return edits ? edits : [];
-    } else {
-      // Nothing we can format
       return [];
     }
+    return allEdits;
   };
 }
 
@@ -145,7 +166,7 @@ export function embeddedDocumentRangeFormattingProvider(
       return [];
     }
 
-    const edits = await formatBlock(document, block);
+    const edits = await formatBlock(document, block, options);
     if (!edits) {
       return [];
     }
@@ -180,9 +201,15 @@ class FormatCellCommand implements Command {
       return;
     }
 
-    const edits = await formatBlock(document, block);
-    if (!edits) {
-      // Nothing to do! Already formatted, or no formatter picked us up, or this language doesn't support formatting.
+    const editorOptions: FormattingOptions = {
+      tabSize: typeof editor.options.tabSize === "number" ? editor.options.tabSize : 4,
+      insertSpaces: typeof editor.options.insertSpaces === "boolean" ? editor.options.insertSpaces : true,
+    };
+    const edits = await formatBlock(document, block, editorOptions);
+    if (!edits || edits.length === 0) {
+      // Nothing to do! Already formatted, no formatter picked us up, this
+      // language doesn't support formatting, or the edits were out of range
+      // (the user already saw a toast from formatBlock in that case).
       return;
     }
 
@@ -235,7 +262,12 @@ async function executeFormatDocumentProvider(
   }
 }
 
-async function formatBlock(doc: TextDocument, block: TokenMath | TokenCodeBlock): Promise<TextEdit[] | undefined> {
+async function formatBlock(
+  doc: TextDocument,
+  block: TokenMath | TokenCodeBlock,
+  defaultOptions?: FormattingOptions,
+  silentOutOfRange: boolean = false
+): Promise<TextEdit[] | undefined> {
   // Extract language
   const language = languageFromBlock(block);
   if (!language) {
@@ -247,44 +279,86 @@ async function formatBlock(doc: TextDocument, block: TokenMath | TokenCodeBlock)
     return undefined;
   }
 
-  // Create virtual document containing the block
   const blockLines = lines(codeForExecutableLanguageBlock(block, false));
-  const vdoc = virtualDocForCode(blockLines, language);
+
+  // Count leading Quarto option directives (e.g. `#| label: foo`) so we can
+  // hide them from the formatter entirely. Feeding these lines to formatters
+  // like Black or styler risks reflowing or rewriting them, which would
+  // silently break the cell's behaviour on the next render. We reuse
+  // `optionCommentPattern` from `cell/options.ts` so this code path can
+  // never drift from Quarto's own cell-option parser: any variant the
+  // executor recognises (`#| label`, `#|label`, `# | label`, `#|  label`,
+  // ...) is automatically protected here. `language.comment` is the
+  // canonical comment string from `editor-core` and covers every formatter
+  // language (including TypeScript, which was missing from the ad-hoc map
+  // the previous implementation used). Note: block-comment languages (C,
+  // CSS, SAS) use a tuple comment-char in `cell/options.ts` with a suffix
+  // check; those languages do not have `canFormat: true` in
+  // `vdoc/languages.ts`, so they never reach this code path.
+  const optionPattern = language.comment
+    ? optionCommentPattern(language.comment)
+    : undefined;
+  let optionLines = 0;
+  if (optionPattern) {
+    for (const line of blockLines) {
+      if (optionPattern.test(line)) {
+        optionLines++;
+      } else {
+        break;
+      }
+    }
+  }
+
+  // Create virtual document containing only the code portion of the block
+  // so the formatter never sees the option directives.
+  const codeLines = blockLines.slice(optionLines);
+
+  // Nothing to format if the block is entirely option directives (or only
+  // trailing whitespace after them, which `lines()` may produce from a
+  // final newline in `token.data`).
+  if (codeLines.every(l => l.trim() === "")) {
+    return undefined;
+  }
+  const vdoc = virtualDocForCode(codeLines, language);
 
   const edits = await executeFormatDocumentProvider(
     vdoc,
     doc,
-    formattingOptions(doc.uri, vdoc.language)
+    formattingOptions(doc.uri, vdoc.language, defaultOptions)
   );
 
-  if (!edits) {
+  if (!edits || edits.length === 0) {
     // Either no formatter picked us up, or there were no edits required.
     // We can't determine the difference though!
     return undefined;
   }
 
   // Because we format with the block code copied in an empty virtual
   // document, we need to adjust the ranges to match the edits to the block
-  // cell in the original file.
+  // cell in the original file. The `+ 1` skips the opening fence line and
+  // `+ optionLines` skips the leading option directives we hid from the
+  // formatter.
+  const lineOffset = block.range.start.line + 1 + optionLines;
   const blockRange = new Range(
     new Position(block.range.start.line, block.range.start.character),
     new Position(block.range.end.line, block.range.end.character)
   );
-  const adjustedEdits = edits
-    .map(edit => {
-      const range = new Range(
-        new Position(edit.range.start.line + block.range.start.line + 1, edit.range.start.character),
-        new Position(edit.range.end.line + block.range.start.line + 1, edit.range.end.character)
-      );
-      return new TextEdit(range, edit.newText);
-    });
+  const adjustedEdits = edits.map(edit => {
+    const range = new Range(
+      new Position(edit.range.start.line + lineOffset, edit.range.start.character),
+      new Position(edit.range.end.line + lineOffset, edit.range.end.character)
+    );
+    return new TextEdit(range, edit.newText);
+  });
 
   // Bail if any edit is out of range. We used to filter these edits out but
   // this could bork the cell. Return `[]` to indicate that we tried.
   if (adjustedEdits.some(edit => !blockRange.contains(edit.range))) {
-    window.showInformationMessage(
-      "Formatting edits were out of range and could not be applied to the code cell."
-    );
+    if (!silentOutOfRange) {
+      window.showInformationMessage(
+        "Formatting edits were out of range and could not be applied to the code cell."
+      );
+    }
     return [];
   }
 

apps/vscode/src/test/examples/format-python-multiline-with-comments.qmd

@@ -0,0 +1,14 @@
+---
+title: Formatting Multiline Python Code Cells with Comments
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{python}
+#| label: multiline
+#| echo: false
+#standalone comment
+x=1  #inline comment
+y=2
+z=x+y
+```

apps/vscode/src/test/examples/format-python-multiple-options.qmd

@@ -0,0 +1,12 @@
+---
+title: Formatting Python Code Cells with Multiple Options
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{python}
+#| label: multi
+#| echo: false
+#| warning: false
+x=1;y=2;z=x+y
+```

apps/vscode/src/test/examples/format-python-no-options.qmd

@@ -0,0 +1,9 @@
+---
+title: Formatting Python Code Cells without Options
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{python}
+x=1;y=2;z=x+y
+```

apps/vscode/src/test/examples/format-python-only-options.qmd

@@ -0,0 +1,9 @@
+---
+title: Formatting Python Code Cells with Only Options
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{python}
+#| label: only-options
+```

apps/vscode/src/test/examples/format-python-option-variants.qmd

@@ -0,0 +1,12 @@
+---
+title: Formatting Python Code Cells with Option Directive Variants
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{python}
+#|label: no-space
+# | space-before-pipe
+#|  extra-space-after
+x=1;y=2;z=x+y
+```

apps/vscode/src/test/examples/format-python.qmd

@@ -0,0 +1,10 @@
+---
+title: Formatting Python Code Cells
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{python}
+#| label: my-code
+x=1;y=2;z=x+y
+```

apps/vscode/src/test/examples/format-r-multiple-blocks.qmd

@@ -0,0 +1,18 @@
+---
+title: Formatting an R Document with Multiple Blocks
+subtitle: https://github.com/quarto-dev/quarto/pull/655
+format: html
+---
+
+```{r}
+#| label: first
+x<-1
+```
+
+Some prose between the cells.
+
+```{r}
+#| label: second
+#| echo: false
+y<-2
+```