test(ecosystem): dogfood devtools-category modules with smoke tests - #1046
Conversation
Expand the ecosystem playground with the Nuxt "Devtools" category modules that register a real DevTools tab (@nuxt/eslint, @nuxt/hints, @nuxt/a11y, @compodium/nuxt, @scalar/nuxt), each with a small fixture so its tab has real data to show. Add an opt-in Playwright suite that boots the combined app and asserts every module's DevTools tab renders against the built devtools client. Align the sealed workspace to Nuxt 4.5 + Vite 8 (the linked @nuxt/devtools's @vitejs/devtools requires Vite 8), and render Scalar's /docs client-side to sidestep an SSR crash on this stack. Findings and runbook captured in the playground's README/REPORTS.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
📝 WalkthroughWalkthroughThe ecosystem playground now includes additional DevTools-category Nuxt modules, updated dependency and runtime configuration, an ESLint config export, a demo component, an accessibility finding, and an OpenAPI widgets endpoint. A dedicated Playwright configuration and smoke suite verify custom DevTools tabs. README and report documentation describe the expanded module set, test workflow, observed behavior, and compatibility findings. Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@playgrounds-ecosystem/modules/package.json`:
- Around line 12-25: Update the dependency entries in the package manifest shown
in the diff to use the centrally managed catalog references from the workspace
catalog instead of raw version ranges. Apply this to each catalog-managed
package in both dependencies and devDependencies, preserving the existing
package names and leaving non-catalog entries unchanged.
In `@playgrounds-ecosystem/REPORTS.md`:
- Line 70: Convert the “Addendum 3 — Devtools-category modules added” text in
REPORTS.md from bold paragraph formatting to a Markdown heading, preferably
using the same heading hierarchy as the surrounding addenda, so it generates the
existing `#addendum-3--devtools-category-modules-added` anchor referenced
elsewhere.
In `@playgrounds-ecosystem/tests/ecosystem-modules.spec.ts`:
- Around line 58-62: Update the iframe assertion in the tab.view === 'iframe'
branch to obtain the matching iframe’s contentFrame(), then assert that the
embedded route responds successfully and contains content specific to the
expected module route before relying on iframe attachment. Preserve the existing
tab.iframeSrc selector and timeout while replacing the attachment-only
validation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 40e32eef-4580-4e1e-acdf-340a825a6ff2
⛔ Files ignored due to path filters (1)
playgrounds-ecosystem/modules/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (13)
package.jsonplaygrounds-ecosystem/README.mdplaygrounds-ecosystem/REPORTS.mdplaygrounds-ecosystem/modules/components/DemoButton.vueplaygrounds-ecosystem/modules/eslint.config.mjsplaygrounds-ecosystem/modules/nuxt.config.tsplaygrounds-ecosystem/modules/package.jsonplaygrounds-ecosystem/modules/pages/index.vueplaygrounds-ecosystem/modules/pnpm-workspace.yamlplaygrounds-ecosystem/modules/server/api/widgets.get.tsplaygrounds-ecosystem/tests/ecosystem-modules.spec.tsplaygrounds-ecosystem/tests/playwright.config.tsplaygrounds-ecosystem/tests/tsconfig.json
| "@compodium/nuxt": "^0.1.0-beta.13", | ||
| "@nuxt/a11y": "^1.0.0-alpha.1", | ||
| "@nuxt/eslint": "^1.16.0", | ||
| "@nuxt/fonts": "^0.14.0", | ||
| "@nuxt/hints": "^1.1.4", | ||
| "@nuxt/scripts": "^1.3.1", | ||
| "@scalar/nuxt": "^0.6.59", | ||
| "nuxt-og-image": "^6.7.2" | ||
| }, | ||
| "devDependencies": { | ||
| "@nuxt/devtools": "link:../../packages/devtools", | ||
| "@types/node": "^26.1.1", | ||
| "nuxt": "^4.4.8", | ||
| "eslint": "^9.0.0", | ||
| "nuxt": "^4.5.0", |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
case "$(pnpm --version)" in
11.*) ;;
*) echo "pnpm 11 is required" >&2; exit 1 ;;
esac
pnpm -C playgrounds-ecosystem/modules exec node --input-type=module <<'NODE'
import fs from 'node:fs'
const pkg = JSON.parse(fs.readFileSync('package.json', 'utf8'))
const names = [
'`@compodium/nuxt`', '`@nuxt/a11y`', '`@nuxt/eslint`', '`@nuxt/fonts`',
'`@nuxt/hints`', '`@nuxt/scripts`', '`@scalar/nuxt`', 'nuxt-og-image',
'`@types/node`', 'eslint', 'nuxt',
]
const deps = { ...pkg.dependencies, ...pkg.devDependencies }
const raw = names.filter(name => deps[name] && !deps[name].startsWith('catalog:'))
if (raw.length) {
console.error(`Non-catalog dependency versions: ${raw.join(', ')}`)
process.exit(1)
}
NODE
Repository: nuxt/devtools
Length of output: 143
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "pnpm version:"
pnpm --version || true
echo
echo "Relevant files:"
git ls-files 'playgrounds-ecosystem/modules/package.json' 'playgrounds-ecosystem/modules/pnpm-workspace.yaml' 'pnpm-workspace.yaml'
echo
echo "playgrounds-ecosystem/modules/package.json dependency/devDependency section:"
cat -n playgrounds-ecosystem/modules/package.json | sed -n '1,80p'
echo
echo "playgrounds-ecosystem/modules/pnpm-workspace.yaml:"
if [ -f playgrounds-ecosystem/modules/pnpm-workspace.yaml ]; then
cat -n playgrounds-ecosystem/modules/pnpm-workspace.yaml
fi
echo
echo "Root pnpm catalog package manifests (matching names):"
python3 - <<'PY'
import json, re
pkg=json.load(open('playgrounds-ecosystem/modules/package.json'))
names=[
'`@compodium/nuxt`','`@nuxt/a11y`','`@nuxt/eslint`','`@nuxt/fonts`',
'`@nuxt/hints`','`@nuxt/scripts`','`@scalar/nuxt`','nuxt-og-image',
'`@types/node`','eslint','nuxt','nuxt'
]
for sec in ('dependencies','devDependencies'):
for name in names:
val=pkg.get(sec,{}).get(name)
print(f'{sec}.{name}: {val}')
PY
Repository: nuxt/devtools
Length of output: 3951
Use the centrally managed catalog versions.
The dependency versions in playgrounds-ecosystem/modules/package.json are already declared in the sealed playgrounds-ecosystem/modules/pnpm-workspace.yaml catalog. Replace the raw ranges with catalog:<name> references in dependencies and devDependencies.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@playgrounds-ecosystem/modules/package.json` around lines 12 - 25, Update the
dependency entries in the package manifest shown in the diff to use the
centrally managed catalog references from the workspace catalog instead of raw
version ranges. Apply this to each catalog-managed package in both dependencies
and devDependencies, preserving the existing package names and leaving
non-catalog entries unchanged.
Source: Coding guidelines
| all three tabs are still reachable (still only via the SideNav overflow | ||
| menu), and `@nuxt/scripts` still shows "No scripts loaded". | ||
|
|
||
| **Addendum 3 — Devtools-category modules added:** the playground was expanded |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Make “Addendum 3” a real Markdown heading.
The text at Line 70 is bold paragraph text, so it does not generate the #addendum-3--devtools-category-modules-added anchor used at Lines 115, 120, 227, and 322. Change it to a heading such as ## Addendum 3 — ..., or update all references to an existing anchor.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@playgrounds-ecosystem/REPORTS.md` at line 70, Convert the “Addendum 3 —
Devtools-category modules added” text in REPORTS.md from bold paragraph
formatting to a Markdown heading, preferably using the same heading hierarchy as
the surrounding addenda, so it generates the existing
`#addendum-3--devtools-category-modules-added` anchor referenced elsewhere.
Source: Linters/SAST tools
| if (tab.view === 'iframe') { | ||
| // The tab mounted its IframeView, which appends the module's dev route | ||
| // as an <iframe> into the devtools client document. | ||
| await expect(frame.locator(`iframe[src*="${tab.iframeSrc}"]`)) | ||
| .toBeAttached({ timeout: 15_000 }) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Candidate files =="
git ls-files | rg 'playgrounds-ecosystem/tests/ecosystem-modules\.spec\.ts|devtools|iframe|IframeView|docs'
echo "== Target file outline =="
ast-grep outline playgrounds-ecosystem/tests/ecosystem-modules.spec.ts || true
echo "== Target lines 1-140 =="
cat -n playgrounds-ecosystem/tests/ecosystem-modules.spec.ts | sed -n '1,140p'
echo "== Search IframeView and iframeSrc usages =="
rg -n "IframeView|iframeSrc|toBeAttached|contentFrame|\.src|/docs|docs" .
Repository: nuxt/devtools
Length of output: 43424
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== IframeView.vue =="
cat -n packages/devtools/client/components/IframeView.vue
echo "== Frames helper fixtures =="
fd -a 'devtools.ts' tests/e2e/fixtures -t f | xargs -r cat -n
echo "== Ecosystem nuxt config around scalar routeRules =="
cat -n playgrounds-ecosystem/modules/nuxt.config.ts | sed -n '45,80p'
echo "== Ecosystem module declarations/source references =="
rg -n "`@scalar/nuxt`|nuxt-hooks|devtools|ModuleCustomTab|iframeSrc|iframe" playgrounds-ecosystem/modules .github packages/playgrounds-ecosystem 2>/dev/null || true
Repository: nuxt/devtools
Length of output: 20406
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Playwright package scripts/features =="
if [ -f node_modules/playwright/package.json ]; then
cat node_modules/playwright/package.json | jq '{name, version, types}'
else
echo "node_modules/playwright not present"
fi
cat >/tmp/check_playwright_docs.js <<'JS'
try {
const docsPath = require.resolve('playwright')
.replace(/\bplaywright\.js$/, 'docs/src/codegen/cli.ts')
console.log(require.resolve('playwright'))
} catch (e) {
console.log('Playwright docs runtime resolution not available:', e.message)
}
JS
node /tmp/check_playwright_docs.js || true
echo "== Tests/e2e spec iframes for contentFrame usage =="
cat -n tests/e2e/specs/iframe.spec.ts | sed -n '1,220p'
echo "== Static extraction of iframe response checks =="
python3 - <<'PY'
from pathlib import Path
for p in [Path('playgrounds-ecosystem/tests/ecosystem-modules.spec.ts')]:
text = p.read_text()
print(p)
print("has contentFrame:", "contentFrame" in text)
print("has getResponse/expect.*ok/toBeAttached:")
for i, line in enumerate(text.splitlines(), 1):
if any(s in line for s in ["contentFrame", "getResponse", "toBeAttached", ".ok(", "error-boundary", "Nuxt DevTools error"]):
print(f"{i}:{line}")
PY
Repository: nuxt/devtools
Length of output: 1600
Assert the embedded route response before relying on iframe presence.
toBeAttached() only waits for IframeView to insert the <iframe>; it does not fail when the module route returns 404/500 or an error page. Use the iframe’s contentFrame() to assert an OK response and the route-specific content.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@playgrounds-ecosystem/tests/ecosystem-modules.spec.ts` around lines 58 - 62,
Update the iframe assertion in the tab.view === 'iframe' branch to obtain the
matching iframe’s contentFrame(), then assert that the embedded route responds
successfully and contains content specific to the expected module route before
relying on iframe attachment. Preserve the existing tab.iframeSrc selector and
timeout while replacing the attachment-only validation.
Add a dedicated devtools-smoke job to the ecosystem workflow that does a full pnpm build (real static devtools client), installs Playwright Chromium, and runs the ecosystem smoke suite — kept separate from the cheap build-only smoke job and from the default e2e workflow.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/ecosystem-playground.yml (1)
60-61: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winPin the Corepack bootstrap dependency.
Installing
corepack@latestat runtime makes this CI job non-reproducible and allows a registry release to change the package-manager bootstrap unexpectedly. Pin an approved Corepack version, or remove the installation ifpnpm/action-setup@v6already provides the required toolchain.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ecosystem-playground.yml around lines 60 - 61, Update the workflow’s Corepack setup by replacing the floating corepack@latest installation with an approved pinned version, or remove the installation when pnpm/action-setup@v6 already supplies the required Corepack toolchain; keep corepack enable only when the installation remains necessary.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/ecosystem-playground.yml:
- Around line 60-61: Update the workflow’s Corepack setup by replacing the
floating corepack@latest installation with an approved pinned version, or remove
the installation when pnpm/action-setup@v6 already supplies the required
Corepack toolchain; keep corepack enable only when the installation remains
necessary.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 2a65bc43-3e94-4536-87b1-00c40712220b
📒 Files selected for processing (2)
.github/workflows/ecosystem-playground.ymlplaygrounds-ecosystem/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
- playgrounds-ecosystem/README.md
2 participants
What & why
Extends the ecosystem dogfooding playground (
playgrounds-ecosystem/) to cover the Nuxt Devtools category modules from https://nuxt.com/modules?category=Devtools, and adds an opt-in Playwright suite that proves each one's DevTools tab still functions against this repo's own devtools client.Following the playground's existing methodology (only keep modules that expose a real DevTools tab), the curated set added is:
@nuxt/eslint— ESLint config inspector tab (launch)@nuxt/hints— performance/security/hydration hints tab@nuxt/a11y— real-time accessibility panel@compodium/nuxt— component playground tab@scalar/nuxt— API reference tabCategory modules with no DevTools surface, secret-gated ones (doppler/ngrok), and heavy/deprecated ones (storybook/workflow/sonda/eslint-module) are intentionally skipped — the full skip-list and rationale live in the playground README.
What's included
eslint.config.mjs,components/DemoButton.vue,server/api/widgets.get.ts(+nitro.experimental.openAPI), and a deliberate no-alt<img>for the a11y finding.playgrounds-ecosystem/tests/(+pnpm run test:e2e:ecosystem) that boots the combined app against the built@nuxt/devtoolsand asserts every module'scustom-*tab renders. Kept out of the default CI path, matching the sealed workspace's existing opt-in stance.README.md/REPORTS.mdupdated with per-module findings, the runbook, and upstream follow-ups.Findings worth flagging
@nuxt/devtools's@vitejs/devtools@0.4.9crashes the dev server on Vite 7 (eager@vitejs/devtools-rolldownresolve).@scalar/nuxt's/docspage 500s under SSR on this stack (threads.workerData); worked around withrouteRules: { '/docs/**': { ssr: false } }and noted as an upstream candidate.Note:
@nuxt/a11y(alpha) and@compodium/nuxt(beta) are pinned to current pre-1.0 versions; worth revisiting once they stabilize.This PR was created with the help of an agent.