feat: add llms markdown in doc pages#891
Conversation
4d75c03 to
5f883db
Compare
…Markdown button Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
9a729fd to
7ed125a
Compare
There was a problem hiding this comment.
Pull request overview
Adds first-class “LLM-friendly” outputs for the docs site by generating per-page .md twins plus llms.txt / llms-full.txt, and exposes the per-page Markdown via UI actions under each doc title.
Changes:
- Add and configure
docusaurus-plugin-llmsto generatellms.txt,llms-full.txt, and per-page Markdown outputs at build time. - Add a
MarkdownPageActionsUI control to “Copy as Markdown” / “View as Markdown” for eligible doc pages. - Wire the new control into the DocItem theme and update lockfile for new dependencies.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Locks new transitive dependencies required by docusaurus-plugin-llms. |
| package.json | Adds docusaurus-plugin-llms dependency. |
| docusaurus.config.js | Configures the plugin to generate llms*.txt and Markdown per doc page. |
| src/theme/DocItem/index.js | Computes .md twin URL and renders Markdown actions under the doc title. |
| src/components/MarkdownPageActions.js | Implements “Copy as Markdown” / “View as Markdown” UI and clipboard logic. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…leak Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
…tripping Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
manasmanohar
left a comment
There was a problem hiding this comment.
Please address the comments below. Specifics left inline; this one isn't tied to a diff line:
Collides with the llms.txt we already ship.
We already have curated static/llms.txt and static/llms-full.txt (content team's — About Keploy, awards, positioning, curated links). The plugin generates the same two filenames, and its postBuild runs after static/ is copied, so it overwrites them: build/llms.txt is the generated version and the curated content is gone. Net effect — the content team's file is silently replaced, and editing static/llms.txt later would have no effect. The static file also drifts today since it's hand-maintained.
Option: merge both — feed the curated text into the plugin via rootContent/fullRootContent (+ title/description) so it stays on top and the auto-generated index below stays fresh. Needs the curated source moved out of static/ (so it isn't overwritten) and the static copies removed. Worth confirming the exact split with the content team.
| + // Prop values can legitimately contain '<' or '>' inside quoted strings (e.g. | ||
| + // tools={["Linux kernel >= 5.10"]}), which would otherwise be mistaken for the |
There was a problem hiding this comment.
These two regexes run over the whole document including fenced code, so real code gets stripped. e.g. installation/windows.md: wsl --install -d <Distribution Name> → wsl --install -d (~5 files); quickstart/java-spring-boot-xml.md: the <User>...</User> wrapper is removed and the XML sample ends up malformed. This is new — those tags were preserved before (the plugin only stripped a fixed HTML allowlist).
Option: skip fenced code blocks when stripping (mask fences → strip prose → restore).
| "@tailwindcss/typography": "^0.5.0", | ||
| "autoprefixer": "^10.4.0", | ||
| "docusaurus": "^1.14.7", | ||
| "docusaurus-plugin-llms": "^0.4.0", |
There was a problem hiding this comment.
Plugin is ^0.4.0 but the patch is pinned to 0.4.0 — patch-package will fail the install if a 0.4.1 publishes. Consider pinning the exact version.
| [ | ||
| "docusaurus-plugin-llms", | ||
| { | ||
| docsDir: "versioned_docs/version-4.0.0", |
There was a problem hiding this comment.
docsDir is hardcoded to versioned_docs/version-4.0.0 — will need updating when we cut the next docs version, since the buttons show on whatever the latest version is (gated on isLatestVersion).
| ? "Copied" | ||
| : copyState === "error" | ||
| ? "Copy failed" | ||
| : "Copy as Markdown"; |
There was a problem hiding this comment.
add fixed width to prevent the div from shrinking when changing text
Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
8595693 to
986babe
Compare
…button width shift Signed-off-by: Shreya Sharma <shreyasharma.1510001@gmail.com>
What has changed?
Every docs page is now also available as plain Markdown, and
llms.txt/llms-full.txtare auto-generated at build time — so AI agents (and humans) can grab clean per-page content instead of parsing HTML.docusaurus-plugin-llms, configured to generatellms.txt,llms-full.txt, and a.mdtwin for every doc page (same URL,.mdextension)..mdfile is actually discoverable, not just guessable. Styled with Tailwind utility classes to match this repo's existing convention.docusaurus-plugin-llms(viapatch-package, auto-reapplied on install) to fix two real content bugs found during review:import WhatIsKeploy from '../concepts/what-is-keploy.md'); the plugin only resolved these when the filename had an underscore, so ~7 pages were leaking that raw import/JSX line into their.mdoutput instead of the real text.<Tabs>,<ProductTier>,<HowTo>, etc.) into their.mdoutput. Fixed by stripping self-closing component tags entirely and unwrapping paired ones while preserving their real inner content — carefully excluding<YOUR_INGRESS_HOST>-style fill-in-your-own-value placeholders, which look similar but aren't components.setTimeoutresetting its label was never cleared, so it could fire after the component unmounted (React warning) or race itself on repeated clicks.Fixes https://github.com/keploy/enterprise/issues/2272
How Has This Been Tested?
Ran
npm run buildthennpm run serve— confirmed the build succeeds,llms.txt/llms-full.txtgenerate correctly, each page's.mdtwin loads at its matching URL, and the Copy/View buttons work in both light and dark mode.Preview.Video.mov