Skip to content

Add firecrawl-web-data skill - #6005

Closed
JuampiHernandez wants to merge 1 commit into
github:mainfrom
JuampiHernandez:add-firecrawl-web-data-skill
Closed

JuampiHernandez wants to merge 1 commit into
github:mainfrom
JuampiHernandez:add-firecrawl-web-data-skill

Conversation

@JuampiHernandez

Copy link
Copy Markdown

Hey, I'm Juampi from the Firecrawl team.

Pull Request Checklist

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have read and followed the Guidance for submissions involving paid services.
  • My contribution adds a new instruction, prompt, agent, skill, workflow, or canvas extension file in the correct directory.
  • The file follows the required naming convention.
  • The content is clearly structured and follows the example format.
  • I have tested my instructions, prompt, agent, skill, workflow, or canvas extension with GitHub Copilot.
  • I have run npm start and verified that README.md is up to date.
  • I am targeting the main branch for this pull request.

Description

Adds skills/firecrawl-web-data, a skill for coding tasks that depend on documentation Copilot can't read with its built-in fetch tool: reference pages that render with JavaScript, specifications published as PDFs, and large docs sites where the right page has to be found first. The skill has Copilot check the project's library version, retrieve only the pages it needs as Markdown files under .firecrawl/, compare them with the existing code, make the change, run the tests, and cite the sources.

I work at Firecrawl, and the skill says so. Firecrawl is a hosted service and is also open source, so the skill covers pointing the CLI at a self-hosted deployment through FIRECRAWL_API_URL. It keeps the built-in fetch tool as the default whenever that tool already returns the content, and it states what needs an API key: map and crawl do; scrape and parse run without one within a daily per-IP limit, except from VPN or cloud IP addresses. It also tells Copilot to treat retrieved pages as untrusted and to ask before uploading a local file with parse. Links are limited to the CLI docs, the CLI source, and the self-hosting docs. There are no scripts and no MCP config, only SKILL.md.

Changes

  • skills/firecrawl-web-data/SKILL.md: when to use it, prerequisites, a documentation-to-code workflow, a command table, tested examples with the output to expect, guidelines on Firecrawl credits and uploads, and limitations.
  • docs/README.skills.md: regenerated with npm start (one new row).

How I tested it

Every example in the skill ran live with firecrawl-cli 1.26.24, with an API key:

  • map https://docs.github.com/en/copilot --search skills: 270 URLs, including the agent skills concept page, which scrape then saved as Markdown.
  • scrape https://quotes.toscrape.com/js/: the 10 rendered quotes, one per line.
  • scrape https://arxiv.org/pdf/1706.03762: 45 KB of Markdown starting with # Attention Is All You Need. parse on the downloaded PDF starts with the same heading.
  • crawl https://quotes.toscrape.com --limit 3 --max-depth 1 --wait: status: completed with 3 pages in data.
  • search "GitHub Copilot agent skills documentation" --limit 3 --json: 3 results under data.web, led by the GitHub Docs agent skills page.
  • Without a key, from a home connection: scrape (the JS page and the PDF) and parse succeeded; map asked for a key. From a cloud IP, keyless requests are refused, which the skill states.

npm run skill:validate passes (449 skills), and the local pr-risk-scan reports nothing for the new file. I haven't run it inside Copilot yet, so I left that box unticked.


Type of Contribution

  • New instruction file.
  • New prompt file.
  • New agent file.
  • New plugin.
  • New skill file.
  • New agentic workflow.
  • New canvas extension.
  • Update to existing instruction, prompt, agent, plugin, skill, workflow, or canvas extension.
  • Other (please specify):

Additional Notes

Happy to rename the skill or cut sections if you'd prefer something smaller.


By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

@JuampiHernandez
JuampiHernandez requested review from a team as code owners October 31, 2026 01:19
@github-actions github-actions Bot added new-submission PR adds at least one new contribution skills PR touches skills labels Oct 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔒 PR Risk Scan Results

Scanned 1 changed file(s).

Severity Count
🔴 High 0
🟠 Medium 0
ℹ️ Info 0

✅ No matching risk patterns were detected in changed files.

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

🚦 Submission status: 👀 Ready for review

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 1 approval from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier
  • Label needs-review:HIGH flags a high contributor-risk signal

Automated checks

Check Status Details
Line endings ✅ Passed Passed · logs
Spelling ✅ Passed Passed · logs
Generated README consistency ✅ Passed Passed · logs
Skill validation ✅ Passed Passed · logs
Skill lint (vally) ✅ Passed Passed · logs
Risk scan ✅ Passed Passed · logs
Contributor reputation ✅ Passed Passed · logs
Duplicate resource scan ✅ Passed Passed · logs
PR quality signal ⏭️ Skipped Skipped by its workflow · logs

Review

  • Approvals: 0/1
  • Assigned reviewer: not assigned yet — comment /request-review to ask for one
  • Review target date: not set
  • Still needed: 1 more approval(s); an approval from a maintainer with admin or maintain permission
  • The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command Who What it does
/rerun-checks PR author, maintainers Re-runs failed or incomplete checks and re-evaluates this gate
/request-review PR author, maintainers Asks the review rotation to assign a reviewer (adds needs-reviewer)

Updated for c6ac329 · gate run · This comment is maintained automatically — see submission gate docs.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Contributor Reputation Check: HIGH risk

Check Risk
Profile HIGH
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot added the needs-review:HIGH Contributor reputation check flagged HIGH risk label Oct 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Vally Lint Results

✅ All checks passed

Scope Checked
Skills 1
Agents 0
Total 1
Severity Count
❌ Errors 0
⚠️ Warnings 0
ℹ️ Advisories 0

Summary

Level Finding
ℹ️ ✅ firecrawl-web-data (2/2 checks passed)
ℹ️ ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
ℹ️ ✓ spec-compliance: All spec checks passed.
ℹ️ ✓ [valid-refs] All file references across 1 skill(s) are valid.
ℹ️ ✓ valid-refs: All file references resolve to existing files within the skill directory.
ℹ️ 1 skill(s) linted, 1 passed
Full linter output
### Linting skills/firecrawl-web-data
✅ firecrawl-web-data (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

@github-actions github-actions Bot added merge-risk:high ready-for-review Submission passed intake validation and is ready for maintainer review and removed merge-risk:medium awaiting-automation labels Oct 31, 2026
@aaronpowell aaronpowell closed this Nov 2, 2026
@aaronpowell

Copy link
Copy Markdown
Contributor

Thanks for disclosing your Firecrawl affiliation and documenting the examples you tested, but ultimately we are not a marketing platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk:high needs-review:HIGH Contributor reputation check flagged HIGH risk new-submission PR adds at least one new contribution ready-for-review Submission passed intake validation and is ready for maintainer review skills PR touches skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants