Commit 7504628
feat: migrate CLI to scrapegraph-js v2 API
Full migration of just-scrape to the v2 API via scrapegraph-js PR #13
(pinned to head 096c110). The CLI surface, commands, and flags all move to
v2 semantics and a single src/commands.ts holds every endpoint.
## CLI surface
- extract, search, scrape, markdownify, crawl, monitor, history, credits
- monitor gains full lifecycle: create, list, get, update, delete, pause,
resume, and activity (paginated tick history with --limit / --cursor)
- scrape supports 8 formats (markdown, html, screenshot, branding, links,
images, summary, json) plus multi-format via comma-separated -f
- crawl uses the v2 formats array and supports multi-format
- search gains --country, --time-range, --format
- Removed from API: agentic-scraper, generate-schema, sitemap, validate
- --stealth is now a separate boolean; fetch mode is --mode auto|fast|js
## Environment
- New default base URL: https://api.scrapegraphai.com/api/v2
- SGAI_TIMEOUT (was SGAI_TIMEOUT_S); legacy SGAI_TIMEOUT_S and
JUST_SCRAPE_TIMEOUT_S are bridged transparently
- SGAI_API_KEY, SGAI_API_URL, SGAI_DEBUG honored; JUST_SCRAPE_* still bridged
## Refactor
- 8 per-endpoint files under src/commands/ merged into one src/commands.ts
with named exports; cli.ts imports them statically
- tsup output: single dist/cli.mjs (~30 KB) instead of 8 chunks
- biome override scoped to src/commands.ts
- scrapegraph-js ships without a prebuilt dist/, so CI builds it in-place
after install
Version bumped 0.3.0 → 1.0.0 to track SDK v2.0.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 2dc4981 commit 7504628
24 files changed
Lines changed: 1166 additions & 902 deletions
File tree
- .github/workflows
- skills/just-scrape
- src
- commands
- lib
- utils
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
17 | 20 | | |
18 | 21 | | |
19 | 22 | | |
| |||
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
| 29 | + | |
26 | 30 | | |
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
| 31 | + | |
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| |||
0 commit comments