Skip to content

Commit d06063c

Browse files
VinciGit00claude
andcommitted
docs(cli): add monitor command + sync with just-scrape v1.0.0
Brings the CLI docs in line with the CLI changes in ScrapeGraphAI/just-scrape#14 (which pulls in scrapegraph-js v2 PR #13 head 096c110): - Document the full `just-scrape monitor` action set, including the new `monitor activity --id <id> [--limit] [--cursor]` for paginated tick history - Replace stale `-m direct+stealth` / `-m js+stealth` with real CLI syntax (`-m js --stealth`, fetch modes: auto/fast/js) - Env vars: `SGAI_TIMEOUT_S` → `SGAI_TIMEOUT`, default base URL now `https://api.scrapegraphai.com/api/v2`, document `SGAI_DEBUG` - Credits example uses `.remaining` (v2 response shape) - Add `schema` to the history services list - Fix `--location-geo-code` → `--country` in search example - Add monitor usage examples (webhook, activity, jq filter for changes) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 7833365 commit d06063c

3 files changed

Lines changed: 63 additions & 14 deletions

File tree

services/cli/commands.mdx

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@ Extract structured data from any URL using AI (replaces `smart-scraper`). [Full
1111
just-scrape extract <url> -p <prompt>
1212
just-scrape extract <url> -p <prompt> --schema <json>
1313
just-scrape extract <url> -p <prompt> --scrolls <n> # infinite scroll (0-100)
14-
just-scrape extract <url> -p <prompt> --mode js+stealth # anti-bot bypass
14+
just-scrape extract <url> -p <prompt> --stealth # anti-bot bypass
15+
just-scrape extract <url> -p <prompt> --mode reader # HTML mode: normal (default), reader, prune
1516
just-scrape extract <url> -p <prompt> --cookies <json> --headers <json>
1617
just-scrape extract <url> -p <prompt> --country <iso> # geo-targeting
1718
```
@@ -37,7 +38,7 @@ Convert any webpage to clean markdown (convenience wrapper for `scrape --format
3738

3839
```bash
3940
just-scrape markdownify <url>
40-
just-scrape markdownify <url> -m direct+stealth # anti-bot bypass
41+
just-scrape markdownify <url> -m js --stealth # anti-bot bypass
4142
just-scrape markdownify <url> --headers <json>
4243
```
4344

@@ -58,7 +59,7 @@ just-scrape scrape <url> -f json -p <prompt> --schema <json> # JSON with enforc
5859
just-scrape scrape <url> -f markdown,links,images # multi-format (comma-separated)
5960
just-scrape scrape <url> --html-mode reader # normal (default), reader, or prune
6061
just-scrape scrape <url> --scrolls <n> # infinite scroll (0-100)
61-
just-scrape scrape <url> -m direct+stealth # anti-bot bypass
62+
just-scrape scrape <url> -m js --stealth # anti-bot bypass (fetch mode: auto, fast, js)
6263
just-scrape scrape <url> --country <iso> # geo-targeting
6364
```
6465

@@ -73,7 +74,28 @@ just-scrape crawl <url> --max-depth <n> # crawl depth (default
7374
just-scrape crawl <url> --max-links-per-page <n> # max links per page (default 10)
7475
just-scrape crawl <url> --allow-external # allow external domains
7576
just-scrape crawl <url> -f html # page format (default markdown)
76-
just-scrape crawl <url> -m direct+stealth # anti-bot bypass
77+
just-scrape crawl <url> -f markdown,links,images # multi-format (comma-separated)
78+
just-scrape crawl <url> -m js --stealth # anti-bot bypass
79+
```
80+
81+
## monitor
82+
83+
Create and manage page-change monitors. A monitor periodically re-scrapes a URL and tracks diffs between ticks. [Full docs →](/api-reference/monitor)
84+
85+
```bash
86+
just-scrape monitor create --url <url> --interval <interval> # e.g. '1h', '30m', '1d'
87+
just-scrape monitor create --url <url> --interval 1h --name "My Monitor"
88+
just-scrape monitor create --url <url> --interval 30m --webhook-url <url>
89+
just-scrape monitor create --url <url> --interval 1d -f markdown,screenshot
90+
just-scrape monitor list # list all monitors
91+
just-scrape monitor get --id <id> # get monitor details
92+
just-scrape monitor update --id <id> --interval 2h # update interval
93+
just-scrape monitor pause --id <id> # pause
94+
just-scrape monitor resume --id <id> # resume
95+
just-scrape monitor delete --id <id> # delete
96+
just-scrape monitor activity --id <id> # paginated tick history
97+
just-scrape monitor activity --id <id> --limit 50 # ticks per page (max 100)
98+
just-scrape monitor activity --id <id> --cursor <cursor> # paginate with a cursor
7799
```
78100

79101
## history
@@ -87,15 +109,15 @@ just-scrape history <service> --page-size <n> # results per page (max
87109
just-scrape history <service> --json
88110
```
89111

90-
Services: `scrape`, `extract`, `search`, `monitor`, `crawl`
112+
Services: `scrape`, `extract`, `schema`, `search`, `monitor`, `crawl`
91113

92114
## credits
93115

94116
Check your credit balance.
95117

96118
```bash
97119
just-scrape credits
98-
just-scrape credits --json | jq '.remainingCredits'
120+
just-scrape credits --json | jq '.remaining'
99121
```
100122

101123
## Global flags

services/cli/examples.mdx

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ just-scrape extract https://news.example.com \
1919
# Anti-bot bypass for JS-heavy SPAs
2020
just-scrape extract https://app.example.com/dashboard \
2121
-p "Extract user stats" \
22-
--mode js+stealth
22+
--stealth
2323
```
2424

2525
## search
@@ -31,7 +31,7 @@ just-scrape search "What are the best Python web frameworks in 2025?" \
3131

3232
# Recent news only, scoped to Germany
3333
just-scrape search "EU AI act latest news" \
34-
--time-range past_week --location-geo-code de
34+
--time-range past_week --country de
3535

3636
# Structured output with schema
3737
just-scrape search "Top 5 cloud providers pricing" \
@@ -53,7 +53,7 @@ just-scrape markdownify https://docs.example.com/api \
5353
--json | jq -r '.markdown' > api-docs.md
5454

5555
# Bypass Cloudflare
56-
just-scrape markdownify https://protected.example.com -m js+stealth
56+
just-scrape markdownify https://protected.example.com -m js --stealth
5757
```
5858

5959
## scrape
@@ -80,7 +80,7 @@ just-scrape scrape https://store.example.com \
8080

8181
# Geo-targeted + anti-bot bypass
8282
just-scrape scrape https://store.example.com \
83-
-m direct+stealth --country DE
83+
-m js --stealth --country DE
8484
```
8585

8686
## crawl
@@ -99,7 +99,33 @@ just-scrape crawl https://example.com \
9999
--max-pages 50 --allow-external
100100

101101
# Anti-bot bypass for protected sites
102-
just-scrape crawl https://example.com -m direct+stealth
102+
just-scrape crawl https://example.com -m js --stealth
103+
```
104+
105+
## monitor
106+
107+
```bash
108+
# Monitor a pricing page every hour
109+
just-scrape monitor create --url https://store.example.com/pricing --interval 1h
110+
111+
# Daily monitor tracking markdown + screenshots, with webhook
112+
just-scrape monitor create --url https://example.com \
113+
--interval 1d -f markdown,screenshot \
114+
--webhook-url https://hooks.example.com/notify \
115+
--name "Daily check"
116+
117+
# List, pause, resume, delete
118+
just-scrape monitor list
119+
just-scrape monitor pause --id abc123
120+
just-scrape monitor resume --id abc123
121+
just-scrape monitor delete --id abc123
122+
123+
# Browse the tick history (runs the monitor has already performed)
124+
just-scrape monitor activity --id abc123 --limit 20
125+
126+
# Only show ticks where a change was detected
127+
just-scrape monitor activity --id abc123 --json \
128+
| jq '.ticks[] | select(.hasChanges == true)'
103129
```
104130

105131
## history

services/cli/introduction.mdx

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,10 +56,11 @@ The CLI needs a ScrapeGraph API key. Four ways to provide it (checked in order):
5656
| Variable | Description | Default |
5757
|---|---|---|
5858
| `SGAI_API_KEY` | ScrapeGraph API key ||
59-
| `SGAI_API_URL` | Override API base URL | `https://api.scrapegraphai.com` |
60-
| `SGAI_TIMEOUT_S` | Request timeout in seconds | `30` |
59+
| `SGAI_API_URL` | Override API base URL | `https://api.scrapegraphai.com/api/v2` |
60+
| `SGAI_TIMEOUT` | Request timeout in seconds | `120` |
61+
| `SGAI_DEBUG` | Set to `1` to log requests/responses ||
6162

62-
Legacy variables (`JUST_SCRAPE_API_URL`, `JUST_SCRAPE_TIMEOUT_S`, `JUST_SCRAPE_DEBUG`) are still bridged.
63+
Legacy variables are still bridged transparently: `JUST_SCRAPE_API_URL``SGAI_API_URL`, `JUST_SCRAPE_TIMEOUT_S` and `SGAI_TIMEOUT_S``SGAI_TIMEOUT`, `JUST_SCRAPE_DEBUG``SGAI_DEBUG`.
6364

6465
## Verify your setup
6566

0 commit comments

Comments
 (0)