Skip to content

Commit 33ea3e6

Browse files
feat: add ScrapeGraphAI client pattern, remove generateSchema
- Add ScrapeGraphAI({ apiKey? }) factory that reads SGAI_API_KEY from env - Rename client methods: getCredits → credits, checkHealth → healthy - Remove generateSchema (no longer in API) - Update all examples to use new client pattern - Update README with client usage Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 3703f8c commit 33ea3e6

24 files changed

+144
-210
lines changed

README.md

Lines changed: 31 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
1-
# ScrapeGraph JS SDK
1+
# ScrapeGraphAI JS SDK
22

33
[![npm version](https://badge.fury.io/js/scrapegraph-js.svg)](https://badge.fury.io/js/scrapegraph-js)
44
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
55

66
<p align="center">
7-
<img src="media/banner.png" alt="ScrapeGraph JS SDK" style="width: 100%;">
7+
<img src="media/banner.png" alt="ScrapeGraphAI JS SDK" style="width: 100%;">
88
</p>
99

10-
Official TypeScript SDK for the [ScrapeGraph AI API](https://scrapegraphai.com).
10+
Official TypeScript SDK for the [ScrapeGraphAI AI API](https://scrapegraphai.com).
1111

1212
## Install
1313

@@ -20,9 +20,12 @@ bun add scrapegraph-js
2020
## Quick Start
2121

2222
```ts
23-
import { scrape } from "scrapegraph-js";
23+
import { ScrapeGraphAI } from "scrapegraph-js";
2424

25-
const result = await scrape("your-api-key", {
25+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
26+
const sgai = ScrapeGraphAI();
27+
28+
const result = await sgai.scrape({
2629
url: "https://example.com",
2730
formats: [{ type: "markdown" }],
2831
});
@@ -52,7 +55,7 @@ type ApiResult<T> = {
5255
Scrape a webpage in multiple formats (markdown, html, screenshot, json, etc).
5356

5457
```ts
55-
const res = await scrape("key", {
58+
const res = await sgai.scrape({
5659
url: "https://example.com",
5760
formats: [
5861
{ type: "markdown", mode: "reader" },
@@ -88,7 +91,7 @@ const res = await scrape("key", {
8891
Extract structured data from a URL, HTML, or markdown using AI.
8992

9093
```ts
91-
const res = await extract("key", {
94+
const res = await sgai.extract({
9295
url: "https://example.com",
9396
prompt: "Extract product names and prices",
9497
schema: { /* JSON schema */ }, // optional
@@ -103,7 +106,7 @@ const res = await extract("key", {
103106
Search the web and optionally extract structured data.
104107

105108
```ts
106-
const res = await search("key", {
109+
const res = await sgai.search({
107110
query: "best programming languages 2024",
108111
numResults: 5, // 1-20, default 3
109112
format: "markdown", // "markdown" | "html"
@@ -115,24 +118,13 @@ const res = await search("key", {
115118
});
116119
```
117120

118-
### generateSchema
119-
120-
Generate a JSON schema from a natural language description.
121-
122-
```ts
123-
const res = await generateSchema("key", {
124-
prompt: "Schema for a product with name, price, and rating",
125-
existingSchema: { /* ... */ }, // optional, to modify
126-
});
127-
```
128-
129121
### crawl
130122

131123
Crawl a website and its linked pages.
132124

133125
```ts
134126
// Start a crawl
135-
const start = await crawl.start("key", {
127+
const start = await sgai.crawl.start({
136128
url: "https://example.com",
137129
formats: [{ type: "markdown" }],
138130
maxPages: 50,
@@ -144,12 +136,12 @@ const start = await crawl.start("key", {
144136
});
145137

146138
// Check status
147-
const status = await crawl.get("key", start.data?.id!);
139+
const status = await sgai.crawl.get(start.data?.id!);
148140

149-
// Control crawl by ID
150-
await crawl.stop("key", start.data?.id!);
151-
await crawl.resume("key", start.data?.id!);
152-
await crawl.delete("key", start.data?.id!);
141+
// Control
142+
await sgai.crawl.stop(id);
143+
await sgai.crawl.resume(id);
144+
await sgai.crawl.delete(id);
153145
```
154146

155147
### monitor
@@ -158,7 +150,7 @@ Monitor a webpage for changes on a schedule.
158150

159151
```ts
160152
// Create a monitor
161-
const mon = await monitor.create("key", {
153+
const mon = await sgai.monitor.create({
162154
url: "https://example.com",
163155
name: "Price Monitor",
164156
interval: "0 * * * *", // cron expression
@@ -168,35 +160,35 @@ const mon = await monitor.create("key", {
168160
});
169161

170162
// Manage monitors
171-
await monitor.list("key");
172-
await monitor.get("key", cronId);
173-
await monitor.update("key", cronId, { interval: "0 */6 * * *" });
174-
await monitor.pause("key", cronId);
175-
await monitor.resume("key", cronId);
176-
await monitor.delete("key", cronId);
163+
await sgai.monitor.list();
164+
await sgai.monitor.get(cronId);
165+
await sgai.monitor.update(cronId, { interval: "0 */6 * * *" });
166+
await sgai.monitor.pause(cronId);
167+
await sgai.monitor.resume(cronId);
168+
await sgai.monitor.delete(cronId);
177169
```
178170

179171
### history
180172

181173
Fetch request history.
182174

183175
```ts
184-
const list = await history.list("key", {
176+
const list = await sgai.history.list({
185177
service: "scrape", // optional filter
186178
page: 1,
187179
limit: 20,
188180
});
189181

190-
const entry = await history.get("key", "request-id");
182+
const entry = await sgai.history.get("request-id");
191183
```
192184

193-
### getCredits / checkHealth
185+
### credits / healthy
194186

195187
```ts
196-
const credits = await getCredits("key");
188+
const credits = await sgai.credits();
197189
// { remaining: 1000, used: 500, plan: "pro", jobs: { crawl: {...}, monitor: {...} } }
198190

199-
const health = await checkHealth("key");
191+
const health = await sgai.healthy();
200192
// { status: "ok", uptime: 12345 }
201193
```
202194

@@ -217,8 +209,6 @@ const health = await checkHealth("key");
217209
| crawl | [`crawl_with_formats.ts`](examples/crawl/crawl_with_formats.ts) | Crawl with screenshots and patterns |
218210
| monitor | [`monitor_basic.ts`](examples/monitor/monitor_basic.ts) | Create a page monitor |
219211
| monitor | [`monitor_with_webhook.ts`](examples/monitor/monitor_with_webhook.ts) | Monitor with webhook notifications |
220-
| schema | [`generate_schema_basic.ts`](examples/schema/generate_schema_basic.ts) | Generate JSON schema from prompt |
221-
| schema | [`modify_existing_schema.ts`](examples/schema/modify_existing_schema.ts) | Modify an existing schema |
222212
| utilities | [`credits.ts`](examples/utilities/credits.ts) | Check account credits and limits |
223213
| utilities | [`health.ts`](examples/utilities/health.ts) | API health check |
224214
| utilities | [`history.ts`](examples/utilities/history.ts) | Request history |
@@ -227,7 +217,7 @@ const health = await checkHealth("key");
227217

228218
| Variable | Description | Default |
229219
|----------|-------------|---------|
230-
| `SGAI_API_KEY` | Your ScrapeGraph API key ||
220+
| `SGAI_API_KEY` | Your ScrapeGraphAI API key ||
231221
| `SGAI_API_URL` | Override API base URL | `https://api.scrapegraphai.com/v2` |
232222
| `SGAI_DEBUG` | Enable debug logging (`"1"`) | off |
233223
| `SGAI_TIMEOUT_S` | Request timeout in seconds | `120` |
@@ -244,4 +234,4 @@ bun run check # tsc --noEmit + biome
244234

245235
## License
246236

247-
MIT - [ScrapeGraph AI](https://scrapegraphai.com)
237+
MIT - [ScrapeGraphAI AI](https://scrapegraphai.com)

examples/crawl/crawl_basic.ts

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { crawl } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const startRes = await crawl.start(apiKey, {
6+
const startRes = await sgai.crawl.start({
67
url: "https://example.com",
78
maxPages: 5,
89
maxDepth: 2,
@@ -14,7 +15,7 @@ if (startRes.status !== "success" || !startRes.data) {
1415
console.log("Crawl started:", startRes.data.id);
1516
console.log("Status:", startRes.data.status);
1617

17-
const getRes = await crawl.get(apiKey, startRes.data.id);
18+
const getRes = await sgai.crawl.get(startRes.data.id);
1819
console.log("\nProgress:", getRes.data?.finished, "/", getRes.data?.total);
1920
console.log("Pages:", getRes.data?.pages.map((p) => p.url));
2021
}

examples/crawl/crawl_with_formats.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { crawl } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const res = await crawl.start(apiKey, {
6+
const res = await sgai.crawl.start({
67
url: "https://example.com",
78
formats: [
89
{ type: "markdown", mode: "reader" },

examples/extract/extract_basic.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { extract } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const res = await extract(apiKey, {
6+
const res = await sgai.extract({
67
url: "https://example.com",
78
prompt: "What is this page about? Extract the main heading and description.",
89
});

examples/extract/extract_with_schema.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { extract } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const res = await extract(apiKey, {
6+
const res = await sgai.extract({
67
url: "https://example.com",
78
prompt: "Extract the page title and description",
89
schema: {

examples/monitor/monitor_basic.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { monitor } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const res = await monitor.create(apiKey, {
6+
const res = await sgai.monitor.create({
67
url: "https://example.com",
78
name: "Example Monitor",
89
interval: "0 * * * *",

examples/monitor/monitor_with_webhook.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { monitor } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const res = await monitor.create(apiKey, {
6+
const res = await sgai.monitor.create({
67
url: "https://example.com/prices",
78
name: "Price Monitor",
89
interval: "0 */6 * * *",

examples/schema/generate_schema_basic.ts

Lines changed: 0 additions & 15 deletions
This file was deleted.

examples/schema/modify_existing_schema.ts

Lines changed: 0 additions & 34 deletions
This file was deleted.

examples/scrape/scrape_basic.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import { scrape } from "scrapegraph-js";
1+
import { ScrapeGraphAI } from "scrapegraph-js";
22

3-
const apiKey = process.env.SGAI_API_KEY!;
3+
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
4+
const sgai = ScrapeGraphAI();
45

5-
const res = await scrape(apiKey, {
6+
const res = await sgai.scrape({
67
url: "https://example.com",
78
formats: [{ type: "markdown" }],
89
});

0 commit comments

Comments
 (0)