Skip to content

Commit 480452b

Browse files
committed
docs: document store capability for large query results
1 parent 3aa6fed commit 480452b

1 file changed

Lines changed: 40 additions & 0 deletions

File tree

README.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,46 @@ It also implements the `close()` method, as suggested by the PEP-2049
5959
specification, to support situations where the cursor is wrapped in a
6060
`contextmanager.closing()`.
6161

62+
### Storing results in cloud storage
63+
64+
For large query results, you can store them directly in cloud storage
65+
instead of retrieving them over the connection. This is useful when
66+
results are too large to transfer efficiently, or when you want to
67+
process them later with other tools.
68+
69+
```python
70+
from wherobots.db import connect, Store, StorageFormat
71+
from wherobots.db.region import Region
72+
from wherobots.db.runtime import Runtime
73+
74+
with connect(
75+
api_key='...',
76+
runtime=Runtime.TINY,
77+
region=Region.AWS_US_WEST_2) as conn:
78+
curr = conn.cursor()
79+
80+
# Store results with a presigned URL for easy download
81+
curr.execute(
82+
"SELECT * FROM wherobots_open_data.overture.places LIMIT 1000",
83+
store=Store.for_download()
84+
)
85+
store_result = curr.get_store_result()
86+
print(f"Results stored at: {store_result.result_uri}")
87+
print(f"Size: {store_result.size} bytes")
88+
```
89+
90+
The `Store` class supports the following options:
91+
92+
* `format`: output format - `StorageFormat.PARQUET` (default),
93+
`StorageFormat.CSV`, or `StorageFormat.GEOJSON`
94+
* `single`: if `True`, write results to a single file instead of
95+
multiple partitioned files (default: `True`)
96+
* `generate_presigned_url`: if `True`, generate a presigned URL for
97+
downloading results (default: `False`)
98+
99+
Use `Store.for_download()` as a convenient shorthand for storing results
100+
as a single Parquet file with a presigned URL.
101+
62102
### Runtime and region selection
63103

64104
You can chose the Wherobots runtime you want to use using the `runtime`

0 commit comments

Comments
 (0)