Skip to content

Using to_windowed_arrays() in docs #2748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
erikvansebille merged 2 commits into from
Jul 14, 2026
Merged

Using to_windowed_arrays() in docs #2748

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
614d3ba
Using to_windowed_arrays in docs
erikvansebille Jul 14, 2026
c8a6def
Merge branch 'main' into use_windowed_arrays_in_docs
erikvansebille Jul 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/user_guide/examples/explanation_kernelloop.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ import parcels.tutorial

# Load the CopernicusMarine data in the Agulhas region from the example_datasets
ds_fields = parcels.tutorial.open_dataset("CopernicusMarine_data_for_Argo_tutorial/data")
ds_fields.load() # load the dataset into memory

# Create an idealised wind field and add it to the dataset
tdim, ydim, xdim = (len(ds_fields.time),len(ds_fields.latitude), len(ds_fields.longitude))
Expand All @@ -83,6 +82,9 @@ fieldset = parcels.FieldSet.from_sgrid_conventions(
"Wind": ("UWind", "VWind")
}
)

# Convert the FieldSet to windowed arrays for better performance
fieldset = fieldset.to_windowed_arrays()
```

Now we define a wind Kernel that uses a forward Euler method to apply the wind forcing. Note that we update the `particles.dx` and `particles.dy` variables, rather than `particles.x` and `particles.y` directly.
Expand Down
8 changes: 5 additions & 3 deletions docs/user_guide/examples/tutorial_delaystart.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,14 @@
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"# Convert to SGRID-compliant dataset and create FieldSet\n",
"fields = {\"U\": ds_fields[\"uo\"], \"V\": ds_fields[\"vo\"]}\n",
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)"
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()"
]
},
{
Expand Down Expand Up @@ -398,7 +400,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Parcels:docs (3.14.6)",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand Down
8 changes: 5 additions & 3 deletions docs/user_guide/examples/tutorial_diffusion.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,8 +472,7 @@
"# Load the CopernicusMarine data in the Agulhas region from the example_datasets\n",
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
").isel(depth=slice(0, 1))\n",
"ds_fields.load() # load the dataset into memory"
").isel(depth=slice(0, 1))"
]
},
{
Expand Down Expand Up @@ -528,7 +527,10 @@
"ds_cell_areas[\"cell_areas\"] = ([\"lat\", \"lon\"], calc_cell_areas(ds_fields))\n",
"fset2 = parcels.FieldSet.from_sgrid_conventions(ds_cell_areas)\n",
"\n",
"fieldset += fset2"
"fieldset += fset2\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()"
]
},
{
Expand Down
4 changes: 3 additions & 1 deletion docs/user_guide/examples/tutorial_dt_integrators.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"OUTPUT_FOLDER = Path(\"output\")\n",
"OUTPUT_FOLDER.mkdir(exist_ok=True)\n",
Expand All @@ -77,6 +76,9 @@
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()\n",
"\n",
"# Check field resolution in time and space\n",
"print(\n",
" f\"dt = {np.unique(np.diff(ds_fields.time.values).astype('timedelta64[h]'))} hours\"\n",
Expand Down
8 changes: 4 additions & 4 deletions docs/user_guide/examples/tutorial_gsw_density.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,6 @@
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"\n",
"# TODO check how we can get good performance without loading full dataset in memory\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"# Convert to SGRID-compliant dataset and create FieldSet\n",
"fields = {\n",
" \"U\": ds_fields[\"uo\"],\n",
Expand All @@ -54,7 +51,10 @@
" \"so\": ds_fields[\"so\"],\n",
"}\n",
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)"
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()"
]
},
{
Expand Down
4 changes: 3 additions & 1 deletion docs/user_guide/examples/tutorial_manipulating_field_data.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"# Create an idealised wind field and add it to the dataset\n",
"ydim, xdim = len(ds_fields.latitude), len(ds_fields.longitude)\n",
Expand Down Expand Up @@ -122,6 +121,9 @@
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()\n",
"\n",
"npart = 10\n",
"z = np.repeat(ds_fields.depth[0].values, npart)\n",
"lons = np.repeat(32.2, npart)\n",
Expand Down
6 changes: 4 additions & 2 deletions docs/user_guide/examples/tutorial_sampling.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,14 @@
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"# Convert to SGRID-compliant dataset and create FieldSet\n",
"fields = {\"U\": ds_fields[\"uo\"], \"V\": ds_fields[\"vo\"], \"thetao\": ds_fields[\"thetao\"]}\n",
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)"
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/examples/tutorial_write_in_kernel.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,12 +122,12 @@
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"# Convert to SGRID-compliant dataset and create FieldSet\n",
"fields = {\"U\": ds_fields[\"uo\"], \"V\": ds_fields[\"vo\"]}\n",
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"fieldset = fieldset.to_windowed_arrays()\n",
"\n",
"pset = parcels.ParticleSet(fieldset=fieldset, x=[31, 32], y=[-32, -31], z=[1, 1])\n",
"\n",
Expand Down
6 changes: 4 additions & 2 deletions docs/user_guide/getting_started/tutorial_output.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,14 @@
"ds_fields = parcels.tutorial.open_dataset(\n",
" \"CopernicusMarine_data_for_Argo_tutorial/data\"\n",
")\n",
"ds_fields.load() # load the dataset into memory\n",
"\n",
"# Convert to SGRID-compliant dataset and create FieldSet\n",
"fields = {\"U\": ds_fields[\"uo\"], \"V\": ds_fields[\"vo\"]}\n",
"ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)\n",
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)"
"fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)\n",
"\n",
"# Convert the FieldSet to windowed arrays for better performance\n",
"fieldset = fieldset.to_windowed_arrays()"
]
},
{
Expand Down
11 changes: 9 additions & 2 deletions docs/user_guide/getting_started/tutorial_quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ hydrodynamics fields in which the particles are tracked. Here we provide an exam

```{code-cell}
ds_fields = parcels.tutorial.open_dataset("CopernicusMarine_data_for_Argo_tutorial/data")

ds_fields.load() # load the dataset into memory
ds_fields
```

Expand All @@ -52,6 +50,15 @@ ds_fset = parcels.convert.copernicusmarine_to_sgrid(fields=fields)
fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)
```

Now, in order to improve performance, we can convert the `parcels.FieldSet` to windowed arrays. This is especially useful for large datasets with many timeslices, as it allows Parcels to load only the necessary timeslices into memory during the simulation.

#TODO add link to performance explanation notebook here.

```{code-cell}
# Convert the FieldSet to windowed arrays for better performance
fieldset = fieldset.to_windowed_arrays()
```

You can inspect the `parcels.FieldSet` object with the `describe` method in order to see which `parcels.Field`s are included, and which grid and interpolation method is used for each field. This also gives information on the type of mesh and the time interval of the `parcels.FieldSet`:

```{code-cell}
Expand Down
2 changes: 1 addition & 1 deletion tests/test_sigmagrids.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ def test_conversion_3DCROCO():

def test_advection_3DCROCO():
ds_fields = parcels.tutorial.open_dataset("CROCOidealized_data/data")
ds_fields.load()

fields = {
"U": ds_fields["u"],
Expand All @@ -61,6 +60,7 @@ def test_advection_3DCROCO():
ds_fset = parcels.convert.croco_to_sgrid(fields=fields, coords=ds_fields)

fieldset = parcels.FieldSet.from_sgrid_conventions(ds_fset)
fieldset = fieldset.to_windowed_arrays()
fieldset.add_context("hc", ds_fields.hc.item())

runtime = 10_000
Expand Down