Skip to content

2026 qartod - #356

Draft
Klankers wants to merge 16 commits into
ioos:mainfrom
Klankers:2026_qartod
Draft

Klankers wants to merge 16 commits into
ioos:mainfrom
Klankers:2026_qartod

Conversation

@Klankers

Copy link
Copy Markdown

Adding an updated version of the QARTOD quality control, focusing on visualizing the different QC results.

In progress - will implement some fixed by EoD.

  • Density inversion test
  • Spell check
  • Pull helper functions out
  • Split in two parts - it's a little dense

@Klankers

Copy link
Copy Markdown
Author

Okay, broke the notebook up into 4 parts:

  • Time and syntax (the more abstract things to open with, plus the more general overview)
  • Position, which is the more processing-intense option
  • Gross range and its relatives
  • Pressure and density inversion

Each says that it's a part of a series of 4 notebooks, including what is covered before/after it if applicable.

I basically copied + pasted the lead into each one, including printing the ioos_qc version for context for whomever runs this next.

And I pulled all the functions out into 2026-10-22-QARTOD-help-plot.py so the notebooks can focus more on the figures and dialogue themselves.

I confirmed everything still ran as intended before deleting the original notebook.

I consulted Claude a couple of times when pulling the plotting functions out into the .py notebook, as I started running into import errors. It suggested to add a couple of other helpers to guarantee things still worked without being in a notebook, which makes sense, so _landmask and _range_spans were entirely made by Claude to guarantee it works.

@Klankers

Copy link
Copy Markdown
Author

Let me know what you think! Feedback would be great regarding:

  • Prose (I've rushed this project this week)
  • Suggestions for figure tweaks
  • Questions about how anything works, which could be better clarified in the notebooks
  • Suggestions regarding other refactor steps or standards to check on

@callumrollo

callumrollo commented Oct 23, 2026 •

Copy link
Copy Markdown

This is a big PR! Here's some suggestions in the four areas requested:

Prose

  • notebook 1 print(dir(QartodFlags)) prints a bunch of uneeded attributes
  • notebook 4 "In 99% of cases" [citation needed] "in the vast majority" perhaps?

As a general note, with these notebooks try to write shorter, simpler sentences with less informal language that might be hard to parse, especially for people whose first language is not english. For example in notebook 1 I woud rewrite

QARTOD commonly flags by cascading flags onto one another by severity. I.e., everything is untested (2) until it is passed into a test, where it is considered good. If anything goes wrong, this good flag is overwritten by a missing flag (9, typically handled with a numpy mask), a suspect flag (3), or straight up failing (4).

as

QARTOD uses a cascading flagging system, where lesser flags like suspect (3) and overwritten by more severe flags like failing (4). All points are initially set as untested (2), until passed into a test where they are flagged as good (1). If anything goes wrong, this good flag is overwritten by a missing flag (9), a suspect flag (3), or a fail flag (4).

Figure tweaks

  • notebook 1 - adjust Denmark label so that it is within the landmask. Consider a better contrast (paler land?) to the black country labels that does not overlap with the time colormap
  • notebook 2 - overlapping arrows on location test plot
  • notebook 2 - shifted 0.9 degrees north arrow doesn't quite point to data it refers to
  • notebook 3 - gross_range_figure_val could use a ylabel for the variable, maybe a title too?
  • notebook 4 - pressure_test figure would be a bit more explicit if pad=0.01 was pad=0.01 dbar imo. At first glance I assumed 0.01 meant pad of 1 %
  • notebook 4 - some captions label as "Downcast" or "upcast" some figures simply use "up". Consistently using "Upcast" probably better

Clarifications

Now that it's been split into 4 notebooks, it could be good to include that in the introductory cell

  1. Timing/Gap Test notebook 1
  2. Syntax Test ntoebook 1
    etc.

Refactor

imo no need to repeat the long cell "IOOS QARTOD on Glider Data". Just keep it in notebook 1. Later notebooks should just have the last para "This notebook is continued from part 2" etc. with a sentence stating that the background/overview is in notebook 1

data loading with pooch can be moved to the helper script imo, these notebooks are for showcasing ioos_qc, not use of pooch, and interested people can always look in the helper script for these functions. Also if the dataset location/loading changes later you only need to update one script not 4 notebooks!

small functions like flag_counts similarly can move to the helper script. As a rule of thumb, if a function in the notebook isn't calling funcionality from ioos_qc, try to move it to the helper script. It makes it easier for the reader to focus on the relevant ioos_qc functionality that you're showcasing

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants