chore(airflow): Bump to 3.3.1

[adwk67]

Description

Part of #1645

N.B.

• most of the changes in this PR are due to updating the uv lock files
• bump to node 26.9.0 not done possible until Nexus limits allow uploads again as it brings a new dependency with it - libatomic - which would need to be added to the image. Nothing requries node 26 so have left this at version 24.19.0.
• bump to uv-version 0.12.15 not possible until action has been fixed:

jq: error (at sbom.json:64): Cannot iterate over null (null)
Possible fix:

-        jq '.components |= map(
  +        jq '.components |= ((. // []) | map(
                  if .properties then
                    ...
  -              | if .properties == [] then del(.properties) else . end)' \
  +              | if .properties == [] then del(.properties) else . end))' \
              sbom.json > sbom.slim.json

Important

Previously the opa-auth-manager was tested against one hard-coded version of airflow and the FAB provider, set in pyproject.toml. This was true for both in-image testing and local testing. For the airflow 3 tree that pin had drifted off every version we actually ship, so the tests were passing against an API the images no longer used. Now the specific version of airflow is tracked in the image and the auth-manager is tested against THAT version, as well as components that are specified in the constraints list and elsewhere (such as python). This means we are actually testing what we bake into the image. The builder stage runs once per image, so this happens for EVERY supported version - 2.9.3, 3.0.6, 3.2.2 and 3.3.1 - each against its own airflow, FAB provider and python. Running the tests by hand is the exception: there is no constraints file involved there, so the versions have to be named explicitly. That is what the local dependency group is for, and it is pinned to the latest supported version.

OPA tests failed due to mismatch between FAB provider and the opa-auth-manager. Fixed as part of this PR:

--- PASS: kuttl (710.00s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/opa_airflow-3.2.2_opa-latest-1.16.2_openshift-false (256.60s)
        --- PASS: kuttl/harness/opa_airflow-2.9.3_opa-latest-1.16.2_openshift-false (262.80s)
        --- PASS: kuttl/harness/opa-interop_airflow-latest-3.3.1_opa-latest-1.16.2_openshift-false (139.12s)
        --- PASS: kuttl/harness/opa_airflow-3.1.6_opa-latest-1.16.2_openshift-false (290.97s)
        --- PASS: kuttl/harness/opa_airflow-3.3.1_opa-latest-1.16.2_openshift-false (206.88s)
        --- PASS: kuttl/harness/opa_airflow-3.0.6_opa-latest-1.16.2_openshift-false (156.22s)
PASS

Nightly tests tested with stackabletech/airflow-operator#865:

--- PASS: kuttl (4389.48s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-3.3.1_openshift-false_executor-celery (266.78s)
        --- PASS: kuttl/harness/ca-cert_airflow-latest-3.3.1_openshift-false (296.60s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.3.1_ldap-authentication-server-verification-tls_openshift-false_executor-kubernetes (244.06s)
        --- PASS: kuttl/harness/remote-logging_airflow-latest-3.3.1_openshift-false_executor-kubernetes (353.92s)
        --- PASS: kuttl/harness/remote-logging_airflow-latest-3.3.1_openshift-false_executor-celery (248.43s)
        --- PASS: kuttl/harness/opa_airflow-3.3.1_opa-latest-1.16.2_openshift-false (229.79s)
        --- PASS: kuttl/harness/smoke_airflow-3.3.1_openshift-false_executor-kubernetes (185.52s)
        --- PASS: kuttl/harness/smoke_airflow-3.3.1_openshift-false_executor-celery (233.50s)
        --- PASS: kuttl/harness/oidc_airflow-latest-3.3.1_openshift-false (375.37s)
        --- PASS: kuttl/harness/versioning_airflow-latest-3.3.1_openshift-false (693.77s)
        --- PASS: kuttl/harness/triggerer_airflow-latest-3.3.1_openshift-false_executor-kubernetes (300.04s)
        --- PASS: kuttl/harness/triggerer_airflow-latest-3.3.1_openshift-false_executor-celery (210.85s)
        --- PASS: kuttl/harness/overrides_airflow-latest-3.3.1_openshift-false (185.54s)
        --- PASS: kuttl/harness/cluster-operation_airflow-latest-3.3.1_openshift-false (456.26s)
        --- PASS: kuttl/harness/orphaned-resources_airflow-latest-3.3.1_openshift-false (332.83s)
        --- PASS: kuttl/harness/external-access_airflow-3.3.1_openshift-false_executor-kubernetes (149.84s)
        --- PASS: kuttl/harness/external-access_airflow-3.3.1_openshift-false_executor-celery (172.54s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.3.1_openshift-false_executor-kubernetes_access-ssh (268.43s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.3.1_openshift-false_executor-kubernetes_access-https (269.07s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.3.1_openshift-false_executor-celery_access-ssh (330.65s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.3.1_openshift-false_executor-celery_access-https (223.79s)
        --- PASS: kuttl/harness/opa-interop_airflow-latest-3.3.1_opa-latest-1.16.2_openshift-false (141.39s)
        --- PASS: kuttl/harness/logging_airflow-3.3.1_openshift-false_executor-celery (393.13s)
        --- PASS: kuttl/harness/logging_airflow-3.3.1_openshift-false_executor-kubernetes (707.55s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.3.1_ldap-authentication-insecure-tls_openshift-false_executor-kubernetes (230.14s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.3.1_ldap-authentication-no-tls_openshift-false_executor-kubernetes (203.24s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.3.1_ldap-authentication-server-verification-tls_openshift-false_executor-celery (231.53s)
        --- PASS: kuttl/harness/resources_airflow-latest-3.3.1_openshift-false (168.21s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.3.1_ldap-authentication-no-tls_openshift-false_executor-celery (243.93s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.3.1_ldap-authentication-insecure-tls_openshift-false_executor-celery (232.98s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-3.3.1_openshift-false_executor-kubernetes (183.23s)
PASS

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

• Changes are OpenShift compatible
• All added packages (via microdnf or otherwise) have a comment on why they are added
• Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
• All packages should have (if available) signatures/hashes verified
• Add an entry to the CHANGELOG.md file
• Integration tests ran successfully

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

[dervoeti]

Just a few minor things