fix: Stabilize integration cluster tests

[aviruthen]

What's changing and why?

This change stabilizes the HyperPod CLI integration tests and adds compatibility with Kubernetes Python client 37.

The failures had two independent causes:

1. The recipe integration test depended on the long-lived default/fsx-claim cluster fixture. That claim’s backing FSx volume could no longer mount after cluster software upgrades, leaving the recipe pod in Pending/ContainerCreating. The test initially reported this indirectly as get-logs and exec failures.
2. Kubernetes client 37 replaced several legacy generated-model behaviors with Pydantic models. This exposed use of the private _metadata pagination field, obsolete model constructor arguments, and invalid unit-test fixture values.

Recipe integration test

The existing recipe test now:

• Creates a uniquely named static PV/PVC for each run using the filesystem provided through FSX_ID.
• Uses DescribeFileSystems to retrieve the filesystem’s DNS name, Lustre mount name, and capacity instead of hard-coding environment-specific values.
• Verifies that the downloaded recipe template still defaults to claimName: fsx-claim, then replaces only that temporary template reference with the run-specific PVC.
• Uses unique output, result, and resume paths to support concurrent test runs.
• Waits for the recipe pod to become Kubernetes Ready before testing logs and exec.
• Retrieves the pod name through Kubernetes JSON output rather than parsing human-formatted CLI table output.
• Deletes only the exact job, pod, ConfigMap, PVC, and PV created by the test.
• Uses Retain and never creates, modifies, or deletes the underlying FSx filesystem.

This preserves the original init → configure → validate → create → logs/exec → delete workflow while removing dependence on mutable cluster-level PV/PVC fixtures.

The CI execution role requires the read-only fsx:DescribeFileSystems permission to resolve the filesystem metadata associated with FSX_ID.

Kubernetes client compatibility

The list-cluster implementation now:

• Uses a dedicated Kubernetes ApiClient for each temporary cluster kubeconfig instead of changing process-global Kubernetes configuration from parallel workers.
• Keeps node, pod, namespace, and ClusterQueue reads bound to the same cluster-specific client.
• Preserves the existing KubernetesClient singleton behavior for normal callers.
• Preserves the existing public ListPods.list_pods_and_get_requested_resources_group_by_node_name() signature by using private client-injection helpers internally.
• Reads pagination through the supported public response.metadata property instead of the removed private response._metadata field.

Without the pagination change, Kubernetes 37 raised:

'V1NodeList' object has no attribute '_metadata'

[papriwal]

LGTM!