Skip to content

fix(kernels): clear stale image pins for TPU pushes - #1219

Open
sean0407 wants to merge 4 commits into
Kaggle:mainfrom
sean0407:codex/fix-tpu-image-pins-1197
Open

sean0407 wants to merge 4 commits into
Kaggle:mainfrom
sean0407:codex/fix-tpu-image-pins-1197

Conversation

@sean0407

Copy link
Copy Markdown

Summary

TPU push
  enable_tpu = true
  clear docker_image from kernel metadata
  clear dockerImageVersionId from notebook metadata
  preserve the remaining notebook metadata

Resolves the stale CPU/GPU image pin reported in #1197.

Evidence

  • Before: kernels_push forwarded docker_image and notebook metadata.kaggle.dockerImageVersionId even for TPU requests, matching the issue reproducer and maintainer-confirmed root cause.
  • After: the isolated unit test confirms the request omits both stale image selectors and keeps other notebook metadata; the source notebook remains unchanged.
  • python tests/unit/test_kernels_push.py: 18 tests passed in an isolated, credential-free environment.

Merge Danger

Door: two-way

Blast Radius: TPU pushes

Only TPU requests drop the CPU/GPU image pins; non-TPU request handling is unchanged.

@sridipbasu sridipbasu added the bug Something isn't working label Oct 16, 2026
@stevemessick

Copy link
Copy Markdown
Contributor

/gcbrun

@stevemessick

Copy link
Copy Markdown
Contributor

@sean0407 Thanks for the PR! The formatting needs minor adjustment. We just use black.

I didn't see any other problems, but will look more closely when the PR is ready.

@stevemessick

Copy link
Copy Markdown
Contributor

Thanks again for working on this, @sean0407! After taking a closer look, here are a few items to address before merging:

  1. Formatting (black)
    The Cloud Build lint step failed on black --check in src/kaggle/api/kaggle_api_extended.py. Since pyproject.toml sets line-length = 120, black formats the request.docker_image assignment on a single line:

    request.docker_image = None if enable_tpu else self.get_or_default(meta_data, "docker_image", None)
  2. TPU requested via --accelerator (acc) or machine_shape without "enable_tpu": true
    Currently this only checks enable_tpu = self.get_bool(meta_data, "enable_tpu", False). However, users can also request a TPU via kaggle kernels push --accelerator TpuV5E8 (acc="TpuV5E8") or by setting "machine_shape": "TpuV5E8" in kernel-metadata.json (which is what docs/kernels_metadata.md documents), and acc overrides the boolean GPU/TPU settings in kernel-metadata.json.
    Resolving machine_shape = acc if acc else self.get_or_default(meta_data, "machine_shape", None) up front and treating TPU machine_shape values (e.g., TpuV5E8, Tpu1VmV38, TpuV38) as TPU pushes—unless acc explicitly overrides to a non-TPU shape—will cover --accelerator TpuV5E8 and machine_shape-only configs as well.

  3. Guard against "metadata": null in notebook JSON
    json_body.get("metadata", {}).get("kaggle", {}) will raise AttributeError if "metadata": null is present in the .ipynb file. Checking isinstance on metadata first avoids that:

    metadata = json_body.get("metadata")
    kaggle_metadata = metadata.get("kaggle") if isinstance(metadata, dict) else None
    if isinstance(kaggle_metadata, dict):
        kaggle_metadata.pop("dockerImageVersionId", None)
  4. CHANGELOG.md conflict and formatting
    Please rebase or merge main to resolve the conflict in CHANGELOG.md (from the 2.2.5 release), and remove the extra blank lines around the bullet and the trailing period to match the rest of the file.

@sean0407

Copy link
Copy Markdown
Author

Thanks for the detailed review. I updated kernels_push() to derive TPU status from the effective accelerator or machine_shape; an explicit --accelerator now overrides the metadata booleans. I also guarded notebook metadata: null, applied Black formatting, merged current main, and kept the changelog entry with the surrounding release entries.

Verification in an isolated Python 3.13 container: pytest -q tests/unit passed (1,424 tests, 38 subtests); Black check passed for both changed Python files; mypy passed for src/kaggle and tests (78 files).

The PR is updated at head 8179bdc6b45707887fdad632f7e3c42c7db603ec. Could you review this follow-up when convenient?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants