Skip to content

feat: Orchestrator - Launchers - Kubernetes - Start labelling created Pods, Jobs and Services with a Tangle label #341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Ark-kun wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: Orchestrator - Launchers - Kubernetes - Start labelling created Pods, Jobs and Services with a Tangle label #341

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions cloud_pipelines_backend/launchers/kubernetes_launchers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,18 @@

# Kubernetes annotation keys. (Has strict naming policy. Single slash only etc.)
_CLOUD_PIPELINES_KUBERNETES_ANNOTATION_KEY = "cloud-pipelines.net"
_TANGLE_KUBERNETES_LABEL_KEY = "tangle.tangleml.com"

@morgan-wowk morgan-wowk Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Would you like me to refactor https://github.com/Shopify/oasis-backend/pull/458 to use this label instead of tangleml.com or refactor this PR to use tangleml.com without the subdomain?

@morgan-wowk morgan-wowk Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default namespace is already created for staging but not yet prod on e09.

@Ark-kun Ark-kun Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I do not have an opinion here.
The main reason I added the tangle. prefix was to distinguish Tangle pods from Tangent pods.
I also thought about recent pushes to make Tangent "bigger" than Tangle (although those were scaled down by senior leadership).

I'd say, just proceed with Shopify/oasis-backend#458 as-is. We are already using the tangleml.com in launchers.py. We can refactor all of that later.

_KUBERNETES_LAUNCHER_ANNOTATION_KEY = "cloud-pipelines.net/launchers.kubernetes"

_DEFAULT_KUBERNETES_ANNOTATIONS = {
_CLOUD_PIPELINES_KUBERNETES_ANNOTATION_KEY: "true",
_TANGLE_KUBERNETES_LABEL_KEY: "true",
_KUBERNETES_LAUNCHER_ANNOTATION_KEY: "true",
}
_DEFAULT_KUBERNETES_LABELS = {
_TANGLE_KUBERNETES_LABEL_KEY: "true",
}

# ComponentSpec annotation keys
RESOURCES_CPU_ANNOTATION_KEY = "cloud-pipelines.net/launchers/generic/resources.cpu"
RESOURCES_MEMORY_ANNOTATION_KEY = (
Expand Down Expand Up @@ -180,11 +191,8 @@ def __init__(
self._storage_provider = _storage_provider
self._request_timeout = request_timeout
self._pod_name_prefix = pod_name_prefix
self._pod_labels = pod_labels
self._pod_annotations = {
_CLOUD_PIPELINES_KUBERNETES_ANNOTATION_KEY: "true",
_KUBERNETES_LAUNCHER_ANNOTATION_KEY: "true",
} | (pod_annotations or {})
self._pod_labels = _DEFAULT_KUBERNETES_LABELS | (pod_labels or {})
self._pod_annotations = _DEFAULT_KUBERNETES_ANNOTATIONS | (pod_annotations or {})
self._pod_postprocessor = pod_postprocessor
self._create_volume_and_volume_mount = _create_volume_and_volume_mount

Expand Down Expand Up @@ -1173,6 +1181,8 @@ def launch_container_task(
metadata=k8s_client_lib.V1ObjectMeta(
name=explicit_service_name,
namespace=namespace,
annotations=_DEFAULT_KUBERNETES_ANNOTATIONS,
labels=_DEFAULT_KUBERNETES_LABELS,
),
spec=k8s_client_lib.V1ServiceSpec(
# "Headless" service.
Expand Down Expand Up @@ -1203,7 +1213,9 @@ def launch_container_task(
name=explicit_job_name,
namespace=namespace,
# annotations=self._pod_annotations,
annotations=_DEFAULT_KUBERNETES_ANNOTATIONS,
# labels=self._pod_labels,
labels=_DEFAULT_KUBERNETES_LABELS,
),
spec=k8s_client_lib.V1JobSpec(
template=k8s_client_lib.V1PodTemplateSpec(
Expand Down
Loading