chore(comps): annotate azl-compatibility overlays with metadata by liunan-ms · Pull Request #17887 · microsoft/azurelinux

liunan-ms · 2026-07-01T22:05:13Z

Summary

This PR migrates all azl-compatibility overlay definitions to the per-file overlay layout with file-level metadata. This is a pure restructuring/annotation change — no overlay is added, removed, reordered, or semantically altered, and the rendered specs are byte-identical to 4.0.

Complements the sibling PRs #17727, #17888. Components already migrated on those branches are skipped.

Changes

chore(comps): annotate azl-compatibility overlays with metadata
Migrates 27 components from inline overlays = [...] in their .comp.toml to per-group overlays/*.overlay.toml files (42 new overlay files).
- One overlay toml file per logical change, even for single-overlay components.
- Each file carries a file-level [metadata] block with category (and upstream-status / commit references where applicable). Multi-category components (apache-sshd, kernel-headers, mock-core-configs, rocm-compilersupport, rust, shim, unicode-ucd) get one file per logical change group, each stamped with that group's own category.
- All original .comp.toml comments (including interior comments between overlay fields) are preserved verbatim into the overlay files; the .comp.toml files are reduced to their bare [components.<name>] headers.
- Components already migrated on the azl-pruning / overlay_metadata branches are skipped.
- Categories used across the migrated files: azl-compatibility, azl-branding-policy, azl-release-management, azl-temp-workaround, azl-pruning, azl-platform-adaptation.
- Components touched: apache-sshd, dnf5, fcode-utils, iscsi-initiator-utils, kernel-headers, libspf2, libyang, llvm, llvm20, mock-core-configs, oneapi-level-zero, onnx, open-vm-tools, openblas, openexr, openpace, pcsc-lite-asekey, perl-Archive-Any-Lite, perl-Crypt-PasswdMD5, python-heatclient, python-manilaclient, python-octaviaclient, python-pyside6, rocm-compilersupport, rust, shim, unicode-ucd.
TEMP: enable overlay-files loading for PR check — ⚠️ PR-CHECK ONLY, REVERT BEFORE MERGE
Temporary. Sets overlay-files = ["overlays/*.overlay.toml"] in the [default-component-config] of base/comps/components.toml, so every component inherits per-file overlay discovery (no per-component key needed) and the new *.overlay.toml files are exercised by PR checks. Standalone so it can be reverted independently.
chore(azldev): pin to 830e6fed and regenerate schema — ⚠️ PR-CHECK ONLY, REVERT BEFORE MERGE
Temporary. Pins .azldev-version to 830e6fed (upstream-backport category, URLRef replacing BugRef, required upstream-status replacing the old upstreamable boolean, azl-dep-missing-workaround → azl-temp-workaround rename, and optional component-group [metadata]) and regenerates external/schemas/azldev.schema.json so PR checks run against a tool that understands the new overlay-files / metadata fields. Must be reverted before this PR merges.

Validation

✅ Structural no-op: all 27 components' reconstructed overlay lists are byte-identical to 4.0.
✅ azldev comp list -a parses cleanly.
✅ comp render in PR check passed.

Copilot

Pull request overview

This PR migrates all azl-compatibility (and adjacent-category) overlays for 27 components from inline overlays = [...] / [[components.<name>.overlays]] blocks in their .comp.toml files into per-group overlays/*.overlay.toml documents, each stamped with a file-level [metadata] block (category + upstreamable). It is presented as a structural no-op: overlays are neither added, removed, reordered, nor semantically changed, and rendered specs are claimed byte-identical. It also carries two clearly-labeled temporary commits needed only for PR checks.

Changes:

Split 27 components' inline overlays into 42 new overlays/*.overlay.toml files, one per logical change group, preserving comments and adding taxonomy [metadata]; multi-category components get one file per group. Local source/patch references are rewritten with a ../ prefix to resolve from the overlays/ subdir back to the component root.
Temporary (revert-before-merge): add overlay-files = ["overlays/*.overlay.toml"] to the 4.0 stage1/stage2 default-component-config so components inherit per-file overlay discovery.
Temporary (revert-before-merge): pin .azldev-version to 35d8fae and regenerate external/schemas/azldev.schema.json (adds OverlayMetadata, BugRef, overlay-files, resources schema) so PR checks run against a tool that understands the new fields.

Reviewed changes

Copilot reviewed 72 out of 72 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`external/schemas/azldev.schema.json`	Regenerated schema adding `OverlayMetadata`/`BugRef`/`overlay-files` + resources defs (temporary).
`distro/azurelinux.distro.toml`	Adds inherited `overlay-files` discovery default to both 4.0 stages (temporary; sole discovery mechanism).
`.azldev-version`	Pins azldev to `35d8fae` (temporary).
`base/comps//.comp.toml` (27 files)	Removes inline overlays, leaving component config/release settings intact.
`base/comps//overlays/.overlay.toml` (42 files)	New per-group overlay docs with `[metadata]`; `../` source paths verified against existing component-root files.

Notes from review: source/patch ../ paths resolve correctly (all referenced files exist at component root), regex conversions from TOML literal to basic strings preserve escaping, and per-file numeric prefixes preserve original overlay order. Two items flagged: the load-bearing-yet-temporary distro discovery default, and apache-sshd/overlays/0001 missing upstreamable.

tobiasb-ms

I spot-checked some things and the fact that the spec renderer ran and didn't complain -- despite no changes to specs/ -- indicates this is a functional no-op. Approved.

reubeno · 2026-07-02T16:56:57Z

I'm going through this PR and will post comments within the day. My main initial feedback is that the categorization is aggressively using azl-compatibility aggressively -- and in a way that I think will make the categorization less useful.

reubeno

I don't think we need the data to be 100% accurate to get started, but I do want us to see if we're going to get the insights we need here.

reubeno · 2026-07-02T16:47:03Z

+
+[metadata]
+category = "azl-pruning"
+upstreamable = false


Worth considering upstreamability here. Fedora will also need to move to 25 as the default at some point.

Marked upstream-status = "upstreamable"

reubeno · 2026-07-02T16:47:50Z

@@ -0,0 +1,26 @@
+[metadata]
+category = "azl-compatibility"


Why? Do we have any indication of what these changes were actually doing?

This change works around an upstream Fedora 43 packaging bug: the installed versions of maven-remote-resources-plugin, plexus-utils, and plexus-xml in the Fedora 43 repos are mutually incompatible during the Maven 3 -> 4 transition. This issue has been existing in Fedora 43 and failing the build. The Fedora 44 (commit) upgraded apache-sshd version to 2.16 and disabled maven-remote-resources-plugin for the rpm build avoid this issue. I think azl-compability is a good fit for this change with upstream-status = "upstreamed" and commits without changing the spec and rebuild, so that we can remove this overlay once pinning to f44.

reubeno · 2026-07-02T16:49:13Z

+# TODO: Remove this overlay once a new upstream release (> 1.0.3) is available.
+
+[metadata]
+category = "azl-compatibility"


I worry that azl-compatibility is being more widely used than expected. I see it used for tdnf compat symlinks, which is for compat with previous AZL versions, but this isn't really the same sort of change.

Agree, I recategorized to category = "azl-temp-workaround" (it forces -std=gnu17 to dodge the GCC 15 C23 bool-keyword conflict) with upstream-status = "upstreamable" and the upstream fix commit in commits. It comments a TODO to remove once an upstream release > 1.0.3 ships. That better reflects that it's a temporary workaround.

reubeno · 2026-07-02T16:50:40Z

@@ -0,0 +1,15 @@
+[metadata]
+category = "azl-platform-adaptation"


The azldev docs call out this category for "architecture-specific" adaptation. Is this architectural?

This only rewrites version-string macros (specversion / tarfile_release) so the spec tracks the AZL kernel's versioning. Moved from azl-platform-adaptation to category = "azl-release-management" with upstream-status = "inapplicable" (permanent AZL-only versioning). I kept azl-platform-adaptation only on 0005-build-generate-native-cross, which genuinely detects native arch and cross-generates headers.

reubeno · 2026-07-02T16:51:25Z

+# Source: use full Azure Linux kernel tarball instead of pre-built headers.
+
+[metadata]
+category = "azl-platform-adaptation"


Architectural?

Agree, this is not arch-specific. This substitutes Fedora's pre-built-headers tarball for the full AZL kernel source tree which should be a Fedora -> AZL source/identity choice. Recategorized to category = "azl-branding-policy", upstream-status = "inapplicable" (Fedora ships their own tarball and would never take this).

reubeno · 2026-07-02T16:54:10Z

@@ -0,0 +1,9 @@
+[metadata]
+category = "azl-compatibility"
+upstreamable = false


Looks upstreamable to me.

Same as above.

reubeno · 2026-07-02T16:54:38Z

+
+[metadata]
+category = "azl-compatibility"
+upstreamable = false


Actually is upstreamable. And in fact, I believe Tobias already upstreamed it.

Confirmed that the AZL4 configs are already upstream to Fedora. Set upstream-status = "upstreamed" with both commits, and a TODO: remove once pinning to f44. Changed to category = "upstream-backport" since the new mock-core-configs version is already in f44, at which point it drops entirely.

reubeno · 2026-07-02T16:58:37Z

+
+[metadata]
+category = "azl-compatibility"
+upstreamable = false


This is an example of a set of changes that aren't directly upstreamable, but which we don't want to carry forever. How would you recommend we capture that?

Set category = "azl-branding-policy" (it's driven by %_vendor = azurelinux), upstream-status = "needs-upstream-hook", indicating the current form (dropping .cfg files in three build locations) isn't directly upstreamable, but it flags that we'd like an upstream hook so we don't carry this forever.

reubeno · 2026-07-02T16:59:00Z

@@ -0,0 +1,17 @@
+[metadata]
+category = "azl-compatibility"
+upstreamable = false


Looks upstreamable.

This uses %{_target_platform} instead of the hardcoded %{_arch}-redhat-linux-gnu for the compiler-rt runtime path. Recategorized to category = "azl-branding-policy" with upstream-status = "upstreamable".

reubeno · 2026-07-02T16:59:17Z

+regex = "%global dist.*"
+replacement = ""
+
+# TEMPORARILY remove release from the dependency on the shim unsigned packages


How do we capture that this is intended as a temporary workaround?

Now captured explicitly: category = "azl-temp-workaround", upstream-status = "inapplicable", meant to be removed once the underlying release-management issue is resolved.

Copilot

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated 12 comments.

+[[overlays]]
+type = "spec-search-replace"
+regex = "%global dist.*"
+replacement = ""


+[[overlays]]
+type = "spec-search-replace"
+regex = "^BuildRequires:.*unsignedx64.*$"
+replacement = "BuildRequires: %{unsignedx64} = 15.8"


+[[overlays]]
+type = "spec-search-replace"
+regex = "^BuildRequires:.*unsignedia32.*$"
+replacement = "BuildRequires: %{unsignedia32} = 15.8"


+[[overlays]]
+type = "spec-search-replace"
+regex = "^BuildRequires:.*unsignedaa64.*$"
+replacement = "BuildRequires: %{unsignedaa64} = 15.8"


+[[overlays]]
+type = "spec-append-lines"
+section = "%prep"
+lines = ["test -e %{shimdirx64}/$(basename %{shimefix64}) && cp -vf %{shimdirx64}/$(basename %{shimefix64}) %{shimefix64} ||:", "test -e %{shimdiria32}/$(basename %{shimefiia32}) && cp -vf %{shimdiria32}/$(basename %{shimefiia32}) %{shimefiia32} ||:", "test -e %{shimdiraa64}/$(basename %{shimefiaa64}) && cp -vf %{shimdiraa64}/$(basename %{shimefiaa64}) %{shimefiaa64} ||:"]


+
+[metadata]
+category = "azl-temp-workaround"
+upstream-status = "upstreamable"


+category = "azl-branding-policy"
+upstream-status = "upstreamable"
+commits = [{ url = "https://github.com/llvm/llvm-project/pull/200894" }]


+category = "azl-branding-policy"
+upstream-status = "upstreamable"
+commits = [{ url = "https://github.com/llvm/llvm-project/pull/200894" }]


+category = "azl-branding-policy"
+upstream-status = "upstreamable"
+commits = [{ url = "https://github.com/llvm/llvm-project/pull/200894" }]


@@ -1 +1 @@
-0256227f5434d9e00d7c8501b16848efa400a72b
+830e6fed79750f8e24ae17cd630248f50c90c8a9


liunan-ms · 2026-07-10T19:15:14Z

I don't think we need the data to be 100% accurate to get started, but I do want us to see if we're going to get the insights we need here.

Thanks! This pushed me to re-audit every azl-compatibility overlay in this PR. In the latest push I moved overlays to more specific buckets: azl-branding-policy (vendor/triple/%_vendor changes), azl-release-management (version/release-string mechanics), azl-platform-adaptation (genuinely arch-specific), azl-temp-workaround (this is newly introduced in azldev for temporary workaround that is explicitly intended to be dropped once an upstream or environmental fix lands, here is the definition), and azl-pruning (dep/feature removals). I also added upstream-status (upstreamable / upstreamed / inapplicable / needs-upstream-hook) and commits refs so we can distinguish "carry forever" from "drop on next rebase." azl-compatibility is now reserved for changes that exist purely to bridge to upstream/newer-Fedora behavior we expect to inherit.

Migrate azl-compatibility components to the per-file overlay layout with file-level metadata. Each component's inline overlays -- along with their explanatory comments -- move into per-group overlays/*.overlay.toml files (one file per logical group, even for single-overlay components), and each file carries a [metadata] block with category, upstream-status, and commit references where applicable. Multi-category components get one file per group, each stamped with that group's own category. The comp.toml files are reduced to their bare [components.<name>] headers. Overlay values and apply order are preserved (validated as a structural no-op against 4.0). Components already migrated on the azl-pruning and overlay_metadata branches are skipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Sets default-component-config.overlay-files globally so the new *.overlay.toml files are exercised by PR checks. DO NOT MERGE — revert this commit before merging the PR.

Pin azldev to 830e6fed79750f8e24ae17cd630248f50c90c8a9, which builds on the upstream-backport overlay category, URLRef (replacing BugRef), and required upstream-status metadata (replacing the old upstreamable boolean). It also renames the azl-dep-missing-workaround category to azl-temp-workaround and adds an optional [metadata] block to component groups. Regenerate external/schemas/azldev.schema.json against the pinned tool so the authoritative schema matches the new overlay metadata model, including upstream-status, the renamed azl-temp-workaround category, and component-group metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 72 out of 72 changed files in this pull request and generated 8 comments.

+upstream-status = "inapplicable"
+
+[[overlays]]
+type = "spec-search-replace"


+
+# TEMPORARILY remove release from the dependency on the shim unsigned packages
+[[overlays]]
+type = "spec-search-replace"


+replacement = "BuildRequires: %{unsignedx64} = 15.8"
+
+[[overlays]]
+type = "spec-search-replace"


+replacement = "BuildRequires: %{unsignedia32} = 15.8"
+
+[[overlays]]
+type = "spec-search-replace"


+upstream-status = "inapplicable"
+
+[[overlays]]
+type = "spec-append-lines"


+upstream-status = "inapplicable"
+
+[[overlays]]
+type = "spec-append-lines"


+# Install: copy headers_install output into RPM buildroot, replacing Fedora's copy logic.
+# The exit 0 at the end short-circuits the original Fedora install logic.
+[[overlays]]
+type = "spec-prepend-lines"


@@ -1 +1 @@
-0256227f5434d9e00d7c8501b16848efa400a72b
+830e6fed79750f8e24ae17cd630248f50c90c8a9


Copilot AI review requested due to automatic review settings July 1, 2026 22:05

Copilot started reviewing on behalf of liunan-ms July 1, 2026 22:05 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread distro/azurelinux.distro.toml Outdated

Comment thread base/comps/apache-sshd/overlays/0001-remove-maven-remote-resources.overlay.toml

liunan-ms mentioned this pull request Jul 1, 2026

chore(comps): annotate overlays in multiple categories with metadata #17888

Open

liunan-ms marked this pull request as ready for review July 2, 2026 03:04

liunan-ms requested a review from a team as a code owner July 2, 2026 03:04

tobiasb-ms approved these changes Jul 2, 2026

View reviewed changes

reubeno requested changes Jul 2, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 10, 2026 19:01

liunan-ms force-pushed the liunan/azl-compatibility branch from 68938da to f32ec72 Compare July 10, 2026 19:01

Copilot started reviewing on behalf of liunan-ms July 10, 2026 19:02 View session

Copilot AI reviewed Jul 10, 2026

View reviewed changes

liunan-ms and others added 3 commits July 10, 2026 19:55

TEMP: enable overlay-files loading for PR check

3181cf4

Sets default-component-config.overlay-files globally so the new *.overlay.toml files are exercised by PR checks. DO NOT MERGE — revert this commit before merging the PR.

Copilot AI review requested due to automatic review settings July 10, 2026 20:01

liunan-ms force-pushed the liunan/azl-compatibility branch from f32ec72 to cc2980f Compare July 10, 2026 20:01

Copilot started reviewing on behalf of liunan-ms July 10, 2026 20:02 View session

Copilot AI reviewed Jul 10, 2026

View reviewed changes

		@@ -0,0 +1,15 @@
		[metadata]
		category = "azl-platform-adaptation"

		@@ -1 +1 @@
		0256227f5434d9e00d7c8501b16848efa400a72b
		830e6fed79750f8e24ae17cd630248f50c90c8a9

Uh oh!

Conversation

liunan-ms commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

tobiasb-ms left a comment

Choose a reason for hiding this comment

Uh oh!

reubeno commented Jul 2, 2026

Uh oh!

reubeno left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liunan-ms Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

liunan-ms commented Jul 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

liunan-ms commented Jul 1, 2026 •

edited

Loading

liunan-ms Jul 9, 2026 •

edited

Loading