feat(component): add overlay metadata and 'component metadata' command by liunan-ms · Pull Request #242 · microsoft/azure-linux-dev-tools

liunan-ms · 2026-06-16T00:12:02Z

Summary
Overlays currently carry no record of why they exist or when they can be removed, so reviewers can't tell a temporary Fedora backport from a permanent AZL branding change. This PR adds an optional [metadata] block to overlays, a per-file .overlay.toml format for grouping related overlays, and a read-only azldev component metadata command for inspecting them.

What's new

Overlay metadata block
Overlays can now declare optional documentation metadata:

Field	Purpose
category	Required. Classifies intent (backport-fedora, upstream-fix, azl-branding-policy, …).
commits	Upstream commit URLs this overlay backports/references.
pr	Upstream pull-request URL.
bug	Issue-tracker URLs.
upstreamability	yes / no / unknown (defaults to unknown).

Per-file .overlay.toml format (overlay-dir)

A component can set overlay-dir = "overlays" to load *.overlay.toml files, one per logical change. Each file has a single file-level [metadata] block (stamped onto every overlay in the file) plus an ordered list of [[overlays]]. Files load in filename order (0001-, 0002- convention), are appended after inline overlays, and resolve source paths relative to the overlay file. Per-overlay metadata inside these files is rejected.

Optional component-group metadata

Component groups can now carry an optional [metadata] block that reuses the same schema as overlay metadata (category, commits, pr, bug, upstreamability). It's documentation-only: it doesn't affect how the group or its components are built and is excluded from fingerprints. The azldev component metadata command surfaces it alongside overlay metadata, tagged with source = group.

azldev component metadata command

A read-only command listing overlays and their metadata across components, with --category, --only-annotated, --upstreamability and --overlays / --groups filters. Table (default) or JSON output. Available as an MCP tool. Does not parse spec files or fetch sources, so it's fast and works with missing/stale locks.

Validated against real overlays (microsoft/azurelinux#17727)

# 1. Config loads and the overlays merge + validate (metadata rules enforced)
azldev component metadata -a --overlays

# 2. Inspect a single annotated component
azldev component metadata -p cpio --overlays

# 3. Exercise the filters
azldev component metadata -a --overlays --only-annotated
azldev component metadata -a --overlays --category backport-dist-git
azldev component metadata -a --overlays --upstreamability yes

# 4. metadata for component groups
azldev comp metadata -p gnutls --groups     # groups only

# 5. metadata declares category = "azl-test-disablement" from both sources 
azldev comp metadata -a --category azl-test-disablement

Add an optional [metadata] block on overlays (category, commits, PR, bug links, upstreamability) and a per-file .overlay.toml format loaded from a component's overlay-dir. Add the read-only 'azldev component overlays' command with --category, --only-annotated, and --upstreamability filters. Metadata and overlay-dir are excluded from fingerprints. Regenerate docs, JSON schema, and scenario snapshots.

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds structured overlay metadata and per-file overlay support, plus a new CLI/MCP command to list overlays across components.

Changes:

Extend project config schema with overlay-dir for components and metadata for overlays.
Implement .overlay.toml overlay directory loading (file-level metadata stamped onto overlays) and validation.
Add azldev component overlays command (and docs/snapshots) to inspect overlays and filter by metadata.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
schemas/azldev.schema.json	Adds `overlay-dir` and `OverlayMetadata` schema definitions.
scenario/snapshots/TestSnapshots_config_generate-schema_stdout_1.snap	Updates schema-generation snapshot to include new fields/defs.
scenario/snapshots/TestSnapshotsContainer_config_generate-schema_stdout_1.snap	Updates container schema-generation snapshot to include new fields/defs.
scenario/snapshots/TestMCPServerMode_1.snap.json	Adds new MCP tool entry for `component-overlays`.
internal/projectconfig/overlay_metadata.go	Introduces metadata types/enums and validation rules.
internal/projectconfig/overlay_metadata_test.go	Adds validation tests for overlay metadata and overlay validation integration.
internal/projectconfig/overlay_file.go	Implements overlay-dir scanning + `.overlay.toml` parsing/stamping behavior.
internal/projectconfig/overlay_file_test.go	Adds tests for overlay-dir load/stamping, ordering, path resolution, and errors.
internal/projectconfig/overlay.go	Adds optional `Metadata` to overlays and validates it when present.
internal/projectconfig/loader.go	Hooks overlay-dir resolution into config load pipeline (pre-validation).
internal/projectconfig/fingerprint_test.go	Marks `ComponentOverlay.Metadata` and `ComponentConfig.OverlayDir` as excluded from fingerprinting.
internal/projectconfig/component.go	Adds `OverlayDir` field and preserves verbatim value in `WithAbsolutePaths`.
internal/app/azldev/cmds/component/overlays.go	Adds `azldev component overlays` command + filtering/sorting logic.
internal/app/azldev/cmds/component/overlays_test.go	Adds tests for listing and filtering overlays by annotation/category/upstreamability.
internal/app/azldev/cmds/component/component.go	Registers the new `overlays` subcommand.
docs/user/reference/config/overlays.md	Documents overlay metadata and per-file `.overlay.toml` overlay-dir format.
docs/user/reference/cli/azldev_component_overlays.md	Adds auto-generated CLI reference for the new command.
docs/user/reference/cli/azldev_component.md	Adds the new subcommand to the CLI index.

dmcilvaney · 2026-06-16T19:37:17Z

 	}

+	if c.Metadata != nil {
+		if err := c.Metadata.Validate(); err != nil {


Need to handle permissive mode.

- Replace commas with semicolons in Upstreamability jsonschema description to prevent jsonschema tag parser from truncating the description. - Replace deep.MustCopy with a non-panicking OverlayMetadata.clone() so config load can never panic when stamping per-file metadata. - Reuse a package-level validator instance in OverlayMetadata.Validate to avoid per-call allocations. - Compute normalized --upstreamability filter once in ListOverlays and pass it to overlayInfoMatchesFilters instead of re-normalizing per overlay. - Fail loudly when an overlay-dir matches no *.overlay.toml files (or the directory is missing) via WithFailOnPatternNotExist and a new ErrOverlayDirNoFiles sentinel. - Reject overlay-dir on default-component-config (project, group, and distro-version layers) with ErrOverlayDirInDefaultConfig as a stopgap until overlay-dir is plumbed through the default-merge path. - Regenerate JSON schema and scenario snapshots.

Per-file overlay metadata validation now warns and continues under --permissive-config instead of failing the load, aligning with the behavior introduced in microsoft#216. This lets older .overlay.toml files that use fields or values deprecated by a newer Metadata.Validate() still be loaded when permissive parsing is requested.

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Add an optional [component-groups.<name>.metadata] block reusing the existing OverlayMetadata schema (required category plus commits, pr, bug, and upstreamability). Metadata is documentation only and does not affect how group members are resolved or built. Validation is wired into ConfigFile.Validate, with tests and reference docs updated and the JSON schema regenerated.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

…adata Replace 'azldev component overlays' with 'azldev component metadata', which lists documentation metadata from both a component's overlays and the component groups it explicitly belongs to. Each row is tagged with its source ('overlay' or 'group'). By default both sources are listed; --overlays or --groups restricts output to one source. The existing --category, --only-annotated, and --upstreamability filters apply across selected sources. The command is read-only and skips lock validation. Regenerate CLI reference docs and update the hand-written overlays config doc to reference the new command.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Swap the tri-state OverlayUpstreamability enum (yes/no/unknown) for a *bool Upstreamable field on OverlayMetadata. A nil pointer (omitted in TOML) means upstreamability has not yet been assessed; true/false are the two assessed outcomes. The CLI flag is renamed --upstreamability -> --upstreamable and accepts 'true', 'false', or 'unknown'. 'unknown' matches both annotated entries that omit the field and entries with no metadata at all. Regenerates JSON schema, CLI reference docs, and scenario snapshots.

Copilot AI review requested due to automatic review settings June 16, 2026 00:12

liunan-ms mentioned this pull request Jun 16, 2026

chore(comps): annotate backport overlays with category metadata microsoft/azurelinux#17727

Draft

Copilot AI reviewed Jun 16, 2026

View reviewed changes

Comment thread internal/projectconfig/overlay_metadata.go Outdated

Comment thread internal/projectconfig/overlay_file.go Outdated

Comment thread internal/projectconfig/overlay_metadata.go Outdated

Comment thread internal/app/azldev/cmds/component/overlays.go Outdated

dmcilvaney reviewed Jun 16, 2026

View reviewed changes

liunan-ms added 2 commits June 17, 2026 00:20

Copilot AI review requested due to automatic review settings June 17, 2026 20:58

Copilot started reviewing on behalf of liunan-ms June 17, 2026 21:00 View session

Merge branch 'main' into liunan/overlay_metadata

ac2242f

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Comment thread internal/projectconfig/overlay_file.go

dmcilvaney approved these changes Jun 17, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings June 19, 2026 21:24

Copilot started reviewing on behalf of liunan-ms June 19, 2026 21:24 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

Comment thread scenario/__snapshots__/TestMCPServerMode_1.snap.json Outdated

Comment thread scenario/__snapshots__/TestMCPServerMode_1.snap.json Outdated

Comment thread internal/app/azldev/cmds/component/metadata.go

Comment thread internal/projectconfig/overlay_metadata.go

liunan-ms changed the title ~~feat(component): add overlay metadata and 'component overlays' command~~ feat(component): add overlay metadata and 'component metadata' command Jun 19, 2026

liunan-ms force-pushed the liunan/overlay_metadata branch from ed53a4d to b7c3712 Compare June 19, 2026 21:52

Copilot AI review requested due to automatic review settings June 19, 2026 22:10

liunan-ms force-pushed the liunan/overlay_metadata branch from b7c3712 to d1ce778 Compare June 19, 2026 22:10

Copilot started reviewing on behalf of liunan-ms June 19, 2026 22:11 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

Comment thread internal/projectconfig/overlay_metadata.go

liunan-ms mentioned this pull request Jun 24, 2026

feat(overlays): add overlay metadata and per-file overlay format (1/3) #249

Merged

Uh oh!

Conversation

liunan-ms commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dmcilvaney Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

liunan-ms commented Jun 16, 2026 •

edited

Loading