Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/user/reference/config/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ A component definition tells azldev where to find the spec file, how to customiz
| Spec source | `spec` | [SpecSource](#spec-source) | No | Where to find the spec file for this component. Inherited from distro defaults if not specified. |
| Release config | `release` | [ReleaseConfig](#release-configuration) | No | Controls how the Release tag is managed during rendering |
| Overlays | `overlays` | array of [Overlay](overlays.md) | No | Modifications to apply to the spec and/or source files |
| Overlay files | `overlay-files` | array of string | No | Glob patterns that load per-file overlay documents. See [Per-file overlay format](overlays.md#per-file-overlay-format). |
| Overlay files | `overlay-files` | array of string | No | Path or glob patterns that load per-file overlay documents after component config resolution. Inherited relative patterns are resolved from the concrete component's config file, or from the matched spec file's directory for spec-discovered components. Use `[]` to disable inherited patterns. See [Per-file overlay format](overlays.md#per-file-overlay-format). |
| Build config | `build` | [BuildConfig](#build-configuration) | No | Build-time options (macros, conditionals, check config) |
| Render config | `render` | [RenderConfig](#render-configuration) | No | Options controlling spec rendering behavior |
| Source files | `source-files` | array of [SourceFileReference](#source-file-references) | No | Additional source files to download for this component |
Expand Down
6 changes: 4 additions & 2 deletions docs/user/reference/config/overlays.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ In addition to per-overlay fields, the following fields are set directly on the

| Field | TOML Key | Description |
|-------|----------|-------------|
| Overlay files | `overlay-files` | List of glob patterns (relative to the component's config file) matched against the filesystem at load time to locate per-file overlay documents. Patterns support `**` (globstar). Matches are concatenated in declaration order; within a single pattern, matches are applied in filename (lexicographic) order, with full path as a tie-breaker for duplicate filenames. Each pattern must match at least one file. Duplicate matches across patterns are de-duplicated. See [Per-file overlay format](#per-file-overlay-format). |
| Overlay files | `overlay-files` | List of path or glob patterns matched against the filesystem after component config resolution to locate per-file overlay documents. Relative patterns are resolved from the concrete component's config file, or from the matched spec file's directory for spec-discovered components. Patterns support `**` (globstar). Matches are concatenated in declaration order; within a single pattern, matches are applied in filename (lexicographic) order, with full path as a tie-breaker for duplicate filenames. Glob patterns that match no files are ignored; literal paths must match a file. Duplicate matches across patterns are de-duplicated. See [Per-file overlay format](#per-file-overlay-format). |

## Overlay Metadata

Expand Down Expand Up @@ -163,6 +163,8 @@ When a single logical change (a CVE backport, a feature disablement, a Fedora ch

Set `overlay-files` on the component to one or more globs (relative to the component config) and drop one overlay file per logical change into a directory of your choosing. The conventional layout uses a sibling `overlays/` directory and a `*.overlay.toml` filename suffix, but neither is required — `overlay-files` is just a glob, so any layout you can describe with `**`/`*` patterns works.

`overlay-files` can also be inherited from `default-component-config` at the project, distro, or component-group level. Inherited relative patterns are still resolved for each concrete component: from its component config file when it has one, or from the matched spec file's directory when it is discovered by a component group's `specs` pattern. This makes defaults useful for component-local discovery patterns such as `overlay-files = ["overlays/*.overlay.toml"]`. If a component sets `overlay-files`, that value replaces the inherited list; use `overlay-files = []` to disable inherited overlay files for a component, or include both patterns explicitly when you want to keep default discovery and add component-specific locations.

```
base/comps/mypackage/
├── mypackage.comp.toml
Expand All @@ -178,7 +180,7 @@ base/comps/mypackage/
overlay-files = ["overlays/*.overlay.toml"]
```

Files are loaded in **filename (lexicographic) order** within each glob, using the full path as a tie-breaker when multiple matches have the same filename. Globs are concatenated in declaration order, so prefix each file with a numeric ordinal (`0001-`, `0002-`, …) to make the apply order obvious and stable. Files that don't match any of your globs are ignored, so you can keep `README.md` or other notes alongside without naming them out explicitly. Each declared glob must match at least one file; an empty match is treated as a misconfiguration and surfaced as an error.
Files are loaded in **filename (lexicographic) order** within each glob, using the full path as a tie-breaker when multiple matches have the same filename. Globs are concatenated in declaration order, so prefix each file with a numeric ordinal (`0001-`, `0002-`, …) to make the apply order obvious and stable. Files that don't match any of your globs are ignored, so you can keep `README.md` or other notes alongside without naming them out explicitly. A declared glob that matches no files contributes no overlays; a literal path without wildcard characters must match a file.

Overlays loaded via `overlay-files` are **appended after** any inline overlays declared directly on the component.

Expand Down
107 changes: 99 additions & 8 deletions internal/app/azldev/core/components/resolver.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"log/slog"
"path"
"path/filepath"
"slices"
"strings"

"github.com/bmatcuk/doublestar/v4"
Expand Down Expand Up @@ -129,7 +130,9 @@ func (r *Resolver) FindAllComponents() (components *ComponentSet, err error) {
var component Component

// Resolve the config for this group member.
component, err = r.getComponentFromNameAndSpecPath(groupMember.ComponentName, groupMember.SpecPath)
component, err = r.getComponentFromNameAndSpecPath(
groupMember.ComponentName, groupMember.SpecPath, []string{groupName},
)
if err != nil {
return components, fmt.Errorf("failed to enumerate components in group '%s':\n%w", groupName, err)
}
Expand All @@ -148,7 +151,9 @@ func (r *Resolver) FindAllComponents() (components *ComponentSet, err error) {
var updatedComponentConfig *projectconfig.ComponentConfig

// Apply defaults from the loaded distro config...
updatedComponentConfig, err = applyInheritedDefaultsToComponent(r.env, componentConfig)
updatedComponentConfig, err = applyInheritedDefaultsToComponent(
r.env, componentConfig, overlayFilesReferenceDir(componentConfig, ""), nil,
)
if err != nil {
return components, err
}
Expand Down Expand Up @@ -343,6 +348,48 @@ func componentGroupExcludesSpec(
return false, nil
}

func componentGroupMatchesSpecPath(
group *projectconfig.ComponentGroupConfig, specPath string,
) (matches bool, err error) {
for _, pattern := range group.SpecPathPatterns {
matched, err := doublestar.PathMatch(pattern, specPath)
if err != nil {
return false, fmt.Errorf(
"failed to compare %#q against spec pattern %#q:\n%w", specPath, pattern, err)
}

if !matched {
continue
}

excludes, err := componentGroupExcludesSpec(group, specPath)
if err != nil {
return false, err
}

return !excludes, nil
}

return false, nil
}

func componentGroupNamesForSpecPath(env *azldev.Env, specPath string) ([]string, error) {
var groupNames []string

for groupName, group := range env.Config().ComponentGroups {
matches, err := componentGroupMatchesSpecPath(&group, specPath)
if err != nil {
return nil, err
}

if matches {
groupNames = append(groupNames, groupName)
}
}

return groupNames, nil
}

func (r *Resolver) addComponentsBySpecPathToSet(specPath string, components *ComponentSet) error {
var component Component

Expand Down Expand Up @@ -395,7 +442,9 @@ func (r *Resolver) addComponentsByGroupNameToSet(groupName string, components *C
for _, groupMember := range componentGroup.Components {
var component Component

component, err = r.getComponentFromNameAndSpecPath(groupMember.ComponentName, groupMember.SpecPath)
component, err = r.getComponentFromNameAndSpecPath(
groupMember.ComponentName, groupMember.SpecPath, []string{groupName},
)
if err != nil {
return fmt.Errorf(
"failed to enumerate components in group '%s':\n%w", groupName, err)
Expand All @@ -421,7 +470,12 @@ func (r *Resolver) getComponentForSpecPath(specPath string) (component Component
return component, fmt.Errorf("failed to verify spec '%s' exists:\n%w", specPath, statErr)
}

return r.getComponentFromNameAndSpecPath(name, specPath)
groupNames, err := componentGroupNamesForSpecPath(r.env, specPath)
if err != nil {
return component, err
}

return r.getComponentFromNameAndSpecPath(name, specPath, groupNames)
}

// Given a path to a .spec file, deduce the component's name.
Expand All @@ -438,7 +492,9 @@ func deduceComponentNameFromSpec(specPath string) string {
}

// Finds the named component in the provided environment; returns its configuration. Returns error if it can't be found.
func (r *Resolver) getComponentFromNameAndSpecPath(name, specPath string) (component Component, err error) {
func (r *Resolver) getComponentFromNameAndSpecPath(
name, specPath string, groupNames []string,
) (component Component, err error) {
config := r.env.Config()
if config == nil {
return component, errors.New("no project config loaded")
Expand All @@ -459,7 +515,11 @@ func (r *Resolver) getComponentFromNameAndSpecPath(name, specPath string) (compo
var updatedComponentConfig *projectconfig.ComponentConfig

// Apply inherited defaults to the component.
updatedComponentConfig, err = applyInheritedDefaultsToComponent(r.env, foundComponentConfig)
overlayReferenceDir := overlayFilesReferenceDir(foundComponentConfig, specPath)

updatedComponentConfig, err = applyInheritedDefaultsToComponent(
r.env, foundComponentConfig, overlayReferenceDir, groupNames,
)
if err != nil {
return component, err
}
Expand Down Expand Up @@ -777,15 +837,27 @@ func (r *Resolver) warnOnLockDrift(resolved *ComponentSet) {
}

// Given an explicit component config, apply all inherited defaults.
func overlayFilesReferenceDir(component projectconfig.ComponentConfig, specPath string) string {
if component.SourceConfigFile != nil {
return component.SourceConfigFile.Dir()
}

if specPath != "" {
return filepath.Dir(specPath)
}

return ""
}

func applyInheritedDefaultsToComponent(
env *azldev.Env, component projectconfig.ComponentConfig,
env *azldev.Env, component projectconfig.ComponentConfig, overlayFilesReferenceDir string, extraGroupNames []string,
) (result *projectconfig.ComponentConfig, err error) {
_, distroVer, err := env.Distro()
if err != nil {
return nil, fmt.Errorf("failed to resolve current distro:\n%w", err)
}

groupNames := env.Config().GroupsByComponent[component.Name]
groupNames := componentGroupNames(env, component.Name, extraGroupNames)

resolved, err := projectconfig.ResolveComponentConfig(
component,
Expand All @@ -798,9 +870,28 @@ func applyInheritedDefaultsToComponent(
return nil, fmt.Errorf("resolving config for component '%s':\n%w", component.Name, err)
}

resolved, err = projectconfig.ExpandResolvedOverlayFiles(
env.FS(), resolved, overlayFilesReferenceDir, env.PermissiveConfigParsing(),
)
if err != nil {
return nil, fmt.Errorf("resolving overlay files for component '%s':\n%w", component.Name, err)
}

return &resolved, nil
}

func componentGroupNames(env *azldev.Env, componentName string, extraGroupNames []string) []string {
groupNames := slices.Clone(env.Config().GroupsByComponent[componentName])

for _, groupName := range extraGroupNames {
if !slices.Contains(groupNames, groupName) {
groupNames = append(groupNames, groupName)
}
}

return groupNames
}

// validateLockFiles checks lock file consistency against the resolved component
// set. Skipped when skipValidation is true (set per-filter or via the global
// '--skip-lock-validation' flag).
Expand Down
92 changes: 89 additions & 3 deletions internal/app/azldev/core/components/resolver_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -201,9 +201,9 @@ func TestFindComponents_FoundGroups(t *testing.T) {
}

// Setup the specs and compose a list of expected components.
expectedComponentConfigs := []projectconfig.ComponentConfig{}
for _, specPath := range specPaths {
expectedComponentConfigs = append(expectedComponentConfigs, setupTestSpec(t, env, specPath))
expectedComponentConfigs := make([]projectconfig.ComponentConfig, len(specPaths))
for idx, specPath := range specPaths {
expectedComponentConfigs[idx] = setupTestSpec(t, env, specPath)
}

// Find!
Expand Down Expand Up @@ -705,6 +705,92 @@ func TestFindAllSpecPaths_MultipleSpecs(t *testing.T) {
assert.ElementsMatch(t, []string{"/specs/a/test-a.spec", "/specs/b/test-b.spec"}, specPaths)
}

func TestFindComponents_SpecDiscoveredComponentInheritsOverlayFilesFromDefaults(t *testing.T) {
env := testutils.NewTestEnv(t)

touchFile(t, env, "/specs/ant/ant.spec")
require.NoError(t, fileutils.WriteFile(env.TestFS,
"/specs/ant/overlays/0001-branding.overlay.toml",
[]byte(`
[metadata]
category = "azl-branding-policy"

[[overlays]]
type = "spec-set-tag"
tag = "Vendor"
value = "Microsoft"
`), fileperms.PrivateFile))

env.Config.ComponentGroups["specs"] = projectconfig.ComponentGroupConfig{
SpecPathPatterns: []string{"/specs/**/*.spec"},
DefaultComponentConfig: projectconfig.ComponentConfig{
OverlayFiles: []string{"overlays/*.overlay.toml"},
},
}

filter := &components.ComponentFilter{
IncludeAllComponents: true,
SkipLockValidation: true,
}

resolved, err := components.NewResolver(env.Env).FindComponents(filter)
require.NoError(t, err)

comp, ok := resolved.TryGet("ant")
require.True(t, ok)

overlays := comp.GetConfig().Overlays
require.Len(t, overlays, 1)
assert.Empty(t, comp.GetConfig().OverlayFiles)
assert.Equal(t, projectconfig.ComponentOverlaySetSpecTag, overlays[0].Type)
assert.Equal(t, "Vendor", overlays[0].Tag)
require.NotNil(t, overlays[0].Metadata)
assert.Equal(t, projectconfig.OverlayCategoryAZLBrandingPolicy, overlays[0].Metadata.Category)
}

func TestFindComponents_SpecPathFilterInheritsOverlayFilesFromGroupDefaults(t *testing.T) {
env := testutils.NewTestEnv(t)

touchFile(t, env, "/specs/ant/ant.spec")
require.NoError(t, fileutils.WriteFile(env.TestFS,
"/specs/ant/overlays/0001-branding.overlay.toml",
[]byte(`
[metadata]
category = "azl-branding-policy"

[[overlays]]
type = "spec-set-tag"
tag = "Vendor"
value = "Microsoft"
`), fileperms.PrivateFile))

env.Config.ComponentGroups["specs"] = projectconfig.ComponentGroupConfig{
SpecPathPatterns: []string{"/specs/**/*.spec"},
DefaultComponentConfig: projectconfig.ComponentConfig{
OverlayFiles: []string{"overlays/*.overlay.toml"},
},
}

filter := &components.ComponentFilter{
SpecPaths: []string{"/specs/ant/ant.spec"},
SkipLockValidation: true,
}

resolved, err := components.NewResolver(env.Env).FindComponents(filter)
require.NoError(t, err)

comp, ok := resolved.TryGet("ant")
require.True(t, ok)

overlays := comp.GetConfig().Overlays
require.Len(t, overlays, 1)
assert.Empty(t, comp.GetConfig().OverlayFiles)
assert.Equal(t, projectconfig.ComponentOverlaySetSpecTag, overlays[0].Type)
assert.Equal(t, "Vendor", overlays[0].Tag)
require.NotNil(t, overlays[0].Metadata)
assert.Equal(t, projectconfig.OverlayCategoryAZLBrandingPolicy, overlays[0].Metadata.Category)
}

// When a lock file exists for a component, the resolver should attach all of
// its data (commit, import-commit, manual-bump, fingerprint) to the resolved
// component via the Locked field — without touching the original Spec config.
Expand Down
24 changes: 16 additions & 8 deletions internal/projectconfig/component.go
Original file line number Diff line number Diff line change
Expand Up @@ -273,18 +273,20 @@ type ComponentConfig struct {
// Overlays to apply to sources after they've been acquired. May mutate the spec as well as sources.
Overlays []ComponentOverlay `toml:"overlays,omitempty" json:"overlays,omitempty" table:"-" jsonschema:"title=Overlays,description=Overlays to apply to this component's spec and/or sources"`

// OverlayFiles, if set, lists glob patterns (relative to this component config file)
// matched against the filesystem at load time to locate per-file overlay documents.
// OverlayFiles, if set, lists path or glob patterns (relative to this component config file)
// matched against the filesystem after component config resolution to locate per-file overlay documents.
// Each matched file is parsed as an [OverlayFile]: one logical change consisting of a
// file-level `[metadata]` block plus an ordered list of `[[overlays]]`. The per-file
// metadata is stamped onto every overlay in the file. Matches are concatenated in the
// order patterns are declared; within a single pattern, matches are applied in
// filename (lexicographic) order, using the full path as a tie-breaker when
// filenames match. Duplicate matches are de-duplicated, preserving first
// occurrence. The resulting overlays are appended to [ComponentConfig.Overlays]
// after any inline overlays. Excluded from the fingerprint because the value affects
// only where overlays are sourced from, not their content.
OverlayFiles []string `toml:"overlay-files,omitempty" json:"overlayFiles,omitempty" table:"-" validate:"dive,required" jsonschema:"title=Overlay files,description=Glob patterns (relative to the component config file) matched against the filesystem to locate per-file overlay documents at load time" fingerprint:"-"`
// after any inline overlays. A value set in a higher-priority config layer replaces
// lower-priority overlay-files values; an explicit empty list disables inherited
// overlay files. Excluded from the fingerprint because the value affects only where
// overlays are sourced from, not their content.
OverlayFiles []string `toml:"overlay-files,omitempty" json:"overlayFiles,omitempty" table:"-" validate:"dive,required" jsonschema:"title=Overlay files,description=Path or glob patterns (relative to the component config file or matched spec directory) matched against the filesystem to locate per-file overlay documents after component config resolution. Use an empty list to disable inherited overlay-file patterns" fingerprint:"-"`

// Configuration for building the component.
Build ComponentBuildConfig `toml:"build,omitempty" json:"build,omitempty" table:"-" jsonschema:"title=Build configuration,description=Configuration for building the component"`
Expand Down Expand Up @@ -317,11 +319,17 @@ var AllowedSourceFilesHashTypes = map[fileutils.HashType]bool{

// Mutates the component config, updating it with overrides present in other.
func (c *ComponentConfig) MergeUpdatesFrom(other *ComponentConfig) error {
otherOverlayFiles := slices.Clone(other.OverlayFiles)

err := mergo.Merge(c, other, mergo.WithOverride, mergo.WithAppendSlice)
if err != nil {
return fmt.Errorf("failed to merge project info:\n%w", err)
}

if other.OverlayFiles != nil {
c.OverlayFiles = otherOverlayFiles
}

return nil
}

Expand Down Expand Up @@ -396,9 +404,9 @@ func (c *ComponentConfig) WithAbsolutePaths(referenceDir string) *ComponentConfi
SourceFiles: deep.MustCopy(c.SourceFiles),
Packages: deep.MustCopy(c.Packages),
Publish: deep.MustCopy(c.Publish),
// OverlayFiles is consumed at load time (see applyOverlayFiles) before paths are
// absolutized; nothing reads it afterward, so preserve the original value verbatim
// rather than redundantly re-rooting each glob.
// OverlayFiles is consumed after component config resolution; preserve it verbatim
// here so inherited patterns can be interpreted relative to the concrete component
// config file.
OverlayFiles: slices.Clone(c.OverlayFiles),
}

Expand Down
Loading
Loading