[file-diet] File Diet: refactor pkg/parser/import_field_extractor.go

[github-actions]

Overview

The file pkg/parser/import_field_extractor.go has grown to 1085 lines, making it difficult to maintain and test. This task involves refactoring it into smaller, focused files with improved test coverage.

Current State

• File: pkg/parser/import_field_extractor.go
• Size: 1085 lines
• Test Coverage: import_field_extractor_test.go is 1163 lines (~1.07x ratio — healthy)
• Complexity: The file mixes 53 functions/methods spanning at least 8 distinct concerns on the single importAccumulator struct: BFS traversal orchestration, engine config extraction, sandbox mount merging, generic JSON/YAML merge helpers, activation-field extraction, step/job/feature merging, model-policy normalization, and final ImportsResult construction.

Full File Analysis

Detailed Breakdown

• Lines 1–110: importAccumulator struct definition (60+ fields) — the shared state bag threaded through all extraction methods.
• Lines 111–310 (core orchestration, ~200 lines): newImportAccumulator, extractAllImportFields, prepareFrontmatter, parseOriginalFrontmatter, frontmatterMapOrEmpty, applyImportDefaultsToContent, collectInlineSubAgentWarnings, validateSubAgentFrontmatterWarnings, extractToolsContent, trackRuntimeOrInlineImport, appendMarkdownWithSeparator, parseFrontmatterForExtraction.
• Lines 311–392 (~82 lines): extractEngineConfig, extractEngineMCPSettings — engine/MCP settings extraction.
• Lines 393–507 (~115 lines): extractConfigFields, mergeSandboxAgentMounts, mergeSandboxAgentRuntimeInstall — sandbox/config merging (duplicate-set-tracking pattern repeated here and elsewhere).
• Lines 508–604 (~97 lines): extractFirstWinsJSONField, appendJSONBuilderField, appendJSONSliceField, appendYAMLBuilderField, mergeJSONStringListField — generic reusable JSON/YAML field merge helpers, not accumulator-specific logic.
• Lines 551–668 (~120 lines, interleaved with above): extractActivationFields, mergeBots, mergeSkipRoles, mergeSkipBots, mergeAmbientFolders, extractActivationSkipMatchFields, extractActivationGitHubToken, extractActivationGitHubAppFields, extractCheckoutField — on:/activation-related field extraction.
• Lines 669–754 (~85 lines): extractStepAndJobFields, extractFeatureAndObservabilityFields, mergeExcludedEnv, mergeLabels, appendCacheField, appendFeaturesField — steps/jobs/features/cache merging.
• Lines 755–927 (~170 lines): appendModelsField, normalizeModelPolicies, normalizeModelAliases, parseModelPolicyField, sanitizeModelProvidersForCosts, parseStringSliceField, isModelPolicyKey — model alias/policy/cost normalization, the single largest cohesive block and a natural extraction candidate.
• Lines 907–973 (~65 lines): extractPlugins, extractRunInstallScripts, hasNodeRuntimeRunInstallScripts, appendObservabilityField — plugin and runtime-install-script handling.
• Lines 974–1085 (~110 lines): toImportsResult, buildImportsResult, populateImportsResultScalars, computeImportRelPath, validateGitHubAppJSON — final ImportsResult assembly from accumulated state.

The generic JSON/YAML merge helpers (appendJSONBuilderField, appendJSONSliceField, appendYAMLBuilderField, mergeJSONStringListField, extractFirstWinsJSONField) are duplicated conceptually across the sandbox-mount, activation, and model-policy sections — good candidates to centralize once extracted into their own file so each domain-specific extractor imports the same primitives instead of re-implementing set/slice merge logic.

Refactoring Strategy

Proposed File Splits

Based on semantic analysis of importAccumulator's responsibilities, split into the following modules (all remain in package parser, same directory):

1. import_field_extractor.go (kept, trimmed to core orchestration)

   • Functions: importAccumulator struct, newImportAccumulator, extractAllImportFields, prepareFrontmatter, parseOriginalFrontmatter, frontmatterMapOrEmpty, applyImportDefaultsToContent, collectInlineSubAgentWarnings, validateSubAgentFrontmatterWarnings, extractToolsContent, trackRuntimeOrInlineImport, appendMarkdownWithSeparator, parseFrontmatterForExtraction
   • Responsibility: struct definition + top-level BFS-traversal orchestration entry point
   • Estimated LOC: ~310

2. import_field_merge_helpers.go

   • Functions: extractFirstWinsJSONField, appendJSONBuilderField, appendJSONSliceField, appendYAMLBuilderField, mergeJSONStringListField
   • Responsibility: generic reusable first-wins and set/slice merge primitives used by every domain-specific extractor below
   • Estimated LOC: ~100

3. import_field_engine.go

   • Functions: extractEngineConfig, extractEngineMCPSettings
   • Responsibility: engine id/model/MCP timeout extraction from imported frontmatter
   • Estimated LOC: ~85

4. import_field_sandbox.go

   • Functions: extractConfigFields, mergeSandboxAgentMounts, mergeSandboxAgentRuntimeInstall
   • Responsibility: sandbox/config-related field merging (mounts, runtime-install flag)
   • Estimated LOC: ~120

5. import_field_activation.go

   • Functions: extractActivationFields, mergeBots, mergeSkipRoles, mergeSkipBots, mergeAmbientFolders, extractActivationSkipMatchFields, extractActivationGitHubToken, extractActivationGitHubAppFields, extractCheckoutField
   • Responsibility: on: trigger / activation-related field extraction (bots, skip-roles, github-token/app, checkout)
   • Estimated LOC: ~125

6. import_field_jobs.go

   • Functions: extractStepAndJobFields, extractFeatureAndObservabilityFields, mergeExcludedEnv, mergeLabels, appendCacheField, appendFeaturesField, appendObservabilityField
   • Responsibility: steps/jobs/features/cache/observability field extraction
   • Estimated LOC: ~100

7. import_field_models.go

   • Functions: appendModelsField, normalizeModelPolicies, normalizeModelAliases, parseModelPolicyField, sanitizeModelProvidersForCosts, parseStringSliceField, isModelPolicyKey
   • Responsibility: model alias/policy/cost normalization (largest, most self-contained domain)
   • Estimated LOC: ~175

8. import_field_plugins.go

   • Functions: extractPlugins, extractRunInstallScripts, hasNodeRuntimeRunInstallScripts
   • Responsibility: plugin list extraction and runtime-install-scripts detection
   • Estimated LOC: ~50

9. import_result_builder.go

   • Functions: toImportsResult, buildImportsResult, populateImportsResultScalars, computeImportRelPath, validateGitHubAppJSON
   • Responsibility: final assembly of ImportsResult from accumulated state
   • Estimated LOC: ~115

Shared Utilities

Extract common functionality into:

• import_field_merge_helpers.go: centralizes the first-wins / JSON-builder / slice-merge primitives currently duplicated conceptually across sandbox, activation, and model sections.

Interface Abstractions

• Consider a small fieldExtractor interface (extract(fm map[string]any, fullPath string)) if further extractors are added later, but given importAccumulator is a straightforward mutable accumulator, plain grouped functions on the existing receiver are sufficient — no interface is required for this refactor.

Test Coverage Plan

Split import_field_extractor_test.go (1163 lines) along the same boundaries so each new source file has a matching test file:

1. import_field_merge_helpers_test.go

   • Test cases: first-wins JSON field precedence across multiple imports; JSON builder/slice append with empty vs. populated values; YAML builder field merge; string-list merge de-duplication
   • Target coverage: >80%

2. import_field_engine_test.go

   • Test cases: engine id/model extraction precedence; MCP tool-timeout/session-timeout first-wins across imports; missing/malformed engine config
   • Target coverage: >80%

3. import_field_sandbox_test.go

   • Test cases: sandbox mount merge and de-duplication; runtime-install: false short-circuit across multiple imports; missing sandbox config
   • Target coverage: >80%

4. import_field_activation_test.go

   • Test cases: bots/skip-roles/skip-bots merge dedup; ambient folders merge; github-token/github-app first-wins; checkout JSON accumulation order
   • Target coverage: >80%

5. import_field_jobs_test.go

   • Test cases: steps/job field extraction errors; excluded-env union dedup; labels merge; cache/feature/observability field appends
   • Target coverage: >80%

6. import_field_models_test.go

   • Test cases: model alias/policy normalization edge cases (empty, malformed, mixed types); cost provider sanitization; policy key validation
   • Target coverage: >80%

7. import_field_plugins_test.go

   • Test cases: plugin extraction from multiple imports; run-install-scripts detection across node runtime variants
   • Target coverage: >80%

8. import_result_builder_test.go

   • Test cases: full ImportsResult assembly from a populated accumulator; scalar population edge cases; relative import path computation; GitHub App JSON validation failures

Implementation Guidelines

1. Preserve Behavior: Ensure all existing functionality works identically
2. Maintain Exports: Keep public API unchanged (exported functions/types) — all functions listed above are currently unexported (importAccumulator methods/package-private helpers), so no external API changes are expected
3. Add Tests First: Split test file alongside each new source file before/while moving code
4. Incremental Changes: Split one module at a time (start with import_field_merge_helpers.go since other extractors depend on it)
5. Run Tests Frequently: Verify make test-unit passes after each split
6. Update Imports: Ensure all import paths are correct (all files stay in package parser, so no import path changes needed — just file moves)
7. Document Changes: Add comments explaining module boundaries per new file's package doc comment

Acceptance Criteria

• Original file is split into 9 focused files
• Each new file is under 500 lines
• All tests pass (make test-unit)
• Test coverage is ≥80% for new files
• No breaking changes to public API
• Code passes linting (make lint)
• Build succeeds (make build)

Additional Context

• Repository Guidelines: Follow patterns in .github/agents/developer.instructions.agent.md
• Code Organization: Prefer many small files grouped by functionality
• Testing: Match existing test patterns in pkg/parser/*_test.go

---

Priority: Medium
Effort: Medium (9 well-bounded, low-risk mechanical extractions with no exported API changes)
Expected Impact: Improved maintainability, easier targeted testing per domain, reduced complexity in the largest file in pkg/

Generated by 🧹 Daily File Diet · copilot · auto · 145.2 AIC · ⌖ 11.5 AIC · ⊞ 10.2K · ◷

• expires on Sep 2, 2026, 5:01 AM UTC-08:00