Skip to content

Issue 4849 omnia modernization - #4917

Open
balajikumaran-c-s wants to merge 8 commits into
dell:issue-4849-omnia-modernizationfrom
balajikumaran-c-s:issue-4849-omnia-modernization
Open

Issue 4849 omnia modernization#4917
balajikumaran-c-s wants to merge 8 commits into
dell:issue-4849-omnia-modernizationfrom
balajikumaran-c-s:issue-4849-omnia-modernization

Conversation

@balajikumaran-c-s

@balajikumaran-c-s balajikumaran-c-s commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a comprehensive Functional Verification Test (FVT) suite for the image_build_manager module, covering the complete image build lifecycle: prepare, build, validate, and cleanup phases.

What This PR Does

End-to-End Image Build Lifecycle Verification

Prepare Phase Validation

Verifies all prerequisites required for image build operations:

  • MinIO (S3) backend availability and configuration
  • Container registry deployment and accessibility
  • Required systemd services status
  • Firewall port configuration
  • s3cmd configuration and connectivity
  • Registry reachability checks
  • S3 bucket creation and validation

Build Phase Validation

Validates successful image creation and storage:

  • Image availability in S3 buckets
  • Image availability in container registry
  • Image build status verification
  • Functional group image build validation
  • Package-level verification for generated images
  • Support for both x86_64 and aarch64 architectures

Validate Phase Verification

Confirms deployment artifacts are correctly synchronized:

  • Input configuration synchronization
  • Credentials synchronization
  • Input validation checks on the target system

Cleanup Phase Verification

Ensures proper teardown and resource cleanup:

  • Removal of generated resources
  • Cleanup validation after execution

Test Framework Features

  • Scenario-based execution using run_validation.sh

    • deploy
    • verify
    • test
  • Flexible test selection:

    • Suite filtering (--suite container, --suite s3)
    • Marker filtering (--marker sanity, --marker x86_64)
  • Automated reporting:

    • HTML report generation
    • JSON report generation
    • Test case IDs
    • Execution duration tracking
    • Pass/fail summary
  • One-time environment setup via setup_env.sh

    • Python virtual environment creation
    • Shell tab-completion support
  • Supports both:

    • Local execution
    • Remote execution over SSH
  • Automatic Ansible Vault encryption for test credentials

  • Dataset-driven testing with configurable:

    • Input files
    • Repository Manager outputs

Test Coverage

Phase Test Count Coverage
Prepare 8 S3 backend, registry, services, ports, s3cmd, buckets
Build 12 S3 images, registry images, build status, package validation (x86_64/aarch64)
Validate 3 Configuration sync, credentials sync, input validation
Cleanup 2 Resource cleanup verification

Integration with omnia-auto

Leverages the shared omnia-auto automation framework for:

  • SSH connectivity
  • Configuration management
  • File synchronization
  • Playbook execution
  • Report generation

This eliminates duplicate common automation code and aligns the test suite with the broader Omnia validation framework.

Testing

Validated successfully on a live deployment:

prepare verify --marker sanity
7 passed, 0 failed

validate verify --marker sanity
2 passed, 0 failed

image_build_manager verify --marker sanity
8 passed, 0 failed, 4 skipped (aarch64-specific tests)

@balajikumaran-c-s
balajikumaran-c-s force-pushed the issue-4849-omnia-modernization branch from 80e6967 to 4533e45 Compare July 31, 2026 11:35
- Adapted from multi-repo image-build-manager test structure
- Replaced git clone with rsync project sync (sync_project_to_remote)
- Input sync reads OMNIA_DATA_PATH and OMNIA_PROJECT_NAME from target env vars
- Removed config.yml sync (monorepo uses env vars instead)
- Playbook workdir: src/image_build_manager/playbooks/
- Uses omnia-auto pip package for common utilities
- All sensitive data (IPs, passwords, credentials) reset to placeholders
- Updated README with monorepo architecture and env var documentation

Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
- Remove config.yml from dataset (monorepo uses env vars via omnia.env)
- Fix image_build_credentials.yml to match target fields (s3_access_id,
  s3_secret_key, aarch64_ssh_password)
- Fix _read_remote_env to source /etc/omnia/omnia.env for non-login shells
  (testinfra uses non-login SSH which skips /etc/profile.d/)
- Add _ensure_remote_dir to create target directories before sync
  (prevents rsync failures when playbook hasn't run yet)
- Remove config.yml from REQUIRED_DATASET_FILES validation
- Update all READMEs: proper setup flow, env var setup via omnia.sh,
  removed config.yml references, dataset sync paths updated
- Reset main test_creds.yml to placeholder (was committed encrypted)
- All sensitive data sanitized (IPs, passwords, hostnames)

Verified: 8 passed, 0 failed, 4 skipped against target server
Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
Documents how the consumer module integrates with the omnia-auto pip
package: configure(), run_playbook wrapper pattern, test file usage.
Updated install instructions to show PyPI as primary method.

Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
Security fixes:
- Reset test_creds.yml: replaced vault-encrypted real credentials
  with plaintext CHANGE_ME placeholder

Lint fixes:
- Removed unused FIREWALL_PORTS import from build_image_func.py
- Refactored host_func.py: use omnia_auto connection_params,
  read_remote_env, ensure_remote_dir, resolve_domain_input_path
  instead of local duplicates
- Added run_playbook wrapper in functions/__init__.py
- Added PLAYBOOK_ENTRY_POINT, PLAYBOOK_WORKDIR to common_vars.py

Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
- docs/test_creds.md: use short placeholders (<SSH_PASSWORD>, any) to avoid
  gitleaks generic-password rule (values under 8 chars)
- build_image_func.py: add nosec B108 for remote SSH temp paths
- repo_status.yml: add missing newline at end of file
- test_config.yml: remove extra trailing blank line
- test_creds.yml: reset to empty placeholder with copyright header
- host_func.py: use __all__ for re-exports instead of noqa/pylint-disable,
  remove all suppression comments, use _host for unused parameter
- __init__.py: move all imports to top, remove pylint-disable
- conftest.py: remove pylint-disable from omnia_auto imports
- build_image_func.py: remove pylint-disable from omnia_auto import
- requirements.txt: add omnia-auto as git dependency

Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
@balajikumaran-c-s
balajikumaran-c-s force-pushed the issue-4849-omnia-modernization branch from 4533e45 to e0ec965 Compare July 31, 2026 12:20
Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
@balajikumaran-c-s
balajikumaran-c-s marked this pull request as ready for review July 31, 2026 13:30
Add reusable test automation utilities package for Dell Omnia test suites.

Module structure (flat layout):
- omnia_auto/functions/ — formatting, host, runner, sync, report
- omnia_auto/vars/ — configuration management
- omnia_auto/messages/ — log and assertion messages
- docs/ — 7 usage guides covering all APIs
- dist/ — pre-built wheel for pip install

Key features:
- TestLogger with colored output and symbols
- Testinfra host setup with vault-encrypted credentials
- run_playbook with live output streaming
- clone_repo/sync_files for local and SSH operations
- TestReport with HTML/JSON generation
- configure() for consumer-driven settings

Install: pip install test/omnia-auto/dist/omnia_auto-1.0.0-py3-none-any.whl
Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
Rename test/omnia-auto/ to test/plugins/ for consistency with src/ structure.
Inner package remains omnia_auto/ to preserve 'import omnia_auto' imports.

Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant