ADR-006: Testing and Execution Environment

Date

2025-03-09

Status

Accepted

Decision Makers

Context

The OpenShift Agent-Based Installer Helper requires a consistent and reliable environment for:

Considered Options

1. Basic Script Testing

2. Comprehensive Testing Framework (Selected)

Decision

Implement a comprehensive testing framework with:

  1. Core Testing Components 
    e2e-tests/
├── bootstrap_env.sh    # Environment setup and dependencies
├── validate_env.sh     # Environment validation
├── run_e2e.sh         # Test orchestration
└── delete_e2e.sh      # Environment cleanup
  2. Infrastructure Management 
    hack/
├── create-iso.sh              # ISO generation
├── deploy-on-kvm.sh           # VM deployment
├── watch-and-reboot-kvm-vms.sh # VM monitoring
├── configure_dns_entries.sh    # DNS configuration
└── deploy-freeipa.sh          # Identity management

Implementation

System Dependencies

Required packages and tools:

Test Framework Components

  1. Environment Setup (bootstrap_env.sh)
    • System package installation
    • Ansible collection setup
    • OpenShift CLI tools installation
    • Container runtime configuration
    • SELinux configuration
    • Infrastructure setup
    • Registry authentication
  2. Environment Validation (validate_env.sh)
    • System requirements verification
    • Network configuration validation
    • Required tools availability
    • Infrastructure service status
  3. Test Execution (run_e2e.sh)
    • Site configuration validation
    • ISO creation and validation
    • VM deployment and monitoring
    • Installation progress tracking
    • Test case execution
    • Results collection
  4. Environment Cleanup (delete_e2e.sh)
    • VM cleanup
    • Network configuration cleanup
    • Resource removal
    • Environment restoration

Test Flow

  1. Pre-execution Setup 
    # Bootstrap environment
./e2e-tests/bootstrap_env.sh

# Validate environment
./e2e-tests/validate_env.sh
  2. Test Execution 
    # Run end-to-end tests with site config
./e2e-tests/run_e2e.sh <site_config_dir>
  3. Cleanup 
    # Clean up test environment
./e2e-tests/delete_e2e.sh

Consequences

Positive

  1. Automated end-to-end testing
  2. Consistent test environments
  3. Comprehensive validation
  4. Infrastructure management
  5. Clear error reporting
  6. Support for different deployment scenarios

Negative

  1. Complex setup requirements
  2. Resource-intensive testing
  3. External dependencies
  4. Platform-specific considerations

Test Coverage

Installation Scenarios

  1. Connected installations
  2. Disconnected installations
  3. Single-node deployments
  4. Multi-node clusters

Infrastructure Testing

  1. Network configuration
  2. Storage setup
  3. DNS management
  4. Identity integration

Validation Testing

  1. Environment prerequisites
  2. Tool availability
  3. Resource requirements
  4. Configuration validation