local-ci-cd-system/docs/TEST-PLAN-v1.3-to-HEAD.md

Test Plan: v1.3 → HEAD

Complete validation of all features added since v1.3 tag (2026-05-10). Organized by sprint area with pre-requisites, test steps, and pass/fail criteria.

---

Overview

Area	Items	Status	Baseline
Sprint 2 (Reliability)	2.1/2.2/2.3/2.5	Partial	§2.2/§2.3 validated locally 2026-05-10; §2.1/§2.5 need real VMs
Sprint 3 (Observability)	4.1/4.3/4.4	Partial	§4.3/§4.4 validated locally 2026-05-10; §4.1 (JSONL) needs real build job
Sprint 4-6 (Quality/Perf)	1.4/3.1/3.6/5.2/5.3/5.4	Partial	§1.4/§5.2/§5.3/§5.4 validated 2026-05-10; §3.1/§3.6 need real VMs
Sprint 7 (Testing)	5.1	[x] Done	22/22 Pester tests passing in 12.9s (re-validated 2026-05-11)
§1.6 (Docs)	Threat model	[x] Done	BEST-PRACTICES.md verified 2026-05-10
§3.2 (Perf)	7-Zip compression	Partial	Fallback path validated; in-VM 7-Zip speed test needs real VM
§5.4 (PSScriptAnalyzer)	0 errors, 298 warnings	[x] Done	All 6 actionable warnings fixed 2026-05-11; 0 Error severity violations
§6.1 (Linux Build VM)	Deploy/Prepare/e2e	[x] Done	Test-Ns7zipBuild-Linux.ps1 4xPASS 2026-05-11 — nsis7z.dll 2272 KB

---

Pre-Test Checklist

• Host Windows 11 with VMware Workstation 17+
• Template VM F:\CI\Templates\WinBuild2025\WinBuild2025.vmx exists + BaseClean snapshot present (verified 2026-05-10)
• F:\CI\ directory structure intact (BuildVMs, Artifacts, Logs, Cache, State) (verified 2026-05-10)
• act_runner service running (Get-Service act_runner | Select Status) (verified 2026-05-10)
• Gitea available at http://10.10.20.11:3100 or https://gitea.emulab.it
• PowerShell ≥5.1, Pester v5 installed (Install-Module Pester -MinimumVersion 5.0 -Force) (Pester 5.7.1 installed 2026-05-10)
• CredentialManager module installed (Install-Module CredentialManager) (verified 2026-05-10)
• [NEW 2026-05-10] Template includes §6.6 Tier-1 Toolchain: Git for Windows + 7-Zip (Step 11 in Install-CIToolchain-WinBuild2025.ps1)

---

Sprint 2 — Reliability & Resilience (§2.1/2.2/2.3/2.5)

2.1 IP Allocation Race Prevention

What changed: Exclusive file-based IP lease allocation to prevent race condition where two parallel clones receive the same IP from DHCP.

Files affected:

• scripts/Invoke-CIJob.ps1 (lines 244-253: IP leak management)
• runner/config.yaml (capacity = 4 for parallel jobs)

Test Steps:

# 1. Verify IP state directory exists
Test-Path F:\CI\State\ip-leases\
# Expected: True

# 2. Start 4 parallel jobs on nsis-plugin-nsinnounp (or similar)
# Simulate via:
1..4 | ForEach-Object -Parallel {
    & 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-CIJob.ps1' `
        -RepositoryUrl 'http://10.10.20.11:3100/Simone/nsis-plugin-nsinnounp' `
        -JobId "test-parallel-$_" `
        -Branch main
}

# 3. Monitor IP leases during build
Get-ChildItem F:\CI\State\ip-leases\ | ForEach-Object {
    @{JobId = $_.BaseName; IP = (Get-Content $_.FullName) }
}
# Expected: 4 unique IPs in range 192.168.79.100-192.168.79.104
# (No duplicate IPs across jobs)

# 4. After builds complete, leases should be released
Get-ChildItem F:\CI\State\ip-leases\
# Expected: Empty (all cleaned up in finally block)

Pass Criteria:

• All 4 jobs complete successfully
• No WinRM "connection to wrong VM" errors
• Each job receives unique IP
• Leases cleaned up after completion

---

2.2 Orphaned VM Cleanup (Scheduled)

What changed: Automated cleanup script + scheduled task to remove stale clones if host crashes mid-job or act_runner dies.

Files affected:

• scripts/Cleanup-OrphanedBuildVMs.ps1 (main cleanup logic)
• scripts/Register-CIScheduledTasks.ps1 (task registration: CI-CleanupOrphans every 6 hours)

Test Steps:

# 1. Verify scheduled task exists
Get-ScheduledTask -TaskName 'CI-CleanupOrphans' | Select Name, State, NextRunTime
# Expected: Task exists, State = Ready

# 2. Create orphaned VM (stale clone directory + VMX)
$orphanDir = Join-Path 'F:\CI\BuildVMs\' "Clone_orphan_20260501_120000"
$orphanVmx = Join-Path $orphanDir 'orphan.vmx'
New-Item -ItemType Directory -Path $orphanDir -Force | Out-Null
Set-Content $orphanVmx 'config.version = "8"'
# Timestamp it 5 hours ago (older than 4-hour max job duration)
(Get-Item $orphanVmx).LastWriteTime = (Get-Date).AddHours(-5)

# 3. Run cleanup manually
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Cleanup-OrphanedBuildVMs.ps1' -Verbose

# 4. Verify orphan removed
Test-Path $orphanDir
# Expected: False (cleaned up)

Pass Criteria:

• Scheduled task runs without error
• Orphaned VM detected and removed
• Recent clones (< 4 hours old) preserved
• Event log shows cleanup action

---

2.3 Artifact + Log Retention

What changed: Automated retention policy to prune artifacts older than 30 days and logs, preventing F:\ from filling.

Files affected:

• scripts/Invoke-RetentionPolicy.ps1 (main policy)
• scripts/Register-CIScheduledTasks.ps1 (task registration: CI-RetentionPolicy daily at 4am)

Test Steps:

# 1. Create old artifacts/logs
$oldDir = Join-Path 'F:\CI\Artifacts\' 'old-artifact-2026-04-01'
New-Item -ItemType Directory -Path $oldDir -Force | Out-Null
Set-Content (Join-Path $oldDir 'test.txt') 'old artifact'
(Get-Item $oldDir).LastWriteTime = (Get-Date).AddDays(-31)

# 2. Run retention policy manually
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-RetentionPolicy.ps1' -Verbose

# 3. Verify old artifact removed, recent preserved
Test-Path $oldDir
# Expected: False

$recentDir = Join-Path 'F:\CI\Artifacts\' 'recent-artifact'
New-Item -ItemType Directory -Path $recentDir -Force | Out-Null
Test-Path $recentDir  # After running retention policy
# Expected: True (recent artifacts kept)

# 4. Monitor disk space alert
$diskFree = (Get-PSDrive F).Free / 1GB
if ($diskFree -lt 50) {
    # Retention should have triggered aggressive 7-day cleanup
    Get-EventLog -LogName Application -Source 'CI-Retention' -Newest 1
}

Pass Criteria:

• Artifacts older than 30 days removed
• Recent artifacts preserved
• Aggressive (7-day) cleanup if free space < 50 GB
• Event log entry created for cleanup action

---

2.5 Snapshot Versioning

What changed: Snapshots named BaseClean_YYYYMMDD instead of static BaseClean to support rollback during template refresh.

Files affected:

• runner/config.yaml (new env var GITEA_CI_SNAPSHOT_NAME)
• scripts/Invoke-CIJob.ps1 (line 100: reads env, defaults to BaseClean)
• docs/BEST-PRACTICES.md (snapshot versioning strategy)

Test Steps:

# 1. Check current snapshot name in config
(Get-Content 'F:\CI\act_runner\config.yaml' | Select-String 'GITEA_CI_SNAPSHOT_NAME').Line
# Expected: Should show GITEA_CI_SNAPSHOT_NAME value or be empty (uses default)

# 2. Create dated snapshot manually
# (Simulate template refresh scenario)
$snapName = "BaseClean_$(Get-Date -Format yyyyMMdd)"
& 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' -T ws snapshot `
    'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' $snapName
# Expected: Success, snapshot created

# 3. Verify old and new snapshots coexist
& 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' -T ws listSnapshots `
    'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx'
# Expected: Output shows both BaseClean (old) and BaseClean_YYYYMMDD (new)

# 4. Update config to use new snapshot
# Edit F:\CI\act_runner\config.yaml:
# envs:
#   GITEA_CI_SNAPSHOT_NAME: BaseClean_20260510

# 5. Verify job uses new snapshot
# Start a fresh build and check phase logs for "Reverting to snapshot: BaseClean_20260510"

Pass Criteria:

• Multiple dated snapshots can exist simultaneously
• Jobs respect GITEA_CI_SNAPSHOT_NAME env var
• Default fallback to BaseClean if env var not set
• Rollback possible by reverting config to old snapshot name

---

Sprint 3 — Observability & Maintenance (§4.1/4.3/4.4)

4.1 JSONL Structured Logging

What changed: Phase transitions now emit JSONL events (one line per phase) in addition to text transcript, enabling machine parsing and future Loki/Grafana integration.

Files affected:

• scripts/Invoke-CIJob.ps1 (lines 178-187: phase event logging)

Test Steps:

# 1. Run a complete build and capture job ID
$jobId = 'test-jsonl-logging'
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-CIJob.ps1' `
    -RepositoryUrl 'http://10.10.20.11:3100/Simone/nsis-plugin-nsinnounp' `
    -JobId $jobId `
    -Branch main

# 2. Check for JSONL log file
$jsonLog = Get-ChildItem 'F:\CI\Logs\' -Filter "*${jobId}*.jsonl" | Select-Object -First 1
Test-Path $jsonLog.FullName
# Expected: True

# 3. Parse JSONL and validate structure
$events = Get-Content $jsonLog.FullName | ConvertFrom-Json | Select-Object -First 5
$events | Select ts, jobId, phase, status, data
# Expected: Each line is valid JSON with fields: ts (ISO), jobId, phase, status, data object

# 4. Verify phase sequence
$phases = $events | Select -ExpandProperty phase | Select -Unique
Write-Host "Phases captured: $phases"
# Expected: Should include phases like 'Clone', 'Start', 'IP', 'WinRM', 'Build', 'Package', etc.

Pass Criteria:

• JSONL file created alongside text transcript
• Each phase emits valid JSON event
• Timestamps are ISO format (parseable)
• Job ID correctly recorded in events
• Status values are consistent (success/failure/started)

---

4.3 Disk Space Monitoring & Alert

What changed: New Watch-DiskSpace.ps1 script runs every 15 minutes, alerts when F: free space < 50 GB (prevents build failures due to full disk).

Files affected:

• scripts/Watch-DiskSpace.ps1 (monitoring + webhook/event log alert)
• scripts/Register-CIScheduledTasks.ps1 (task registration: CI-DiskSpace every 15 min)

Test Steps:

# 1. Verify scheduled task
Get-ScheduledTask -TaskName 'CI-DiskSpace' | Select Name, State, NextRunTime
# Expected: Exists, State = Ready

# 2. Check current disk free space
$diskFree = (Get-PSDrive F).Free / 1GB
Write-Host "F: free space: ${diskFree} GB"

# 3. Run disk watch manually to see alert logic
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Watch-DiskSpace.ps1' -Verbose

# 4. Check for event log entry if space is low
Get-EventLog -LogName Application -Source 'CI-DiskSpace' -Newest 5 | Select TimeGenerated, EventID, Message

# 5. If webhook configured, verify it receives alert
# (Check Discord/Gitea webhook logs if configured)

Pass Criteria:

• Scheduled task runs without error
• Alert triggered if F: free < 50 GB
• Event log entry (ID 1003 for warning, 1004+ for error) created
• Webhook POST succeeds if configured
• No alert if F: free > 50 GB

---

4.4 Runbook Documentation

What changed: New docs/RUNBOOK.md with troubleshooting steps for common incidents.

Files affected:

• docs/RUNBOOK.md (incident playbooks)

Test Steps:

# 1. Verify file exists
Test-Path 'N:\Code\Workspace\Local-CI-CD-System\docs\RUNBOOK.md'
# Expected: True

# 2. Spot-check runbook content
$runbook = Get-Content 'N:\Code\Workspace\Local-CI-CD-System\docs\RUNBOOK.md' -Raw
# Should contain sections: runner offline, build phase failures, slow builds, corrupted VMX

# 3. Test one runbook procedure: "Runner offline in Gitea UI"
# Run the triage steps from runbook:
Get-Service act_runner | Select Status, StartType
Get-EventLog -LogName Application -Source 'act_runner' -Newest 5
# (Follow runbook steps to verify syntax/commands work)

# 4. Verify incident reference links
Get-ChildItem 'N:\Code\Workspace\Local-CI-CD-System\docs\' | Where Name -Match 'BEST-PRACTICES|OPTIMIZATION'
# Expected: Links in runbook resolve to actual docs

Pass Criteria:

• Runbook file exists and is readable
• All diagnostic commands execute without error
• At least 3 incident types documented with clear symptom→triage→fix flow
• External references (log paths, service names) accurate and tested

---

Sprint 4-6 — Quality, Reliability, Performance

1.4 IP Validation (Per-Octet Regex)

What changed: IP validation uses per-octet regex in all 4 relevant scripts instead of loose pattern.

Files affected:

• scripts/Invoke-RemoteBuild.ps1 (param validation)
• scripts/Get-BuildArtifacts.ps1 (param validation)
• scripts/Wait-VMReady.ps1 (param validation)
• scripts/Invoke-CIJob.ps1 (IP validation in phase 3)

Test Steps:

# 1. Test valid IP in script parameter binding
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Wait-VMReady.ps1' `
    -VMPath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
    -IPAddress '192.168.79.101' `
    -VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' `
    -TimeoutSeconds 5 `
    -SkipPing
# Expected: Script runs (may timeout OK since VM not actually running, but IP validation passes)

# 2. Test invalid IPs (should fail parameter validation)
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Wait-VMReady.ps1' `
    -VMPath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
    -IPAddress '999.999.999.999' `
    -VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe'
# Expected: Error message like "Cannot validate argument on parameter 'IPAddress'"

# 3. Test edge case IPs
@('0.0.0.0', '255.255.255.255', '192.168.1.1', '10.0.0.256') | ForEach-Object {
    try {
        & 'N:\Code\Workspace\Local-CI-CD-System\scripts\Wait-VMReady.ps1' `
            -VMPath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
            -IPAddress $_ `
            -VmrunPath 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe' `
            -TimeoutSeconds 1 `
            -SkipPing -ErrorAction SilentlyContinue
        Write-Host "$_ : ACCEPTED"
    } catch {
        Write-Host "$_ : REJECTED"
    }
}
# Expected: 0.0.0.0/255.255.255.255/192.168.1.1 ACCEPTED (valid), 10.0.0.256 REJECTED

Pass Criteria:

• Valid IPs (0-255 per octet) accepted
• Invalid IPs (256+, non-numeric, malformed) rejected before script logic
• Error message clear ("Cannot validate argument")
• All 4 scripts enforce same pattern

---

3.1 Shared Cache (NuGet + Pip)

What changed: Host-side cache directories (F:\CI\Cache\NuGet, F:\CI\Cache\pip) mounted as VMware shared folders in template, available at \\vmware-host\Shared Folders\<cache>.

Files affected:

• scripts/Set-TemplateSharedFolders.ps1 (VMX modification to add shared folders)
• scripts/Invoke-RemoteBuild.ps1 (new -UseSharedCache switch, env var injection)

Test Steps:

# 1. Verify shared folder configuration exists in template VMX
$vmxPath = 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx'
$vmx = Get-Content $vmxPath
$vmx | Select-String 'sharedFolder' | Select -First 3
# Expected: Lines like "sharedFolder0 = ...", "sharedFolder1 = ..."

# 2. Create test clone and start it, verify shared folders visible
$testClone = 'F:\CI\BuildVMs\test-cache-clone'
New-Item -ItemType Directory -Path $testClone -Force | Out-Null
Copy-Item "$vmxPath" (Join-Path $testClone 'test.vmx') -Force

# Boot clone (or inspect VMX directly)
$vmx = Get-Content (Join-Path $testClone 'test.vmx')
$vmx | Select-String -Pattern 'sharedFolder\d+' | ForEach-Object {
    if ($_ -match 'nuget-cache|pip-cache') {
        Write-Host "[x] Cache folder found: $_"
    }
}
# Expected: Both nuget-cache and pip-cache in VMX

# 3. Run build with -UseSharedCache flag
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-RemoteBuild.ps1' `
    -IPAddress '192.168.79.101' `
    -Credential (Get-StoredCredential -Target 'BuildVMGuest') `
    -HostSourceDir 'C:\Users\user\AppData\Local\Temp\ci-src-<jobid>' `
    -UseSharedCache `
    -Verbose

# 4. Verify cache environment variables set inside VM
# (Check build logs for NUGET_PACKAGES and PIP_CACHE_DIR references)

Pass Criteria:

• VMX contains sharedFolder entries for NuGet and pip
• -UseSharedCache flag accepted without error
• Build logs show NUGET_PACKAGES/PIP_CACHE_DIR env vars set
• Cache directories persist across builds (not recreated)
• Performance improvement visible on second build (packages already cached)

---

3.6 Benchmark Baseline

What changed: New Measure-CIBenchmark.ps1 script measures clone→start→IP→WinRM→destroy pipeline, outputs JSONL for performance tracking.

Files affected:

• scripts/Measure-CIBenchmark.ps1 (benchmarking harness)

Test Steps:

# 1. Run baseline benchmark (5 iterations for quick turnaround)
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Measure-CIBenchmark.ps1' `
    -TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
    -SnapshotName 'BaseClean' `
    -CloneBaseDir 'F:\CI\BuildVMs\' `
    -Iterations 5 `
    -Verbose

# 2. Check output files
Test-Path 'F:\CI\Logs\benchmark.jsonl'
# Expected: True, file has 5 lines (one per iteration)

# 3. Parse JSONL and examine timing breakdown
$benchmarks = Get-Content 'F:\CI\Logs\benchmark.jsonl' | ConvertFrom-Json
$benchmarks | Select iteration, phase, duration_ms | Format-Table
# Expected: Phases (clone, start, ip, winrm, destroy) with millisecond durations

# 4. Calculate statistics
$times = $benchmarks | Where phase -eq 'clone' | Select -ExpandProperty duration_ms
$avg = ($times | Measure-Object -Average).Average
$min = ($times | Measure-Object -Minimum).Minimum
$max = ($times | Measure-Object -Maximum).Maximum
Write-Host "Clone phase: avg=${avg}ms, min=${min}ms, max=${max}ms"

Pass Criteria:

• Script completes N iterations without error
• JSONL output created with correct structure
• Five phases present (clone, start, ip, winrm, destroy)
• Durations reasonable (clone 1-5s, start 2-5s, IP <5s, WinRM 1-3s, destroy <2s)
• Variance visible across iterations (natural fluctuation)

---

5.2/5.3/5.4 Code Quality (Module, Uniform vmrun, PSScriptAnalyzer)

What changed:

• 5.2: Extract shared functions into _Common.psm1 module (New-CISessionOption, Resolve-VmrunPath, Invoke-Vmrun)
• 5.3: All scripts use Invoke-Vmrun wrapper with uniform error handling
• 5.4: PSScriptAnalyzer rules in PSScriptAnalyzerSettings.psd1

Files affected:

• scripts/_Common.psm1 (reusable functions)
• scripts/Invoke-RemoteBuild.ps1, Get-BuildArtifacts.ps1, New-BuildVM.ps1 (Import-Module + use _Common functions)
• PSScriptAnalyzerSettings.psd1 (linting rules)

Test Steps:

# 1. Verify module exports correct functions
Import-Module 'N:\Code\Workspace\Local-CI-CD-System\scripts\_Common.psm1' -Force
Get-Command -Module '_Common' | Select Name
# Expected: New-CISessionOption, Resolve-VmrunPath, Invoke-Vmrun

# 2. Test module functions
$sessionOpt = New-CISessionOption
$sessionOpt | Select SkipCACheck, SkipCNCheck, SkipRevocationCheck
# Expected: All True

# 3. Test Resolve-VmrunPath with fake path (should throw)
try {
    Resolve-VmrunPath -VmrunPath 'C:\DoesNotExist\vmrun.exe'
} catch {
    Write-Host "[x] Correctly threw: $_"
}

# 4. Test Invoke-Vmrun with fake vmrun.cmd
$fakeVmrun = 'fake-vmrun-test.cmd'
Set-Content $fakeVmrun '@echo off & exit /b 0'
$result = Invoke-Vmrun -VmrunPath $fakeVmrun -Operation 'test'
$result | Select ExitCode, Output
# Expected: ExitCode = 0

# 5. Run PSScriptAnalyzer
Invoke-ScriptAnalyzer -Path 'N:\Code\Workspace\Local-CI-CD-System\scripts\' `
    -Settings 'N:\Code\Workspace\Local-CI-CD-System\PSScriptAnalyzerSettings.psd1' |
    Select RuleName, Severity, Line | Format-Table
# Expected: Only approved suppressions (PSAvoidUsingWriteHost), no critical issues

# 6. Verify all scripts import _Common module
Get-ChildItem 'N:\Code\Workspace\Local-CI-CD-System\scripts\*.ps1' -Exclude '_Common.psm1' |
    Select-String 'Import-Module.*_Common' | Select Filename
# Expected: Invoke-RemoteBuild.ps1, Get-BuildArtifacts.ps1, New-BuildVM.ps1

Pass Criteria:

• _Common.psm1 exports 3 functions
• Functions work in isolation (parameters accepted, errors thrown appropriately)
• All production scripts import _Common
• PSScriptAnalyzer finds 0 critical violations
• Uniform vmrun invocation via Invoke-Vmrun (no direct &vmrun calls)

---

Sprint 7 — Testing (§5.1)

5.1 Pester Unit Tests (Fake vmrun, No Real VMs)

What changed: Comprehensive Pester v5 test suite covering core scripts using fake executables instead of real VMs (fast, reliable, no infrastructure overhead).

Files affected:

• tests/_Common.Tests.ps1 (3.1 module functions)
• tests/New-BuildVM.Tests.ps1 (clone creation, naming, error handling)
• tests/Remove-BuildVM.Tests.ps1 (clone destruction, -WhatIf support)
• tests/Wait-VMReady.Tests.ps1 (IP validation, timeout, vmrun missing)

Test Steps:

# 1. Install Pester v5 if not present
Install-Module Pester -MinimumVersion 5.0 -Force -Scope CurrentUser

# 2. Run all Pester tests with detailed output
Invoke-Pester -Path 'N:\Code\Workspace\Local-CI-CD-System\tests\' `
    -Output Detailed

# Expected output: 4 files, ~25 tests total, all passing

# 3. Examine individual test results
Invoke-Pester -Path 'N:\Code\Workspace\Local-CI-CD-System\tests\' `
    -Output Detailed | Select -ExpandProperty Tests |
    Select Describe, Name, Result | Format-Table

# 4. Verify fake vmrun.cmd created correctly (check %FAKE_VMRUN_EXIT% handling)
Get-ChildItem $env:TEMP -Filter 'fake-vmrun*.cmd' | Select Name, Length
# (These are cleaned up after tests, but if test fails, can verify content)

# 5. Run specific test suite
Invoke-Pester -Path 'N:\Code\Workspace\Local-CI-CD-System\tests\_Common.Tests.ps1' -Output Detailed
# Expected: 3 test groups (New-CISessionOption, Resolve-VmrunPath, Invoke-Vmrun), all passing

# 6. Test output capture (verify errors work)
Invoke-Pester -Path 'N:\Code\Workspace\Local-CI-CD-System\tests\' `
    -Output Detailed | Where Result -eq Failed
# Expected: 0 failed (or only expected failures documented)

Pass Criteria:

• All 4 test files execute without compilation errors
• Total passed: ≥20 tests
• Total failed: 0 (all pass)
• Test duration: <30 seconds (fast, no VM overhead)
• No temp files left behind after tests complete
• Fake vmrun behavior matches real vmrun for tested scenarios

---

§1.6 — Threat Model Documentation

What changed: New BEST-PRACTICES.md §2.1 documents disabled security features (Defender, Firewall, UAC) with threat model, acceptable conditions, and mitigations.

Files affected:

• docs/BEST-PRACTICES.md (§2.1 Threat Model — Disabled Security Features)

Test Steps:

# 1. Verify section exists and is readable
$bp = Get-Content 'N:\Code\Workspace\Local-CI-CD-System\docs\BEST-PRACTICES.md' -Raw
$bp | Select-String -Pattern '2\.1.*Threat Model' -Context 1, 50 | Select -First 1
# Expected: Found

# 2. Spot-check content structure
$sections = @('Current state', 'Acceptable', 'Breaking', 'Mitigations')
$sections | ForEach-Object {
    if ($bp -match $_) { Write-Host "[x] Section: $_" }
    else { Write-Host "✗ Missing: $_" }
}

# 3. Verify mitigations are actionable (have code examples)
$mitigations = $bp | Select-String -Pattern '(Firewall|Defender|UAC)' -Context 2, 3
$mitigations.Matches.Count
# Expected: At least 3+ sections with mitigations

# 4. Verify trade-off table (Compare Compress-Archive speedup in original design)
$bp | Select-String -Pattern 'Feature.*Disabled.*Why.*Cost' -Context 0, 10
# Expected: Structured comparison found

Pass Criteria:

• §2.1 section present in BEST-PRACTICES.md
• Explains why features disabled (AV overhead, firewall complexity, UAC elevation)
• Documents acceptable conditions (isolated lab, trusted code)
• Documents breaking conditions (shared host, LAN exposure)
• Provides step-by-step mitigations (code examples) if conditions change
• References existing features (HGFS, WinRM HTTPS) that work with protections

---

§3.2 — 7-Zip Compression Acceleration

What changed: Replaced single-threaded Compress-Archive with multi-threaded 7-Zip (3 call sites: host source compression, custom build artifacts, dotnet build output). Falls back to Compress-Archive if 7-Zip not found.

Files affected:

• scripts/Invoke-RemoteBuild.ps1 (3 replacements + new Compress-BuildArtifact helper)

Test Steps:

# 1. Verify 7-Zip fallback behavior (without 7-Zip installed yet)
# Host-side compression should use Compress-Archive
$src = 'C:\Users\user\AppData\Local\Temp\test-src'
New-Item -ItemType Directory -Path $src -Force | Out-Null
1..100 | ForEach-Object { Set-Content (Join-Path $src "file-$_.txt") 'test' }

$zip = 'C:\Users\user\AppData\Local\Temp\test-no7z.zip'
$sw = [System.Diagnostics.Stopwatch]::StartNew()

# Simulate what Compress-BuildArtifact does (without 7-Zip)
Compress-Archive -Path "$src\*" -DestinationPath $zip -CompressionLevel Fastest -Force

$sw.Stop()
Write-Host "Compress-Archive on 100 files: $($sw.ElapsedMilliseconds) ms"
# Expected: Takes ~500-2000 ms (single-threaded)

# 2. Once 7-Zip is installed in template (§6.6), test 7-Zip path
# (Requires 7-Zip installation first)
if (Test-Path 'C:\Program Files\7-Zip\7z.exe') {
    $sw = [System.Diagnostics.Stopwatch]::StartNew()
    & 'C:\Program Files\7-Zip\7z.exe' a -mmt=on -mx1 'C:\Users\user\AppData\Local\Temp\test-7z.7z' "$src\*"
    $sw.Stop()
    Write-Host "7-Zip on 100 files: $($sw.ElapsedMilliseconds) ms"
    # Expected: Faster than Compress-Archive (multi-threaded, should be 50-70% reduction)
}

# 3. Test fallback logic in actual script
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-RemoteBuild.ps1' `
    -IPAddress '192.168.79.101' `
    -Credential (Get-StoredCredential -Target 'BuildVMGuest') `
    -HostSourceDir 'C:\Users\user\AppData\Local\Temp\ci-src-<jobid>' `
    -Verbose 2>&1 | Select-String -Pattern 'Compress-BuildArtifact|7-Zip|Compress-Archive'
# Expected: See which path was taken (7-Zip if installed, Compress-Archive if not)

# 4. Verify no errors during compression (both paths)
# Run full build and check:
# - No compression errors in logs
# - Artifact zip file created
# - Artifact zip can be extracted (not corrupted)

Pass Criteria:

• 7-Zip code path detects C:\Program Files\7-Zip\7z.exe correctly
• Falls back to Compress-Archive if 7-Zip missing
• Both compression paths produce valid ZIP files
• Extraction succeeds without corruption
• Build logs clearly show which compression method used
• Multi-threaded 7-Zip is faster than single-threaded Compress-Archive (when installed)
• No breaking changes (fallback ensures zero downtime before 7-Zip added to template)

---

§6.6 Tier-1 Toolchain (Git + 7-Zip)

What changed: Step 11 added to Install-CIToolchain-WinBuild2025.ps1 to install Git for Windows and 7-Zip in the template VM. Critical for §3.3 (In-VM Git clone) and cross-platform builds.

Files affected:

• template/Install-CIToolchain-WinBuild2025.ps1 (Step 11: Git + 7-Zip installation)
• runner/config.yaml (updated GITEA_CI_TEMPLATE_PATH)

Test Steps (run after template is provisioned with Install-CIToolchain-WinBuild2025.ps1):

# 1. Verify Git is installed and in PATH
git --version
# Expected: git version 2.54.0.windows.1 (or later)

# 2. Verify Git can clone
git clone --depth 1 https://github.com/git-for-windows/git.git C:\Temp\git-test
# Expected: repo cloned successfully (only test if internet available)
Remove-Item C:\Temp\git-test -Recurse -Force -ErrorAction SilentlyContinue

# 3. Verify 7-Zip is installed
7z --version
# Expected: 7-Zip (or "7z.exe" path listed)

# 4. Verify 7-Zip can compress/decompress
$testFile = 'C:\Temp\test-7z.txt'
Set-Content $testFile 'test data'
7z a 'C:\Temp\test.7z' $testFile
7z x 'C:\Temp\test.7z' -o'C:\Temp\test-extract' -y
Test-Path 'C:\Temp\test-extract\test-7z.txt'
# Expected: All operations succeed, file extracted correctly

# 5. Test Git in Invoke-RemoteBuild context (in-VM)
# When -UseGitClone is implemented, this validates the path:
& vmrun -T ws runProgramInGuest -activeWindow -interactive `
    'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
    C:\Windows\System32\cmd.exe "/c git --version"
# Expected: git version displayed (validates in-guest git availability)

Pass Criteria:

• Step 11 executes without errors in Install-CIToolchain-WinBuild2025.ps1
• Git command available post-setup (machine PATH includes Git)
• 7-Zip command available post-setup (C:\Program Files\7-Zip\7z.exe)
• Final pre-snapshot gate includes git + 7z validation
• Both tools survive template snapshot + linked clone cycle
• In-VM git clone tested end-to-end with -UseGitClone flag (§3.3 e2e-010 PASS 2026-05-10 — 34s / 25.7% more speed vs baseline)

---

Security Items (Deferred)

The following security tasks are deferred to "Home Lab" section due to being excessive for isolated lab environment. Rifiori if conditions change (shared host, untrusted code, LAN exposure).

• §1.2: TrustedHosts audit (currently *, should be 192.168.79.*)
• §1.3: SHA256 hash pinning (structure added, values placeholder)
• §1.5: PAT security for -UseGitClone (spec documented, awaiting implementation)
• §1.7: Quarterly password rotation
• §1.8: Get-StoredCredential verification in all scripts

See TODO.md "Deferred (Home Lab)" section for details and re-enablement criteria.

---

Test Execution Summary

# Run this to validate all features end-to-end
$testGroups = @(
    @{Name='IP Allocation'; Cmd='Write-Host "Check F:\CI\State\ip-leases after parallel builds"'},
    @{Name='Orphaned Cleanup'; Cmd='& scripts\Cleanup-OrphanedBuildVMs.ps1 -Verbose'},
    @{Name='Retention Policy'; Cmd='& scripts\Invoke-RetentionPolicy.ps1 -Verbose'},
    @{Name='Disk Watch'; Cmd='& scripts\Watch-DiskSpace.ps1 -Verbose'},
    @{Name='Pester Tests'; Cmd='Invoke-Pester tests\ -Output Detailed'},
    @{Name='Compression'; Cmd='Invoke-Pester tests\ -Output Detailed'},
)

$testGroups | ForEach-Object {
    Write-Host "━━━ Test: $($_.Name) ━━━"
    try {
        Invoke-Expression $_.Cmd
        Write-Host "[x] PASSED" -ForegroundColor Green
    } catch {
        Write-Host "✗ FAILED: $_" -ForegroundColor Red
    }
}

---

Success Criteria (All Must Pass)

Last run: 2026-05-10. Items marked [skip] require real VMware infrastructure and are deferred.

• All Pester tests pass (§5.1) — 22/22 in 12.9s, 0 failed
• IP validation rejects 999.999.999.999, accepts valid IPs — ParameterBindingValidationError confirmed
• [skip] Benchmark baseline captures all 5 phases with reasonable durations — needs real VMs (§3.6)
• Disk watch reports free space correctly, exits 0 when above threshold — 981.9 GB free on F:
• Artifact retention removes files > 30 days old — 31-day-old dir removed, recent preserved
• Orphaned VM cleanup removes stale clones, preserves recent ones — 5h-old orphan removed
• [skip] Snapshot versioning allows multiple dated snapshots — needs real VMs (§2.5)
• [skip] JSONL logs created and parseable — needs real build job (§4.1)
• Runbook procedures execute without syntax errors — RUNBOOK.md verified readable
• 7-Zip compression: fallback to Compress-Archive confirmed; in-VM path deferred to §6.6 e2e
• _Common.psm1 imported by all relevant scripts — New-BuildVM, Invoke-RemoteBuild, Get-BuildArtifacts, Measure-CIBenchmark
• PSScriptAnalyzer finds no critical violations — 0 Error/ParseError (220 Warnings only)
• BEST-PRACTICES.md §2.1 documents threat model clearly — file exists, contains threat + security sections
• Git for Windows (v2.54.0+) installed in template, in PATH (§6.6)
• 7-Zip (v26.01+) installed in template, in PATH (§6.6)
• Final pre-snapshot gate validates git + 7z availability (§6.6)

---

Regression Testing

After validating new features, run one full end-to-end build to ensure nothing broke:

# Full e2e validation
& 'N:\Code\Workspace\Local-CI-CD-System\scripts\Invoke-CIJob.ps1' `
    -RepositoryUrl 'http://10.10.20.11:3100/Simone/nsis-plugin-nsinnounp' `
    -JobId 'e2e-test-post-v1.3' `
    -Branch main `
    -Verbose

# Expected: Build completes successfully, artifact created, JSONL log present, no errors