Simone
210b4f5f16
Postmortem of the Linux template deploy/prepare 'no IP' investigation: separates the three distinct causes (apt index stale from --skip-update, deploy --force snapshot-chain, wedged host vmnet-dhcpd) and documents the recovery (restart vmware-networks). Changelog entry summarising the toolchain, xvfb, --force cleanup and cloud-init seal changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
176 lines
9.0 KiB
Markdown
176 lines
9.0 KiB
Markdown
# CHANGELOG — Local CI/CD System
|
|
|
|
Completed sprint work, in reverse-chronological order.
|
|
Open backlog items remain in [TODO.md](../TODO.md).
|
|
|
|
---
|
|
|
|
## 2026-06-06 — Linux template: GUI toolchain, headless builds, deploy hardening
|
|
|
|
- **Toolchain** (`Install-CIToolchain-Linux2404.sh`): new Step 4a installs
|
|
`python3-gi gir1.2-gtk-4.0 xvfb` (headless PyGObject/GTK4 GUI tests). Step 1
|
|
now always runs `apt-get update`; `--skip-update` only skips the full upgrade
|
|
(previously it skipped the index refresh too, so newly-added packages could
|
|
not be located).
|
|
- **`build run` / `job`**: new `-Xvfb` / `--xvfb` switch (Linux only) runs the
|
|
build under `xvfb-run` so GUI toolkits get a `$DISPLAY`. Ignored for Windows.
|
|
See [WORKFLOW-AUTHORING.md](WORKFLOW-AUTHORING.md) §5/§8.2.
|
|
- **`deploy-linux --force`**: now clears stale snapshot-chain artifacts
|
|
(`-NNNNNN.vmdk`, `.vmsn`, `.vmsd`, `.lck`) before the Step 4 resize, fixing
|
|
*"disk is part of a snapshot chain"*.
|
|
- **`prepare-linux`**: Step 7d seals cloud-init (drops the stale
|
|
`50-cloud-init.yaml`, keeps the MAC-agnostic `99-ci-dhcp-allnics`, disables
|
|
cloud-init) so it never re-runs on linked clones; deploy user-data sets
|
|
`package_update/upgrade: false` and the Step 7 IP-detect timeout is raised to
|
|
600 s.
|
|
- **Postmortem** [postmortem-2026-06-06-linux-template-no-ip.md](postmortem-2026-06-06-linux-template-no-ip.md):
|
|
intermittent deploy/prepare "no IP" traced to a wedged host `vmnet-dhcpd`;
|
|
recovery is `vmware-networks --stop/--start`.
|
|
|
|
---
|
|
|
|
## 2026-05-13 — Sprint 16: Medium/Low hardening (final-master-plan)
|
|
|
|
- **Watch-DiskSpace.ps1**: Event Log source renamed `CI-DiskAlert` → `CI-DiskSpaceAlert`
|
|
(aligned with task name registered by `Register-CIScheduledTasks.ps1`).
|
|
- **Watch-RunnerHealth.ps1**: maintenance flag support — if `F:\CI\State\runner-maintenance.flag`
|
|
exists, restart logic is skipped entirely.
|
|
- **Invoke-CIJob.ps1**:
|
|
- New `WebhookUrl` parameter (default `''`).
|
|
- `TemplatePath` sanity check: `Write-Warning` if not under `F:\CI\Templates\`.
|
|
- 90-minute duration warning: `Start-Job` fires `[WARNING]` webhook after 5400 s; cancelled
|
|
in `finally` if the build completes first.
|
|
- **BEST-PRACTICES.md** section 11: VMware HGFS shared folder write semantics, cache-poisoning
|
|
risk model, safe-use rules for `UseSharedCache`, and cache invalidation procedure.
|
|
- **Validate-HostState.ps1** (new): checks SSH key ACLs, Credential Manager target
|
|
accessibility, CI directory structure, and vmrun.exe availability.
|
|
- **Validate-SetupState.ps1** docblock: `HTTP/Basic` → `HTTPS/Basic, port 5986`.
|
|
- **TEST-PLAN-v1.3-to-HEAD.md** §3.6: benchmark JSONL field names corrected to match actual
|
|
script output (`cloneSec`, `startSec`, `ipSec`, `winrmSec`, `destroySec`, `totalBootSec`).
|
|
- **CI-FLOW.md** failure table: "collects partial artifacts" replaced with accurate description
|
|
of `Get-GuestDiagnostics` post-failure collection.
|
|
- **final-master-plan.md**: all Medium and most Low items marked done.
|
|
|
|
---
|
|
|
|
## 2026-05-12 — Sprint 15b: Hardcoded path fix
|
|
|
|
- **action.yml** + **runner/config.yaml**: replaced hardcoded `N:\Code\...` script path with
|
|
`GITEA_CI_SCRIPT_ROOT` env var injected by the runner. act_runner restarted with labels
|
|
`windows-build:host dotnet:host msbuild:host linux-build:host`.
|
|
|
|
---
|
|
|
|
## 2026-05-11 — Sprint 15: Rename Setup-* → Install-CIToolchain-*
|
|
|
|
Renamed three guest-side provisioning scripts and updated all references across 14 files:
|
|
|
|
| Old name | New name |
|
|
| ---------------------------------------- | ---------------------------------------------- |
|
|
| `template/Setup-WinBuild2025.ps1` | `template/Install-CIToolchain-WinBuild2025.ps1` |
|
|
| `template/Setup-WinBuild2022.ps1` | `template/Install-CIToolchain-WinBuild2022.ps1` |
|
|
| `template/Setup-LinuxBuild2404.sh` | `template/Install-CIToolchain-Linux2404.sh` |
|
|
|
|
References updated in: Prepare-WinBuild2025/2022.ps1, Prepare-LinuxBuild2404.ps1,
|
|
Deploy-WinBuild2025/2022.ps1, Deploy-LinuxBuild2404.ps1, README.md, TODO.md,
|
|
docs/WINDOWS-TEMPLATE-SETUP.md, docs/LINUX-TEMPLATE-SETUP.md,
|
|
docs/TEST-PLAN-v1.3-to-HEAD.md, gitea/actions/local-ci-build/action.yml.
|
|
|
|
---
|
|
|
|
## 2026-05-11 — Sprint 14: §6.4 Build matrix + §6.5 Secret injection
|
|
|
|
**§6.4** — `build-ns7zip.yml` rewritten with matrix strategy `target: [windows, linux]`.
|
|
Separate artifact dirs; `fail-fast: false`; composite action used for both targets.
|
|
|
|
**§6.5** — `ExtraGuestEnv [hashtable]` added to `Invoke-CIJob.ps1` and `Invoke-RemoteBuild.ps1`:
|
|
- Windows: `[System.Environment]::SetEnvironmentVariable` in session ScriptBlock.
|
|
- Linux: `export KEY='val'; ` prefix before build command.
|
|
- `action.yml`: `extra-guest-env-json` input (JSON object), never logged.
|
|
|
|
---
|
|
|
|
## 2026-05-11 — Sprint 13: §6.2 Composite Action
|
|
|
|
**action.yml** (`gitea/actions/local-ci-build/action.yml`) fully implemented.
|
|
Inputs: `build-command`, `artifact-source`, `submodules`, `guest-os`, `use-git-clone`,
|
|
`configuration`, `template-path`, `snapshot-name`, `artifact-name`, `artifact-retention-days`,
|
|
`extra-guest-env-json`, `use-shared-cache`, `job-id-suffix`, `repo-url`, `webhook-url`.
|
|
Outputs: `artifact-path`, `artifact-name`.
|
|
Artifact upload included; no orchestration logic required in calling repos.
|
|
|
|
---
|
|
|
|
## 2026-05-10 — Sprint 12: §6.1 Linux Build VM
|
|
|
|
End-to-end Linux build pipeline implemented:
|
|
- `Deploy-LinuxBuild2404.ps1`: Ubuntu 24.04 cloud image deployment via VMware vmrun.
|
|
- `Prepare-LinuxBuild2404.ps1`: SSH key injection, CI dirs, IP reporting service.
|
|
- `Install-CIToolchain-Linux2404.sh`: GCC, G++, Clang, CMake, Ninja, Python 3, Git, 7-Zip.
|
|
- `scripts/_Transport.psm1`: `Invoke-SshCommand`, `Copy-SshItem`, `Test-SshReady` with
|
|
`StrictHostKeyChecking` and `KnownHostsFile` parametrisation.
|
|
- E2E verified: `nsis7z.dll` 2272 KB on parity with Windows build.
|
|
|
|
---
|
|
|
|
## 2026-05-10 — Sprint 11: §6.6 Tier-2 Toolchain (WinBuild2025)
|
|
|
|
Tier-1 (Step 11) and Tier-2 (Step 12) added to `Install-CIToolchain-WinBuild2025.ps1`:
|
|
|
|
| Tool | Version |
|
|
| ---------------- | ---------------- |
|
|
| Git for Windows | 2.54.0.windows.1 |
|
|
| 7-Zip | 26.01 |
|
|
| PowerShell 7 | 7.6.1 |
|
|
| NSIS | 3.12 |
|
|
| gh CLI | 2.92.0 |
|
|
| CMake | 3.31.6 |
|
|
| Node.js LTS | 22.14.0 |
|
|
| WiX Toolset | 4.0.5 |
|
|
| Sysinternals | latest |
|
|
| vcpkg | HEAD |
|
|
|
|
All tools hash-pinned (SHA256), path-extended machine-wide, validated via `where.exe` in
|
|
`Assert-Step` final gate.
|
|
|
|
---
|
|
|
|
## 2026-05-10 — Sprint 10: §7 Refactor Deploy ↔ Setup
|
|
|
|
Separated OS hardening (Deploy responsibility) from CI toolchain setup (Install-CIToolchain
|
|
responsibility). Setup Steps 1/3/4b/5b/5c converted to assertion-only (`Assert-Step`).
|
|
|
|
New scripts: `Validate-DeployState.ps1`, `Validate-SetupState.ps1`.
|
|
WU lifecycle strategy B: Deploy sets GPO `NoAutoUpdate`; Install-CIToolchain runs WU then
|
|
disables wuauserv/UsoSvc permanently.
|
|
|
|
---
|
|
|
|
## 2026-05-10 — Sprint 9: §3.3 In-VM Git Clone
|
|
|
|
`-UseGitClone` switch added to `Invoke-CIJob.ps1` and `Invoke-RemoteBuild.ps1`.
|
|
- Skips host git clone + zip + WinRM transfer overhead.
|
|
- PAT injected at runtime, never persisted in snapshot.
|
|
- E2E result: 34 s saving, 25.7% reduction vs baseline on nsis-plugin-ns7zip.
|
|
|
|
---
|
|
|
|
## 2026-05-10 — Sprints 1-8: Core infrastructure
|
|
|
|
| Sprint | Area | Key deliverable |
|
|
| ------ | ----------------------------- | ------------------------------------------------------ |
|
|
| 1 | §1.1 WinRM HTTP → HTTPS | Self-signed cert, port 5986, AllowUnencrypted=false |
|
|
| 2 | §1.4 IP validation | Per-octet regex extracted to `_Common.psm1` |
|
|
| 3 | §1.6 Threat model | `BEST-PRACTICES.md` §2.1 Threat Model documented |
|
|
| 4 | §2.1 IP race / lease | File-based IP allocator in `Invoke-CIJob.ps1` |
|
|
| 5 | §2.2-2.7 Reliability | Orphan cleanup, retention, WhatIf, snapshot versioning, backup, health watch |
|
|
| 6 | §3.1 NuGet/pip shared cache | `Set-TemplateSharedFolders.ps1`, `UseSharedCache` switch |
|
|
| 7 | §3.2 7-Zip compression | 7-Zip replaces Compress-Archive in `Invoke-RemoteBuild.ps1` |
|
|
| 8 | §3.6 Benchmark baseline | `Measure-CIBenchmark.ps1` — 5-phase JSONL output |
|
|
| - | §4.1 Structured logging | `Write-JobEvent` + `invoke-ci.jsonl` in `Invoke-CIJob.ps1` |
|
|
| - | §4.3 Disk alert | `Watch-DiskSpace.ps1` + scheduled task |
|
|
| - | §4.4 Runbook | `docs/RUNBOOK.md` — 5 incident categories |
|
|
| - | §5.1 Pester tests | `tests/` — 4 test files, fake vmrun harness |
|
|
| - | §5.2 `_Common.psm1` | `New-CISessionOption`, `Resolve-VmrunPath`, `Invoke-Vmrun` |
|
|
| - | §5.4 PSScriptAnalyzer | `PSScriptAnalyzerSettings.psd1` |
|