fix(benchmark): adapt Measure-CIBenchmark to Linux host

Measure-CIBenchmark.ps1:
- Default params now Linux-aware via $IsWindows conditional
  (TemplatePath, CloneBaseDir, VmrunPath, OutputDir)
- Add -GuestOS Auto/Windows/Linux param; Auto reads VMX guestOS field
  (ubuntu-* → Linux, else Windows)
- Replace Test-NetConnection (Windows-only) with Test-TcpPort helper
  using raw TcpClient async connect — works on PS5.1 and PS7/Linux
- Probe SSH/22 for Linux guests, WinRM/5986 for Windows guests
- Rename winrm→ready throughout ($t, JSONL field readySec, column header)
- Add guestOS field to JSONL output

_Common.psm1 / Invoke-VmrunBounded:
- Replace taskkill /F /T with platform-aware kill:
  Windows → taskkill; Linux → Process.Kill(entireProcessTree) (.NET 5+)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-23 23:57:32 +02:00
parent c1147c5b19
commit 4f9149f7e2
2 changed files with 106 additions and 47 deletions
+100 -46
View File
@@ -1,63 +1,83 @@
#Requires -Version 5.1 #Requires -Version 5.1
<# <#
.SYNOPSIS .SYNOPSIS
Measures CI infrastructure phase timings: clone, start, IP acquire, WinRM ready, destroy. Measures CI infrastructure phase timings: clone, start, IP acquire, transport ready, destroy.
.DESCRIPTION .DESCRIPTION
Creates one or more ephemeral VMs from the template, times each phase, destroys Creates one or more ephemeral VMs from the template, times each phase, destroys
them, then prints a summary table. Results are appended to F:\CI\Logs\benchmark.jsonl them, then prints a summary table. Results are appended to benchmark.jsonl
for long-term trend tracking (compare before/after §3.1 cache, §3.2 7-Zip, etc.). for long-term trend tracking.
Phases measured: Phases measured:
clone — vmrun clone (linked clone from snapshot) clone — vmrun clone (linked clone from snapshot)
start — vmrun start (until command returns) start — vmrun start (until command returns)
ip — vmrun getGuestIPAddress (polling until IP appears) ip — vmrun getGuestIPAddress (polling until IP appears)
winrm — TCP 5986 reachable (polling) ready — TCP port reachable: 5986 (WinRM, Windows) or 22 (SSH, Linux)
destroy — stop + deleteVM + dir removal destroy — stop + deleteVM + dir removal
No actual build is run. Phase 5 build timings come from invoke-ci.jsonl logs No actual build is run.
produced by Invoke-CIJob.ps1.
.PARAMETER TemplatePath .PARAMETER TemplatePath
Full path to the template VM's .vmx file. Full path to the template VM's .vmx file.
Default: F:\CI\Templates\WinBuild2025\WinBuild2025.vmx
.PARAMETER SnapshotName .PARAMETER SnapshotName
Snapshot name to clone from. Default: BaseClean Snapshot name to clone from. Default: BaseClean
.PARAMETER CloneBaseDir .PARAMETER CloneBaseDir
Directory for ephemeral clone folders. Default: F:\CI\BuildVMs Directory for ephemeral clone folders.
.PARAMETER VmrunPath .PARAMETER VmrunPath
Path to vmrun.exe. Path to vmrun executable.
Default: C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe
.PARAMETER GuestOS
'Windows', 'Linux', or 'Auto' (default). Auto reads guestOS from the VMX:
ubuntu-* → Linux (probe SSH/22); anything else → Windows (probe WinRM/5986).
.PARAMETER Iterations .PARAMETER Iterations
Number of clone→ready→destroy cycles to run. Multiple iterations reduce timing Number of clone→ready→destroy cycles to run. Default: 1
variance from cold-cache effects. Default: 1
.PARAMETER TimeoutSeconds .PARAMETER TimeoutSeconds
Maximum seconds to wait for IP and WinRM per iteration. Default: 300 Maximum seconds to wait for IP and transport per iteration. Default: 300
.PARAMETER OutputDir .PARAMETER OutputDir
Directory where benchmark.jsonl is appended. Default: F:\CI\Logs Directory where benchmark.jsonl is appended.
.EXAMPLE .EXAMPLE
# Single baseline run # Single baseline run (Linux host, Windows template)
.\Measure-CIBenchmark.ps1 .\Measure-CIBenchmark.ps1
# 3 iterations for average (first may be slow due to cold host disk cache) # 4 iterations, Linux guest template
.\Measure-CIBenchmark.ps1 -Iterations 3 .\Measure-CIBenchmark.ps1 -TemplatePath /var/lib/ci/templates/LinuxBuild2404/LinuxBuild2404.vmx `
-SnapshotName BaseClean-Linux -Iterations 4
# Custom snapshot # Windows host
.\Measure-CIBenchmark.ps1 -SnapshotName BaseClean_20260510 .\Measure-CIBenchmark.ps1 -TemplatePath 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx' `
-Iterations 3
#> #>
[CmdletBinding()] [CmdletBinding()]
param( param(
[string] $TemplatePath = 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx', [string] $TemplatePath = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
[string] $SnapshotName = 'BaseClean', '/var/lib/ci/templates/WinBuild2025/WinBuild2025.vmx'
[string] $CloneBaseDir = 'F:\CI\BuildVMs', } else {
[string] $VmrunPath = 'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe', 'F:\CI\Templates\WinBuild2025\WinBuild2025.vmx'
}),
[string] $SnapshotName = 'BaseClean',
[string] $CloneBaseDir = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/var/lib/ci/build-vms/'
} else {
'F:\CI\BuildVMs'
}),
[string] $VmrunPath = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/usr/bin/vmrun'
} else {
'C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe'
}),
[ValidateSet('Auto', 'Windows', 'Linux')]
[string] $GuestOS = 'Auto',
[ValidateRange(1, 10)] [ValidateRange(1, 10)]
[int] $Iterations = 1, [int] $Iterations = 1,
@@ -65,16 +85,49 @@ param(
[ValidateRange(60, 600)] [ValidateRange(60, 600)]
[int] $TimeoutSeconds = 300, [int] $TimeoutSeconds = 300,
[string] $OutputDir = 'F:\CI\Logs' [string] $OutputDir = $(if ($null -ne $IsWindows -and $IsWindows -eq $false) {
'/var/lib/ci/logs/'
} else {
'F:\CI\Logs'
})
) )
Set-StrictMode -Version Latest Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue' # suppress Write-Progress (Test-NetConnection, IWR, etc.) $ProgressPreference = 'SilentlyContinue'
Import-Module (Join-Path $PSScriptRoot '_Common.psm1') -Force Import-Module (Join-Path $PSScriptRoot '_Common.psm1') -Force
Resolve-VmrunPath -VmrunPath $VmrunPath | Out-Null Resolve-VmrunPath -VmrunPath $VmrunPath | Out-Null
# Cross-platform TCP port probe (replaces Test-NetConnection which is Windows-only).
function Test-TcpPort {
param([string]$ComputerName, [int]$Port, [int]$TimeoutMs = 3000)
try {
$tcp = [System.Net.Sockets.TcpClient]::new()
$ar = $tcp.BeginConnect($ComputerName, $Port, $null, $null)
$ok = $ar.AsyncWaitHandle.WaitOne($TimeoutMs, $false)
$connected = $ok -and $tcp.Connected
$tcp.Close()
return $connected
} catch {
return $false
}
}
# Resolve effective GuestOS from VMX when Auto.
$resolvedGuestOS = $GuestOS
if ($resolvedGuestOS -eq 'Auto') {
$vmxRaw = Get-Content -LiteralPath $TemplatePath -Raw -ErrorAction SilentlyContinue
if ($vmxRaw -match 'guestOS\s*=\s*"([^"]+)"' -and $Matches[1] -like 'ubuntu*') {
$resolvedGuestOS = 'Linux'
} else {
$resolvedGuestOS = 'Windows'
}
Write-Host "[bench] GuestOS auto-detected: $resolvedGuestOS"
}
$transportPort = if ($resolvedGuestOS -eq 'Linux') { 22 } else { 5986 }
$transportLabel = if ($resolvedGuestOS -eq 'Linux') { 'SSH/22' } else { 'WinRM/5986' }
if (-not (Test-Path $TemplatePath -PathType Leaf)) { if (-not (Test-Path $TemplatePath -PathType Leaf)) {
throw "Template VMX not found: $TemplatePath" throw "Template VMX not found: $TemplatePath"
} }
@@ -102,7 +155,7 @@ for ($i = 1; $i -le $Iterations; $i++) {
clone = $null clone = $null
start = $null start = $null
ip = $null ip = $null
winrm = $null ready = $null
destroy = $null destroy = $null
deltaKB = $null deltaKB = $null
vmIP = $null vmIP = $null
@@ -151,21 +204,21 @@ for ($i = 1; $i -le $Iterations; $i++) {
$t.vmIP = $vmIP $t.vmIP = $vmIP
Write-Host "[bench] ip ($vmIP): $($t.ip)s" Write-Host "[bench] ip ($vmIP): $($t.ip)s"
# ── Phase: WinRM ready (poll TCP 5986) ──────────────────────────────── # ── Phase: transport ready (SSH/22 for Linux, WinRM/5986 for Windows) ──
Write-Host "[bench] Waiting for WinRM (TCP 5986)..." Write-Host "[bench] Waiting for $transportLabel..."
$sw.Restart() $sw.Restart()
$deadline = (Get-Date).AddSeconds($TimeoutSeconds) $deadline = (Get-Date).AddSeconds($TimeoutSeconds)
$winrmUp = $false $transportUp = $false
while ((Get-Date) -lt $deadline) { while ((Get-Date) -lt $deadline) {
$winrmUp = Test-NetConnection -ComputerName $vmIP -Port 5986 ` $transportUp = Test-TcpPort -ComputerName $vmIP -Port $transportPort
-InformationLevel Quiet -WarningAction SilentlyContinue ` if ($transportUp) { break }
-ErrorAction SilentlyContinue
if ($winrmUp) { break }
Start-Sleep -Seconds 3 Start-Sleep -Seconds 3
} }
if (-not $winrmUp) { throw "Timed out waiting for WinRM on $vmIP`:5986 after $TimeoutSeconds s" } if (-not $transportUp) {
$t.winrm = [math]::Round($sw.Elapsed.TotalSeconds, 2) throw "Timed out waiting for $transportLabel on $vmIP after $TimeoutSeconds s"
Write-Host "[bench] winrm: $($t.winrm)s" }
$t.ready = [math]::Round($sw.Elapsed.TotalSeconds, 2)
Write-Host "[bench] ready ($transportLabel): $($t.ready)s"
} }
catch { catch {
$t.error = "$_" $t.error = "$_"
@@ -186,8 +239,8 @@ for ($i = 1; $i -le $Iterations; $i++) {
} }
} }
$totalBootSec = if ($t.ip -and $t.winrm) { $totalBootSec = if ($t.ip -and $t.ready) {
[math]::Round($t.clone + $t.start + $t.ip + $t.winrm, 2) [math]::Round($t.clone + $t.start + $t.ip + $t.ready, 2)
} else { $null } } else { $null }
$result = [pscustomobject]@{ $result = [pscustomobject]@{
@@ -196,10 +249,11 @@ for ($i = 1; $i -le $Iterations; $i++) {
iteration = $i iteration = $i
template = Split-Path $TemplatePath -Leaf template = Split-Path $TemplatePath -Leaf
snapshot = $SnapshotName snapshot = $SnapshotName
guestOS = $resolvedGuestOS
cloneSec = $t.clone cloneSec = $t.clone
startSec = $t.start startSec = $t.start
ipSec = $t.ip ipSec = $t.ip
winrmSec = $t.winrm readySec = $t.ready
destroySec = $t.destroy destroySec = $t.destroy
totalBootSec = $totalBootSec totalBootSec = $totalBootSec
deltaKB = $t.deltaKB deltaKB = $t.deltaKB
@@ -226,7 +280,7 @@ $results | Format-Table @(
@{ L = 'Clone(s)'; E = { $_.cloneSec } } @{ L = 'Clone(s)'; E = { $_.cloneSec } }
@{ L = 'Start(s)'; E = { $_.startSec } } @{ L = 'Start(s)'; E = { $_.startSec } }
@{ L = 'IP(s)'; E = { $_.ipSec } } @{ L = 'IP(s)'; E = { $_.ipSec } }
@{ L = 'WinRM(s)'; E = { $_.winrmSec } } @{ L = 'Ready(s)'; E = { $_.readySec } }
@{ L = 'Destroy(s)'; E = { $_.destroySec } } @{ L = 'Destroy(s)'; E = { $_.destroySec } }
@{ L = 'Boot total'; E = { $_.totalBootSec } } @{ L = 'Boot total'; E = { $_.totalBootSec } }
@{ L = 'Delta(KB)'; E = { $_.deltaKB } } @{ L = 'Delta(KB)'; E = { $_.deltaKB } }
@@ -239,7 +293,7 @@ if ($Iterations -gt 1) {
if ($ok.Count -gt 0) { if ($ok.Count -gt 0) {
$avgBoot = [math]::Round(($ok | ForEach-Object { $_.totalBootSec } | $avgBoot = [math]::Round(($ok | ForEach-Object { $_.totalBootSec } |
Measure-Object -Average).Average, 2) Measure-Object -Average).Average, 2)
Write-Host "Average boot-to-WinRM ($($ok.Count) successful runs): $avgBoot s" Write-Host "Average boot-to-ready ($($ok.Count) successful runs): $avgBoot s"
} }
} }
+6 -1
View File
@@ -182,7 +182,12 @@ function Invoke-VmrunBounded {
# handles held by child processes (e.g. cmd.exe spawning ping.exe in tests, # handles held by child processes (e.g. cmd.exe spawning ping.exe in tests,
# or vmrun spawning helper subprocesses). Plain proc.Kill() only kills the # or vmrun spawning helper subprocesses). Plain proc.Kill() only kills the
# direct process; child processes keep the write end of stdout open. # direct process; child processes keep the write end of stdout open.
& taskkill /F /T /PID $proc.Id 2>&1 | Out-Null if ($null -ne $IsWindows -and $IsWindows -eq $false) {
# .NET 5+ Kill(entireProcessTree) — works on Linux/macOS.
$proc.Kill($true)
} else {
& taskkill /F /T /PID $proc.Id 2>&1 | Out-Null
}
} }
$sw.Stop() $sw.Stop()