适用于 PowerShell 7.0 及以上版本
在日常运维工作中,我们经常遇到需要同时处理大量独立任务的场景。比如批量检查 100 台服务器的连通性、并行下载数十个日志文件、同时对多个数据库执行查询。如果逐个串行执行,100 台服务器每台超时 5 秒就需要等待超过 8 分钟;而合理使用并发执行,可以将总耗时压缩到几秒到几十秒。
PowerShell 历经多年发展,提供了多种并发执行机制。最早期的 Start-Job 基于独立的 PowerShell 进程,开销大但隔离性好;后来的 ThreadJob 模块改用 .NET 线程,启动更快、内存占用更低;PowerShell 7 引入了 ForEach-Object -Parallel,让并发处理像管道一样简单;而底层的 Runspaces 机制则提供了最精细的控制能力。
本文将从易到难,依次介绍这四种并发机制的核心用法、适用场景和注意事项,帮助你根据实际需求选择最合适的方案。
Start-Job 与 ThreadJob:经典后台任务 Start-Job 是 PowerShell 内置的后台任务机制,每个任务在独立的 PowerShell 进程中运行,天然具备良好的隔离性。ThreadJob 是社区贡献的模块(PowerShell 7 中已内置),改用 .NET 线程而非进程,启动速度更快、资源开销更小。
下面的示例演示如何创建后台任务、跟踪执行进度并收集结果:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 $servers = @ ( 'server-01.example.com' 'server-02.example.com' 'server-03.example.com' 'server-04.example.com' 'server-05.example.com' ) $processJobs = foreach ($server in $servers ) { Start-Job -ScriptBlock { param ($target ) $result = Test-Connection -TargetName $target -Count 2 -Quiet [PSCustomObject ]@ { Server = $target Online = $result Response = if ($result ) { 'OK' } else { 'Timeout' } } } -ArgumentList $server -Name "Ping_$ ($server )" } $threadJobs = foreach ($server in $servers ) { Start-ThreadJob -ScriptBlock { param ($target ) $sw = [System.Diagnostics.Stopwatch ]::StartNew() $result = Test-Connection -TargetName $target -Count 2 -Quiet $sw .Stop() [PSCustomObject ]@ { Server = $target Online = $result Duration = $sw .ElapsedMilliseconds } } -ArgumentList $server -Name "TPing_$ ($server )" } Write-Host "=== 任务状态 ===" Get-Job | Format-Table Name, State, HasMoreData -AutoSize Wait-Job -Job $threadJobs -Timeout 30 | Out-Null Write-Host "`n=== ThreadJob 结果 ===" $results = $threadJobs | Receive-Job $results | Format-Table Server, Online, Duration -AutoSize Get-Job | Remove-Job
执行结果示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 === 任务状态 === Name State HasMoreData ---- ----- -----------Ping_server-01.example.com Running True Ping_server-02.example.com Running True Ping_server-03.example.com Running True Ping_server-04.example.com Running True Ping_server-05.example.com Running True TPing_server-01.example.com Completed True TPing_server-02.example.com Completed True TPing_server-03.example.com Completed True TPing_server-04.example.com Completed True TPing_ server-05.example.com Completed True=== ThreadJob 结果 === Server Online Duration ------ ----- --------server-01.example.com True 23 server-02.example.com True 25 server-03.example.com False 5012 server-04.example.com True 18 server-05.example.com True 31
从结果可以看到,ThreadJob 的任务已经快速完成,而 Start-Job 的任务仍在运行中。这是因为 Start-Job 每个任务都会启动一个全新的 pwsh 进程,初始化开销远大于线程。
ForEach-Object -Parallel:管道式并发 PowerShell 7 引入了 ForEach-Object -Parallel 参数,这是最简洁的并发处理方式。它基于 ThreadJob 实现,在管道中即可实现并行处理,非常适合对集合元素进行并发操作。
-ThrottleLimit 参数控制最大并发数,避免资源耗尽;$using: 语法允许在并行代码块中引用外部变量。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 $urls = @ ( 'https://httpbin.org/delay/1' 'https://httpbin.org/delay/2' 'https://httpbin.org/delay/1' 'https://httpbin.org/delay/3' 'https://httpbin.org/delay/1' 'https://httpbin.org/delay/2' 'https://httpbin.org/delay/1' 'https://httpbin.org/delay/2' ) $timeoutSec = 10 $maxRetries = 2 $measure = Measure-Command { $results = $urls | ForEach-Object -ThrottleLimit 4 -Parallel { $url = $_ $timeout = $using:timeoutSec $retries = $using:maxRetries $attempt = 0 $success = $false $sw = [System.Diagnostics.Stopwatch ]::StartNew() while ($attempt -lt $retries -and -not $success ) { $attempt ++ try { $response = Invoke-WebRequest -Uri $url -TimeoutSec $timeout ` -UseBasicParsing -ErrorAction Stop $success = $true $sw .Stop() } catch { $sw .Stop() if ($attempt -ge $retries ) { return [PSCustomObject ]@ { Url = $url Status = "Failed" Attempt = $attempt Duration = $sw .ElapsedMilliseconds Error = $_ .Exception.Message } } $sw .Start() } } [PSCustomObject ]@ { Url = $url Status = "$ ($response .StatusCode)" Attempt = $attempt Duration = $sw .ElapsedMilliseconds } } $results | Format-Table Url, Status, Attempt, Duration -AutoSize } Write-Host "`n总耗时: $ ($measure .TotalSeconds.ToString('F1')) 秒"
执行结果示例:
1 2 3 4 5 6 7 8 9 10 11 12 Url Status Attempt Duration --- ------ ------- -------- https:// httpbin.org/delay/ 1 200 1 1023 https:// httpbin.org/delay/ 2 200 1 2015 https:// httpbin.org/delay/ 1 200 1 1018 https:// httpbin.org/delay/ 3 200 1 3012 https:// httpbin.org/delay/ 1 200 1 1025 https:// httpbin.org/delay/ 2 200 1 2020 https:// httpbin.org/delay/ 1 200 1 1019 https:// httpbin.org/delay/ 2 200 1 2015 总耗时: 6.1 秒
8 个请求如果串行执行需要约 13 秒(1+2+1+3+1+2+1+2),并行执行(并发数 4)只用了 6.1 秒,效率提升超过 50%。注意 $using: 语法是在并行代码块中访问外部变量的唯一方式,直接引用外部变量会导致作用域错误。
Runspaces:高级并发控制 Runspaces 是 PowerShell 底层的执行环境抽象,每个 PowerShell 实例都运行在一个 Runspace 中。通过直接创建和管理 Runspace 池,我们可以获得最精细的并发控制——包括动态调整并发数、异步回调、自定义初始 Session State 等。
下面演示如何创建 Runspace 池、分发任务并异步收集结果:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 $targets = 1 ..20 | ForEach-Object { "host-$ ($_ .ToString('00'))" }$initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState ]::CreateDefault()$runspacePool = [System.Management.Automation.Runspaces.RunspaceFactory ]::CreateRunspacePool( 1 , 8 , $initialSessionState , $Host ) $runspacePool .Open()$scriptBlock = { param ($computerName ) $sw = [System.Diagnostics.Stopwatch ]::StartNew() Start-Sleep -Milliseconds (Get-Random -Minimum 100 -Maximum 2000 ) $sw .Stop() [PSCustomObject ]@ { ComputerName = $computerName Status = 'Online' CpuUsage = [math ]::Round((Get-Random -Minimum 5 -Maximum 95 ), 1 ) MemUsage = [math ]::Round((Get-Random -Minimum 20 -Maximum 90 ), 1 ) QueryMs = $sw .ElapsedMilliseconds } } $runspaces = foreach ($target in $targets ) { $powershell = [System.Management.Automation.PowerShell ]::Create().AddScript($scriptBlock ).AddArgument($target ) $powershell .RunspacePool = $runspacePool [PSCustomObject ]@ { PowerShell = $powershell Handle = $powershell .BeginInvoke() Target = $target } } $completed = [System.Collections.Generic.List [PSObject ]]::new()$totalMs = [System.Diagnostics.Stopwatch ]::StartNew()foreach ($rs in $runspaces ) { try { $result = $rs .PowerShell.EndInvoke($rs .Handle) if ($result ) { $completed .Add($result [0 ]) } } catch { $completed .Add([PSCustomObject ]@ { ComputerName = $rs .Target Status = "Error: $ ($_ .Exception.Message)" CpuUsage = 0 MemUsage = 0 QueryMs = 0 }) } finally { $rs .PowerShell.Dispose() } } $totalMs .Stop()$completed | Sort-Object QueryMs | Format-Table ComputerName, Status, CpuUsage, MemUsage, QueryMs -AutoSize Write-Host "`n完成 $ ($completed .Count)/$ ($targets .Count) 台,总耗时: $ ($totalMs .ElapsedMilliseconds) ms" $runspacePool .Close()$runspacePool .Dispose()
执行结果示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ComputerName Status CpuUsage MemUsage QueryMs ------------ ------ -------- -------- ------- host-01 Online 12.3 45.2 134 host-05 Online 67.8 52.1 287 host-08 Online 34.5 61.3 412 host-03 Online 89.2 38.7 523 host-12 Online 45.6 72.4 678 host-17 Online 23.1 55.9 789 host-02 Online 56.7 41.8 901 host-19 Online 78.3 63.2 1024 host-06 Online 41.2 47.5 1156 host-10 Online 62.4 58.3 1289 host-15 Online 15.7 69.1 1345 host-04 Online 53.8 43.6 1501 host-09 Online 71.2 51.7 1623 host-14 Online 38.9 65.4 1789 host-20 Online 82.1 37.2 1834 host-07 Online 29.4 54.8 1901 host-11 Online 47.3 62.9 1945 host-13 Online 58.6 49.3 1967 host-16 Online 36.1 71.8 1988 host-18 Online 64.9 44.1 1999 完成 20/20 台,总耗时: 5203 ms
20 台主机串行执行需要约 20 秒(平均每台 1 秒),使用 Runspace 池(8 并发)仅用 5.2 秒完成。Runspaces 方案的优势在于可以对执行环境进行深度定制——比如预加载特定模块、设置执行策略、注入共享变量等。
注意事项
选择合适的并发机制 :简单场景优先使用 ForEach-Object -Parallel(代码最简洁);需要精细控制并发数和生命周期时使用 Start-ThreadJob;需要最大性能和自定义环境时才使用 Runspaces;Start-Job 仅在需要完全进程隔离时使用。
合理设置并发数 :并发数并非越高越好。网络 IO 密集型任务可以设置较高并发(如 16-32),CPU 密集型任务建议设置为逻辑核心数。过高的并发会导致上下文切换开销激增,反而降低性能。
注意变量作用域 :ForEach-Object -Parallel 中无法直接访问外部变量,必须使用 $using: 前缀。Start-Job 和 Start-ThreadJob 通过 -ArgumentList 传参。Runspaces 需要通过 InitialSessionState 或参数注入共享数据。
错误处理不能省 :并发任务中的异常不会自动传播到主线程。必须使用 try/catch 包裹每个任务逻辑,并在 Receive-Job 或 EndInvoke 时检查错误。忽略错误处理会导致静默失败,排查困难。
资源清理是必须的 :Start-Job 创建的进程、ThreadJob 创建的线程、Runspaces 创建的运行空间,都必须在用完后显式清理。使用 Remove-Job 清理 Job,使用 Dispose() 释放 Runspace 和 PowerShell 对象,避免内存泄漏。
避免共享可变状态 :并发任务中修改同一个集合或变量会导致竞态条件。应让每个任务返回独立的结果对象,最后在主线程中统一汇总。如果必须共享状态,请使用 .NET 的线程安全集合(如 ConcurrentDictionary)。