大网数据广告活动
故障转移群集

CSV Goes Offline after a Node or Storage Component Goes Offline During Active I/O

· 52CE 我爱测

Discusses a change in Cluster Shared Volume behavior that sends the CSV offline and could cause it to fail.

This article discusses a situation in which the Cluster Shared Volume (CSV) of a cluster goes offline after other components go offline. The article includes steps to resolve the issue and, if it's necessary, recover any affected virtual machines (VMs).

Symptoms

This issue starts under the following circumstances:

  1. A cluster node or storage component becomes unavailable, but I/O operations continue. For example, a disk array fails or requires maintenance.
  2. As I/O operations continue, metadata records accumulate.
  3. When the metadata records reach their allocated limits, I/O operations fail.
  4. The associated CSV enters a Failed state.
  5. Every 15 minutes (the default setting), the cluster tries to bring the CSV online. If the Virtual Machine Management Service (VMMS) manages VMs on the cluster, VMMS periodically tries to start the VMs.
  6. After 30 minutes, VMMS stops trying to start the VMs. Any VMs that use the affected CSV can't automatically recover.

Cause

A recent change in cluster behavior affects how the CSV responds in the situation that's mentioned in the "Symptoms" section. Previously, when metadata records accumulated to the allocated limits, I/O operations could stop indefinitely. Because of the change, I/O operations fail in this situation instead of hanging. The I/O failure, in turn, causes the CSV to go offline and enter a Failed state.

Recovery

You can use one of two methods to recover the cluster. The method to use depends on whether you can restore the previous cluster components, or you have to replace parts of the cluster.

Method 1: Restore the offline component and automatically repair the cluster

After you restore the offline node or storage, the following steps occur automatically.

  1. The next time that the cluster automatically tries to bring the CSV online, it succeeds.
  2. Automatic repair processes start, and then the volume becomes available.

Important:
After the cluster recovers, you might have to manually start any VMMS-managed VMs that use the cluster. After the cluster is down for 30 minutes, VMMS stops automatically trying to restart the VMs.

Method 2: Replace the offline component and manually recover the cluster

If you can't restore the missing node or storage, follow these steps to manually recover the cluster.

Important:

  • This procedure temporarily takes all volumes in the pool offline.
  • You can use this procedure for either Storage Spaces or failover clusters. Every step that applies to virtual disks or volumes also applies to Cluster Virtual Disks and Cluster Shared Volumes.

Run the following steps as a cluster administrator on a node that has full access to the storage pool.

  1. On a cluster node that has full access to the storage pool, open an administrative PowerShell Command Prompt window.

  2. To get the properties of the affected storage pool, run the following command at the PowerShell command prompt:

    Get-ClusterResource <Pool>  
    

    [!NOTE]

    • In this cmdlet, <Pool> is the name of the storage pool resource.
    • Later steps in this procedure use properties such as the name and owner group of the resource.
  3. To get the properties of the storage pool's virtual disks and CSVs, run the following command:

    Get-ClusterResource | Where-Object { $_.ResourceType -eq "Physical Disk" }
    
    Get-ClusterSharedVolume
    
  4. Review the properties to determine which resources are in a "Failed" state.

    [!NOTE]
    If you intend to reuse name and ID information for any resources that you replace, you can use Get-ClusterResource and Get-ClusterParameter to get that information.

  5. Whether you're replacing a node, or just storage, run the following cmdlets to add unpooled disks to the storage pool:

    Get-StoragePool -isprimordial $false | Add-PhysicalDisk -PhysicalDisks $(Get-PhysicalDisk -CanPool $true)
    
  6. To retire unhealthy disks (or disks that are associated with a failed node), run the following cmdlets:

    Get-PhysicalDisk | Where-Object { $_.OperationalStatus -eq "Lost Communication" } | Set-PhysicalDisk -usage Retired
    
  7. Monitor the virtual disks by running Get-VirtualDisk and looking for OperationalStatus = InService in the cmdlet output. When the OperationalStatus parameter is clear for all the virtual disks, go to the next step.

  8. To move the affected storage pool (that you identified previously) to the current node, run a PowerShell cmdlet that resembles the following command:

    Move-ClusterResource -node <Current Node> -name <OwnerGroup>
    

    [!NOTE]
    In this command, <Current Node> is the name of the node that you're working from, and <OwnerGroup> is the value of the OwnerGroup property of the storage group resource.

  9. To move the failed disk and CSV resources to the current node, run Move-ClusterResource again for each physical disk and CSV resource. To see the OwnerGroup value of the CSV, run Get-ClusterSharedVolume | get-ClusterGroup.

  10. To remove all cluster virtual disks and CSVs from cluster management, run the following PowerShell commands in sequence:

    Remove-ClusterSharedVolume
    Get-ClusterResource | Where-Object { $_.ResourceType -eq "Physical Disk" } | Remove-ClusterResource
    
  11. To remove the storage pool from cluster management, run the Remove-ClusterResource command for the storage pool objects that you identified in step 2 of this procedure.

  12. To make the storage pool writable, run the following commands:

    Get-StoragePool -isPrimordial $false | Set-StoragePool -IsReadOnly $false
    
  13. To configure the virtual disks, run the following commands for each virtual disk that you identified in step 3 of this procedure.

    Get-VirtualDisk | Set-VirtualDisk -IsManualAttach $false
    
  14. Use the Get-StorageJob cmdlet to monitor the storage jobs that are related to repair. After the jobs start (the percentage completed is greater than 0), go to the next step.

  15. To restore the storage pool to cluster management, run the following commands:

    Get-CimInstance -Namespace "root\MSCluster" -ClassName "MSCluster_AvailableStoragePool" | invoke-cimmethod -MethodName AddToCluster
    
  16. Restore all non-failed virtual disks to cluster management. If any of the virtual disks from the previous step were configured as CSVs before the failure, convert them to CSVs.

    For example, you can bring back any of the virtual disk or CSV resources that weren't in a failed state. To restore these resources, use the virtualdiskid and name property values from step 3, and then run commands that resemble the following script excerpt:

    `$virtualdiskname = "ClusterPerformanceHistory"`
    `$virtualdiskid = "603bb5d0-9c4d-4fc6-9c25-eec92a478733"`
    `(get-clusteravailabledisk  | Where-Object { $_.Id -eq $virtualdiskid} | add-clusterdisk).Name = $virtualdiskname`
    

    You can use Add-ClusterSharedVolume to reconfigure the CSVs.

  17. Monitor the virtual disks by running Get-VirtualDisk and looking for OperationalStatus = InService in the cmdlet output. When the OperationalStatus parameter is clear for all of the virtual disks, continue to the next step.

  18. To bring the virtual disks online and configure them as read/write, run the following commands:

    Get-VirtualDisk | get-disk | Where-Object { $_.IsReadonly -eq $true } | set-disk -IsReadOnly $false
    Get-VirtualDisk | get-disk | Where-Object { $_.IsOffline -eq $true} | set-disk -IsOffline $false
    
  19. Monitor the virtual disk footprints of the retired physical disks by running the following cmdlet:

     get-physicaldisk -Usage Retired | ft Deviceid, Usage, VirtualDiskFootprint 
    

    When the footprint reaches zero, go to the next step.

  20. Restore the previously failed virtual disks to cluster management. If any of these virtual disks were previously configured as CSVs, convert them to CSVs.

  21. Bring the virtual disks from the previous step online, and configure them as read/write.

Important:
After the cluster recovers, you might have to manually start any VMMS-managed VMs that use the cluster. After the cluster is down for 30 minutes, VMMS stops automatically trying to restart the VMs.

Status

This behavior is by design in Windows Server 2025. It's intended to prevent indefinite I/O unresponsiveness.

Source: Microsoft Learn / IT Pro troubleshooting by Microsoft and contributors (upstream author metadata: kaushika-msft). Documentation licensed CC BY 4.0. Code samples are covered by the MIT License. Converted from Markdown, with links normalized and embedded media omitted.

资料来源与适用范围

资料来源
Microsoft Learn / IT Pro troubleshooting
适用范围
- Windows Server 2025 · Source date 02/12/2026
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY 4.0 (documentation), MIT (code samples)

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。