Original KB number: 4016173
Summary
Resilient File System (ReFS) can consume large amounts of memory because of its allocate-on-write metadata semantics and block caching logic. This behavior can cause memory pressure that degrades server performance.
This article describes the cause of the issue and explains how to apply a cumulative update and configure registry entries to reduce ReFS memory usage.
Symptoms
You notice heavy memory usage on a computer that uses the Resilient File System (ReFS).
Cause
To provide greater resiliency for its metadata, ReFS uses allocate-on-write semantics for all metadata updates. This approach means that ReFS never makes in-place updates to metadata. Instead, it makes all writes to newly allocated regions.
However, using allocate-on-write causes ReFS to issue more metadata I/O to new regions of the volume than write-in-place file systems do. Additionally, ReFS uses block caching logic to cache its metadata in RAM. It isn't as resource efficient as file caching logic.
Because of the ReFS block caching logic and allocate-on-write semantics, ReFS uses large metadata streams. ReFS uses the cache manager to create the metadata streams, and the cache manager unmaps inactive views in a lazy manner. In some situations, this lazy unmapping causes the active working set on the server to grow. This growth creates memory pressure that can, in turn, cause poor performance.
Resolution
Cumulative update (CU) 4013429 that was released on March 14, 2017 introduced three tunable registry entries that you can use to adjust large ReFS metadata streams.
How to set the tunable registry entries
Important: This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For protection, back up the registry before you modify it so that you can restore it if a problem occurs. For more information about how to back up and restore the registry, see How to back up and restore the registry in Windows.
You can use Registry Editor or command-line commands (such as reg add) to set the registry entries. The scopes of these parameters don't overlap, so you can use them in any combination. The registry entries are all under the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem subkey.
| Value name | Type | Default value | Effects |
|---|---|---|---|
RefsEnableLargeWorkingSetTrim |
REG_DWORD | When set to 1, ReFS tries to a complete an MM unmap of all metadata streams at every checkpoint. | |
RefsNumberOfChunksToTrim |
REG_DWORD | 4 | Affects the granularity at which ReFS unmaps when it cycles the entire namespace. Setting RefsNumberOfChunksToTrim to higher values causes ReFS to trim more aggressively. It reduces the amount of memory that's being used. |
RefsEnableInlineTrim |
REG_DWORD |
Important:
- After you change these parameters, you have to restart the computer.
- On a failover cluster, you have to set these parameters in the same manner on every cluster node.
If you see poor performance because of a large active working set, follow these steps to tune ReFS behavior:
-
Verify that the volume is idle and doesn't have any mapped pages. Then, set
RefsEnableLargeWorkingSetTrimto1, and monitor the system to see whether performance improves. -
If performance doesn't improve sufficiently, verify that the VA range that's being unmapped doesn't have any active references (that is, mapped metadata pages).
-
Change the value of
RefsNumberOfChunksToTrim. Try different values, such as8,16,32, and so forth.[!NOTE]
The following formulas determine the granularity at which ReFS unmaps virtual address space:RefsNumberOfChunksToTrim* 128 MB (for volume of size > 10 TB)RefsNumberOfChunksToTrim* 64 MB (for volume of size < 10 TB)
-
If performance still doesn't improve, set
RefsEnableInlineTrimto1.
More information
To improve its resiliency to corruption, ReFS uses allocate-on-write instead of write-in-place to update its metadata. A power failure or an unexpected dismount can cause "torn writes" (writes that are only partially completed) in a system that uses write-in-place. However, under the same circumstances, a system that uses allocate-on-write can still reference the previous, consistent metadata copy. In this manner, ReFS reliably maintains metadata consistency.
References
Source: Microsoft Learn / IT Pro troubleshooting by Microsoft and contributors. Documentation licensed CC BY 4.0. Code samples are covered by the MIT License. Converted from Markdown, with links normalized and embedded media omitted.
