Excessive Memory Usage on Windows machines

Article Number: 


In some rare situations, a Microsoft Windows system may consume excessive amounts of memory. This can cause system performance issues and, in extreme cases, a crash of the system. The problem can have the appearance of a memory leak, with the memory not obviously tied to a process when viewed in Windows Task Manager. StorageCraft has seen this in a very small number of cases with ShadowProtect and ImageManager. Stopping the application or service may result in an instant memory usage drop, which may give the incorrect impression that it is a problem with the application or service, when it may be caused by a bug in the Windows operating system. 


This problem appears to be rare and may be triggered by a whole range of different situations, including reading or writing large files over a network connection, or even from some local disks. In addition, while this condition can occur with a wide range of applications, it may not occur with all applications, even if they perform similar functions. So simply copying a file may not trigger the problem, but a read of that same file may cause the issue.

Hardware problems or configurations may exaggarate the issue further. For example, disabling the ‘Write Through’ option on some RAID controller cards can make the problem much worse as data is streamed to the local disk. This can seem counterintuitive because a RAID with multiple drives should support fast write speeds and high I/O– both of which should limit the need for caching. We have seen this issue on some high performance systems.

The workaround provided by Microsoft is a HotFix that contains a Windows Service called “Dynamic Cache Service (DynCache)”. DynCache may be downloaded for a number of different versions of Windows operating systems and allows the user to set the maximum amount of memory used for Windows caching. This setting is in the Windows Registry.

The Microsoft article found at : http://www.microsoft.com/en-us/download/details.aspx?id=9258 provides more information about the fix. 

Supported Operating Systems:

    • Windows Server 2003 R2 x64 editions
    • Windows Server 2003 x64 editions 
    • Windows Server 2008 Datacenter
    • Windows Server 2008 Enterprise
    • Windows Server 2008 R2
    • Windows Server 2008 R2 Datacenter
    • Windows Server 2008 R2 Enterprise
    • Windows Server 2008 R2 Standard
    • Windows Server 2008 Standard
    • Windows Vista 64-bit Editions Service Pack 1
    • Windows Vista Enterprise 64-bit edition
    • Windows Vista Home Basic 64-bit edition
    • Windows Vista Home Premium 64-bit edition
    • Windows Vista Ultimate 64-bit edition

Please note: The foregoing HotFix and workaround are both provided and recommended by Microsoft. StorageCraft therefore makes no warranty about the effectiveness or suitability of this fix and similarly assumes no liability for issues or problems resulting from application of the fix. Its use is at the sole discretion of the end user. The details provided here are for information purposes only and we recommend the user contact Microsoft if they have any additional questions or concerns.



Hi, The problem occurs


The problem occurs because Image Manager uses Memory Mapping to access to the file. Whilst a good way to handle large file access in general - particularly if multiple file access may be required (for example Image Manager needs to read file for replication AND a client is reading the file to do a diffgen) then you need to use Memory Mapping.

Read https://msdn.microsoft.com/en-us/library/ms810613.aspx for more information about how it works. From this article you can rapidly see how we're going to repeatedly end up in trouble using this.

Firstly - image files (particularly the base images) are typically very large - and will easily consume all available memory on a host. The registry keys suggested in the STC tuning guide are designed to alleviate this behaviour by putting pressure on the host to release the memory. I haven't had a chance to test this yet, but given that the memory allocated to Memory Mapping the file is actually "active" and not "standby" I'm not certain that this will actually have the desired effect. Sorry - this is two problems in one. The first problem is that if these are recommended registry settings - why aren't they applied - or at least give you the option of applying these recommended settings during installation? Particularly because support refuse to help you until you've put the settings in. The other problem is that it may not fix the problem.

Next - I suspect Image Manager should be using something like the "CloseHandle" or "FlushViewOfFile" function that is built into Memory Mapping to notify the OS that it can release the memory if required. It's reading from the file - not writing - so it shouldn't have an issue doing this. Memory Mapping will only be writing to the file if it's generating a new image file (for example during a consolidation event).

If this happened periodically then you wouldn't get this problem in the first place.

Terms and Conditions of Use - Privacy Policy - Cookies