Alex Ionescu Blog
January 3rd, 2008, 16:50
After my departure from the ReactOS ("http://www.reactos.org")*project and subsequent new work ("http://www.solsem.com") for David Solomon, it wasn’t clear how much research and development on Windows internals I would still be able to do on a daily basis. Thankfully, I haven’t given up my number one passion — innovating, pushing the boundaries of internals knowledge, and educating users through utilities and applications.
In this vein, I have been working during my spare time on various new utilities that use new undocumented APIs and expose the internals behind Windows Vista to discover more about how the operating system works, as well as to be able to provide useful information to administrators, developers, students, and anyone else in between. In this post, I want to introduce my latest tool, MemInfo. I’ll show you how MemInfo can help you find bad memory modules (RAM sticks) on your system, track down memory leaks and even assist in detecting rootkits!
One of the major new features present in Windows Vista is Superfetch. Mark Russinovich did an excellent writeup on this as part of his series on Windows Vista Kernel Changes ("http://www.microsoft.com/technet/technetmag/issues/2007/03/VistaKernel/default.aspx") in TechNet Magazine. Because Superfetch’s profiling and management does not occur at the kernel layer (but rather as a service, by design choice), there had to be a new system call to communicate with the parts of Superfetch that do live in the kernel, just like Windows XP’s prefetcher code, so that the user-mode service could request information as well as send commands on operations to be performed.
http://www.alex-ionescu.com/spservice.jpg
Here’s an image of Process Explorer identifying the Superfetch service inside one of the service hosting processes.
Because Superfetch goes much deeper than the simple file-based prefetching Windows XP and later offer, it requires knowledge of information such as the internal memory manager lists, page counts and usage of pages on the system, memory range information, and more. The new SuperfetchInformationClass added to NtQuery/SetInformationSystem provides this data, and much more.
MemInfo uses this API to query three kinds of information:
a list of physical address ranges on the system, which describe the system memory available to Windows
information about each page on the system (including its location on the memory manager lists, its usage, and owning process, if any)
a list system/session-wide process information to correlate a process’ image name with its kernel-mode object*
MemInfo ultimately provides this information to the user through a variety of command line options described when you run the utility. Some of its various uses include:
Seeing how exactly Windows is manipulating your memory, by looking at the page list summaries.
The Windows memory manager puts every page on the system on one of the many page lists that it manages (i.e. the standby list, the zeroed page list, etc). Windows Internals ("http://www.microsoft.com/mspress/books/6710.aspx") covers these lists and usage in detail, and MemInfo is capable of showing their sizes to you (including pages which are marked Active, meaning currently in-use and by the operating system and occupying physical memory (such as working sets) and not on any of the lists). This information can help you answer questions such as “Am I making good use of my memory?” or “Do I have damaged RAM modules installed?”.
For example, because Windows includes a bad page list where it stores pages that have failed internal consistency checks, MemInfo is an easy way (but not 100% foolproof, since Windows’ internal checks might not have detected the RAM is bad) to verify if any memory hardware on the system is misbehaving. Look for signs such as a highly elevated count of pages in the zeroed page list (after a day’s worth of computer use) to spot if Windows hasn’t been fully using your RAM to its potential (you may have too much!) or to detect a large memory deallocation by a process (which implies large allocations previously done).
http://www.alex-ionescu.com/meminfo.jpg
Here’s MemInfo on my 32-bit Vista system, displaying summary page list information.
Windows Vista also includes a new memory manager optimization called prioritized standby lists — the standby state is the state in which pages find themselves when they have been cached by Windows (various mechanisms are responsible for this of caching, including the cache manager and Superfetch) and are not currently active in memory. Mark covered these new lists in his excellent article as well.
To expose this information to system administrators, three new performance counters were added to Windows, displaying the size of the prioritized standby lists in groupings: priorities 0 through 3 are called Standby Cache Reserve, 4 and 5 are called Standby Cache Normal Priority, and finally, 6 and 7 are called Standby Cache Core. MemInfo on the other hand, which can also display these new lists, is an even better tool to identify memory in the standby state, since it is able to display the size of these lists individually.
While memory allocations on Windows XP (which could be part of application startup, the kernel-mode heap, or simple memory allocations coming from various processes) would consume pages from a single standby list and thus possibly steal away pages that more critical processes would’ve liked to have on standby, Windows Vista adds 8 prioritized lists, so that critical pages can be separated from less important pages and nearly useless pages. This way, when pages are needed for an allocation, the lower priority standby lists are used first (a process called repurposing). By making snapshots of MemInfo’s output over a period of time, you can easily see this behavior.
Here’s MemInfo output before, during, and after a large allocation of process private memory. Notice how initially, the bulk of my memory was cached on the standby lists. Most of the memory then became Active due to the ongoing large allocation, emptying the standby lists, starting by the lowest priority. Finally, after the memory was freed, most of the memory now went on the zero page list (meaning the system just had to zero 1GB+ of data).
http://www.alex-ionescu.com/alloc.jpg
http://www.alex-ionescu.com/during.jpg
http://www.alex-ionescu.com/free.jpg
Seeing to what use are your pages being put to by Windows
Apart from their location on one of the page lists, Windows also tracks the usage of each page on the system. The full list includes about a dozen usages, ranging from non-paged pool to private process allocations to kernel stacks. MemInfo shows you the partitioning of all your pages according to their usage, which can help pinpoint memory leaks. High page counts in the driver locked pages, non-paged pool pages and/or kernel stack pages could be indicative of abnormal system behavior.
The first two are critical resources on the system (much information is available on the Internet for tracking down pool leaks), while the latter is typically tightly maintained for each thread, so a large number may indicate leaked threads. Other usages should also expect to see a lower number of pages than ones like process private pages, which is usually the largest of the group.
At the time of this writing, here’s how Windows is using my 4GB of memory:
http://www.alex-ionescu.com/memusage.jpg
Looking at per-process memory usage, and detecting hidden processes
Internally, Windows associates private process pages with the kernel executive object that represents processes as managed by the process manager — the EPROCESS structure. When querying information about pages, the API mentioned earlier returns EPROCESS pointers — not something very usable from user-mode! However, another usage of this API is to query the internal list of processes that Superfetch’s kernel-mode component manages. This list not only allows to take a look at how much memory, exactly, belongs to each process on the system, but also to detect some forms of hidden processes!
Hidden processes are usually the cause of two things. The first is processes which have been terminated, but not yet fully cleaned up by the kernel, because of handles which are still open to them. Task Manager and Process Explorer will not show these processes, but MemInfo is the only tool apart from the kernel debugger which can (so you don’t have to reboot in debugging mode). See below on how MemInfo is showing a SndVol32.exe process, created by Windows Explorer when clicking on the speaker icon in the system tray — Explorer has a bug which leaks the handles, so the process is never fully deleted until Explorer is closed.
http://www.alex-ionescu.com/soundleak.jpg
The second cause of why a process may be hidden is a rootkit that’s hooking various system calls and modifying the information returned to user-mode to hide a certain process name. More advanced rootkits will edit the actual system lists that the process manager maintains (as well as try to intercept any alternate methods that rootkit detection applications may use), but MemInfo adds a new twist, by using a totally new and undocumented Superfetch interface in Windows Vista. It’s likely that no rootkit in the wild currently knows about Superfetch’s own process database, so MemInfo may reveal previously hidden processes on your system. Unfortunately, as with all information, it’s only a matter of time until new rootkits adapt to this new API, so expect this to be obsolete in the next two years.
There’s many more uses for MemInfo that I’m sure you can find — including as a much faster replacement for !memusage 8 if you’ve used that command in WinDBG before. MemInfo is fully compatible with both 32-bit and 64-bit versions of Windows Vista (including SP1 and Windows Server 2008 RC1 – even though Server 2008 does not include Superfetch, because both client and server versions of the OS will use the same kernel as of Vista SP1, the API set is identical), but not any earlier version of Windows. Apart from these simple summary views, MemInfo is powerful enough to dump the entire database of pages on your system, with detailed information on each — valuable information for anyone that needs to deal with this kind of data. Furthermore, unlike using WinDBG to attach to the local kernel, it doesn’t require booting the system into debug mode.
You can download a .zip file containing both versions from this link ("http://www.winsiderss.com/tools/meminfo/meminfo.zip"). Make sure to run MemInfo in an elevated command prompt — since it does require administrative privileges. The documentation for MemInfo is located on the following page ("http://www.winsiderss.com/tools/meminfo/meminfo.htm") (this page is part of an upcoming website on which I plan to organize and offer help/links to my tools and articles).
http://www.alex-ionescu.com/?p=51
In this vein, I have been working during my spare time on various new utilities that use new undocumented APIs and expose the internals behind Windows Vista to discover more about how the operating system works, as well as to be able to provide useful information to administrators, developers, students, and anyone else in between. In this post, I want to introduce my latest tool, MemInfo. I’ll show you how MemInfo can help you find bad memory modules (RAM sticks) on your system, track down memory leaks and even assist in detecting rootkits!
One of the major new features present in Windows Vista is Superfetch. Mark Russinovich did an excellent writeup on this as part of his series on Windows Vista Kernel Changes ("http://www.microsoft.com/technet/technetmag/issues/2007/03/VistaKernel/default.aspx") in TechNet Magazine. Because Superfetch’s profiling and management does not occur at the kernel layer (but rather as a service, by design choice), there had to be a new system call to communicate with the parts of Superfetch that do live in the kernel, just like Windows XP’s prefetcher code, so that the user-mode service could request information as well as send commands on operations to be performed.
http://www.alex-ionescu.com/spservice.jpg
Here’s an image of Process Explorer identifying the Superfetch service inside one of the service hosting processes.
Because Superfetch goes much deeper than the simple file-based prefetching Windows XP and later offer, it requires knowledge of information such as the internal memory manager lists, page counts and usage of pages on the system, memory range information, and more. The new SuperfetchInformationClass added to NtQuery/SetInformationSystem provides this data, and much more.
MemInfo uses this API to query three kinds of information:
a list of physical address ranges on the system, which describe the system memory available to Windows
information about each page on the system (including its location on the memory manager lists, its usage, and owning process, if any)
a list system/session-wide process information to correlate a process’ image name with its kernel-mode object*
MemInfo ultimately provides this information to the user through a variety of command line options described when you run the utility. Some of its various uses include:
Seeing how exactly Windows is manipulating your memory, by looking at the page list summaries.
The Windows memory manager puts every page on the system on one of the many page lists that it manages (i.e. the standby list, the zeroed page list, etc). Windows Internals ("http://www.microsoft.com/mspress/books/6710.aspx") covers these lists and usage in detail, and MemInfo is capable of showing their sizes to you (including pages which are marked Active, meaning currently in-use and by the operating system and occupying physical memory (such as working sets) and not on any of the lists). This information can help you answer questions such as “Am I making good use of my memory?” or “Do I have damaged RAM modules installed?”.
For example, because Windows includes a bad page list where it stores pages that have failed internal consistency checks, MemInfo is an easy way (but not 100% foolproof, since Windows’ internal checks might not have detected the RAM is bad) to verify if any memory hardware on the system is misbehaving. Look for signs such as a highly elevated count of pages in the zeroed page list (after a day’s worth of computer use) to spot if Windows hasn’t been fully using your RAM to its potential (you may have too much!) or to detect a large memory deallocation by a process (which implies large allocations previously done).
http://www.alex-ionescu.com/meminfo.jpg
Here’s MemInfo on my 32-bit Vista system, displaying summary page list information.
Windows Vista also includes a new memory manager optimization called prioritized standby lists — the standby state is the state in which pages find themselves when they have been cached by Windows (various mechanisms are responsible for this of caching, including the cache manager and Superfetch) and are not currently active in memory. Mark covered these new lists in his excellent article as well.
To expose this information to system administrators, three new performance counters were added to Windows, displaying the size of the prioritized standby lists in groupings: priorities 0 through 3 are called Standby Cache Reserve, 4 and 5 are called Standby Cache Normal Priority, and finally, 6 and 7 are called Standby Cache Core. MemInfo on the other hand, which can also display these new lists, is an even better tool to identify memory in the standby state, since it is able to display the size of these lists individually.
While memory allocations on Windows XP (which could be part of application startup, the kernel-mode heap, or simple memory allocations coming from various processes) would consume pages from a single standby list and thus possibly steal away pages that more critical processes would’ve liked to have on standby, Windows Vista adds 8 prioritized lists, so that critical pages can be separated from less important pages and nearly useless pages. This way, when pages are needed for an allocation, the lower priority standby lists are used first (a process called repurposing). By making snapshots of MemInfo’s output over a period of time, you can easily see this behavior.
Here’s MemInfo output before, during, and after a large allocation of process private memory. Notice how initially, the bulk of my memory was cached on the standby lists. Most of the memory then became Active due to the ongoing large allocation, emptying the standby lists, starting by the lowest priority. Finally, after the memory was freed, most of the memory now went on the zero page list (meaning the system just had to zero 1GB+ of data).
http://www.alex-ionescu.com/alloc.jpg
http://www.alex-ionescu.com/during.jpg
http://www.alex-ionescu.com/free.jpg
Seeing to what use are your pages being put to by Windows
Apart from their location on one of the page lists, Windows also tracks the usage of each page on the system. The full list includes about a dozen usages, ranging from non-paged pool to private process allocations to kernel stacks. MemInfo shows you the partitioning of all your pages according to their usage, which can help pinpoint memory leaks. High page counts in the driver locked pages, non-paged pool pages and/or kernel stack pages could be indicative of abnormal system behavior.
The first two are critical resources on the system (much information is available on the Internet for tracking down pool leaks), while the latter is typically tightly maintained for each thread, so a large number may indicate leaked threads. Other usages should also expect to see a lower number of pages than ones like process private pages, which is usually the largest of the group.
At the time of this writing, here’s how Windows is using my 4GB of memory:
http://www.alex-ionescu.com/memusage.jpg
Looking at per-process memory usage, and detecting hidden processes
Internally, Windows associates private process pages with the kernel executive object that represents processes as managed by the process manager — the EPROCESS structure. When querying information about pages, the API mentioned earlier returns EPROCESS pointers — not something very usable from user-mode! However, another usage of this API is to query the internal list of processes that Superfetch’s kernel-mode component manages. This list not only allows to take a look at how much memory, exactly, belongs to each process on the system, but also to detect some forms of hidden processes!
Hidden processes are usually the cause of two things. The first is processes which have been terminated, but not yet fully cleaned up by the kernel, because of handles which are still open to them. Task Manager and Process Explorer will not show these processes, but MemInfo is the only tool apart from the kernel debugger which can (so you don’t have to reboot in debugging mode). See below on how MemInfo is showing a SndVol32.exe process, created by Windows Explorer when clicking on the speaker icon in the system tray — Explorer has a bug which leaks the handles, so the process is never fully deleted until Explorer is closed.
http://www.alex-ionescu.com/soundleak.jpg
The second cause of why a process may be hidden is a rootkit that’s hooking various system calls and modifying the information returned to user-mode to hide a certain process name. More advanced rootkits will edit the actual system lists that the process manager maintains (as well as try to intercept any alternate methods that rootkit detection applications may use), but MemInfo adds a new twist, by using a totally new and undocumented Superfetch interface in Windows Vista. It’s likely that no rootkit in the wild currently knows about Superfetch’s own process database, so MemInfo may reveal previously hidden processes on your system. Unfortunately, as with all information, it’s only a matter of time until new rootkits adapt to this new API, so expect this to be obsolete in the next two years.
There’s many more uses for MemInfo that I’m sure you can find — including as a much faster replacement for !memusage 8 if you’ve used that command in WinDBG before. MemInfo is fully compatible with both 32-bit and 64-bit versions of Windows Vista (including SP1 and Windows Server 2008 RC1 – even though Server 2008 does not include Superfetch, because both client and server versions of the OS will use the same kernel as of Vista SP1, the API set is identical), but not any earlier version of Windows. Apart from these simple summary views, MemInfo is powerful enough to dump the entire database of pages on your system, with detailed information on each — valuable information for anyone that needs to deal with this kind of data. Furthermore, unlike using WinDBG to attach to the local kernel, it doesn’t require booting the system into debug mode.
You can download a .zip file containing both versions from this link ("http://www.winsiderss.com/tools/meminfo/meminfo.zip"). Make sure to run MemInfo in an elevated command prompt — since it does require administrative privileges. The documentation for MemInfo is located on the following page ("http://www.winsiderss.com/tools/meminfo/meminfo.htm") (this page is part of an upcoming website on which I plan to organize and offer help/links to my tools and articles).
http://www.alex-ionescu.com/?p=51
, it affected my "stability rating" of your hosting, so I uploaded it here. We will never host any tools against the will of their author though, so we have no problem at all with your request, but rather just appreciate it.