Log in

View Full Version : Windbg showing unexpected FP register content in kernel mode

niaren
December 10th, 2009, 17:01
Hi all, this is my first post.
I will begin by thanking all the people that are devoting time to moderate and otherwise keeping this place up and running. I have used this forum quite a lot recently and thought maybe it is time to chime in although I'm a little unsure if this post qualifies as a first post but I'll give it a try. Now to the business

I'm trying to reverse engineer part of an audio driver. I'm interested in the part which actually modifies the audio. I did some initial analysis using IDA which provided me with some good spots as to where to set some breakpoints for further analysis in a debugger. It took me some time to figure out how to create an appropriate setup for single-stepping in kernel mode. I ended up having XP SP2 (guest OS) running in a Virtual Machine (VMWare) on a Vista laptop (Host OS). Using Windbg on the Host OS it is possible to do kernel mode debugging in the guest OS (this link provide the details on how to do this: http://www.catch22.net/tuts/vmware).

Now I'm able to single-step through the code in the audio driver and everything seems perfectly alright. All the calculations modifying the audio stream are done in floating point and that is where my understanding of things starts to fall short. In Windbg the FP registers st0-st7 are not updated in response to executing FP instructions. Other registers are updated alright for instance eip, esp and ebp are updated when executing move esp, ebp.

In order to find out what is going on I checked the problem with this simple 'driver'

Code:


#include <wdm.h>

#include <windef.h>



KFLOATING_SAVE saveData;

NTSTATUS status;

double floatValue;



VOID OnUnload(PDRIVER_OBJECT DriverObject)

{

	status = KeSaveFloatingPointState(&saveData);

	if (status == STATUS_SUCCESS) 

	{

		floatValue = (double)1;

		KeRestoreFloatingPointState(&saveData);

	}

}



NTSTATUS DriverEntry(PDRIVER_OBJECT DriverObject, PUNICODE_STRING RegistryPath)

{

	DriverObject->DriverUnload = OnUnload;

	return STATUS_SUCCESS;

}


Link to code: http://msdn.microsoft.com/en-us/library/aa489566.aspx
The code depends on libcntpr.lib and ks.lib

The disassembly for the code in the OnUnload routine is

Code:


8cd7530 8bff            mov     edi,edi

f8cd7532 55              push    ebp

f8cd7533 8bec            mov     ebp,esp

f8cd7535 684077cdf8      push    offset hello!saveData (f8cd7740)

f8cd753a ff151076cdf8    call    dword ptr [hello!_imp__KeSaveFloatingPointState (f8cd7610)]

f8cd7540 a33077cdf8      mov     dword ptr [hello!status (f8cd7730)],eax

f8cd7545 833d3077cdf800  cmp     dword ptr [hello!status (f8cd7730)],0

f8cd754c 7513            jne     hello!OnUnload+0x31 (f8cd7561)

f8cd754e d9e8            fld1

f8cd7550 dd1d2877cdf8    fstp    qword ptr [hello!floatValue (f8cd7728)]

f8cd7556 684077cdf8      push    offset hello!saveData (f8cd7740)

f8cd755b ff151476cdf8    call    dword ptr [hello!_imp__KeRestoreFloatingPointState (f8cd7614)]

f8cd7561 5d              pop     ebp

f8cd7562 c20400          ret     4


After stepping over the fld1 instruction nothing happens with st0. So the problem persists in this much simpler setup.

The best explanation I have found about this is in the Windbg help

Quote:
In kernel-mode debugging, there are many processes, threads, and sometimes user sessions that are executing at the same time. Therefore, phrases such as "virtual address 0x80002000" or "the eax register" are ambiguous. You must specify the context in which such phrases can be understood.
........
The register context determines how the debugger interprets registers and also controls the results of a stack trace. This context is also known as the thread context, although that term is not completely accurate. An explicit context is also a type of register context. If you specify an explicit context, that context is used instead of the current register context.


One problem is how to figure out what is the right thread context but another problem is how this can explain why it is only the FP registers that are not updated. My laptop processor is an Intel Pentium Dual CPU.

Maybe I'm overlooking something very simple, that is probably the most likely thing. For the moment this problem has set me stymied so I'm very interested in any kind of input of what could be the problem.

niaren
Kayaker
December 11th, 2009, 00:35
Hi

I can confirm that behaviour. When remote debugging a *usermode* process with FP instructions, the FP registers change, but for a kernelmode process they don't.

One thing I noticed however, the rF command shows a different set of FP values than that displayed in the register window - for BOTH user and kernel mode debugging.

One suggestion, I didn't check what the FP register values were *supposed* to be when tracing over the FP instructions (I used the same fld1/fstp as your example as inline asm). If you could check in both user and kernel mode to see if the values displayed with the rF command are accurate (or if the register window values in usermode are accurate). If they are, then at least you could determine them with rF, even if they aren't updated correctly in the register window.

If the FP values are NOT accurate under any condition of remote debugging, and we can't figure out how to change the context to be correct, then I'm wondering if it's a problem with VMWare emulation of the FP registers?

Hopefully someone here can figure this out, because it's somewhat important in the grand scheme of all things reversing, but if not, you should ask this question in the OSR WinDbg forum, where the developers themselves hang out. I wouldn't mention you're doing anything reversing related, just give 'em the example If you do get an explanation please relay the solution.


Oh, and if you're doing a lot of remote kernel debugging (and don't feel like using Softice, which should actually be perfectly fine for xpsp2 on vmware and would solve this FP problem once and for all), you might be interested in trying

Windows Kernel Debugger booster for Virtual Machines
http://virtualkd.sysprogs.org/

Hmm, after mentioning Softice/vmware and FP registers, checking them directly in this way for accuracy should resolve whether it's an issue with FP emulation while in a debugger. If not, then it's a problem when being piped to Windbg.

Kayaker
niaren
December 15th, 2009, 15:15
Hi Kayaker,
I'm still *processing* your very helpful input. I can confirm that for kernel mode the FP registers are not correct in the Register window or when using the rF command and that they are showing different values.

I still haven't found out yet how to do remote user mode debugging from kernel mode but I haven't researched this enough myself. So I haven't checked the FP registers for this case yet.

I have had a heck of a time getting Softice to work in the VM. I ended up installing driverstudio 3.2 and tried a lot of things to get softice to work. I tried installing the 3.2 patch and follow the ideas presented in these threads
http://www.woodmann.com/forum/showthread.php?t=10428&highlight=softice+nightmare
http://www.woodmann.com/forum/showthread.php?t=5806
but nothing worked. Especially the int2d trick gave some gray hairs as I couldn't find the byte pattern which had to be looked up. I found out that it was a display problem as Softice indeed would break on ctrl-D but the screen just frooze until pressing ctrl-D again.
In the end what worked for me was to first uninstall and then reinstall driverstudio. Then it worked. Even without the 3.2 patch. For the record I should mention that in the reinstall I chose not to install visual softice and that the second install was performed after I had added these two lines
vmmouse.present = "FALSE"
svga.maxFullscreenRefreshTick = "5"
to the VMs .vmx file.

When using softice in the VM for debugging in kernel mode the FP registers are updated and displayed as expected.

Maybe this indicates that the problem is not with emulation of the FP registers while in a debugger but maybe more with the piping to windbg.

Kayaker, wrt to the osr posting, if this is still a good idea can I send you a draft of a posting for you to review/edit? I think this would make it more likely that anyone will reply to the post.

And finally thank you for the VirtualKD link I wasn't aware of it.

niaren
Kayaker
December 19th, 2009, 07:18
That's a good bug find. If indeed it's a "bug". I can't see it really being a bug in the usual Windbg kernel debugging 2 computer/null-modem cable setup. You would have thunk it would have been picked up by now if there was a problem with displaying correct FP register values.

Of course the only way to check is to set Windbg up "properly" in that manner and test for proper procedure and results. Then one could compare with using VMWare.

I issued the .thread command before stepping over the FP instructions, to make sure the context was set and recognized correctly in Windbg - it was. I double checked that from within the test driver itself with
DbgPrint ("%x", PsGetCurrentThread() );

Still, no joy. No response/change from the CPU window FP registers, nor from the rF command. Maybe we just don't know how to do it properly.


Quote:
Generally, when you are doing kernel debugging, the only visible registers are the ones associated with the current thread.

The .thread command instructs the kernel debugger to use the specified thread as the register context. After this command is executed, the debugger will have access to the most important registers and the stack trace for this thread. This register context persists until you allow the target to execute or use another register context command (.thread, .cxr, or .trap). See Register Context for full details.


Quote:
In kernel mode, the r command displays registers that are associated with the current register context. You can set the register context to match a specific thread, context record, or trap frame. Only the most important registers for the specified register context are actually displayed, and you cannot change their values.



Just to answer your comment about remote usermode debugging - I just embedded _asm{int 3} before the FP instructions to step over in a test app, same as in the test driver. When the VM window freezes, Windbg in the host takes over.

re OSR - Yes, sure. But I think your posting would be just fine as long as you don't mention reversing or using undocumented stuph (which they really hate ). It's a valid question otherwise and would probably interest them.