Kayaker
December 27th, 2005, 01:43
Hi All,
Chances are I might find a few more clues to this with some hands on tracing, but I throw this out as a general interest question.
A ring 3 debugger like Olly or MSDev can be registered as a JIT (Just in Time) debugger which will trigger on DebugBreak API calls (basically an embedded INT3). The default JIT debugger is registered under
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
"Debugger"="\"C:\\Program Files\\Microsoft Visual Studio\\Common\\MSDev98\\Bin\\msdev.exe\" -p %ld -e %ld"
References to AeDebug leads to UnhandledExceptionFilter in kernel32.dll. The answer to my question may be buried in UnhandledExceptionFilter, or it might be lower, deeper and earlier in how the system handles DebugBreak. I figure there are others who have reversed UnhandledExceptionFilter in greater detail, for other reasons, and might have some clues.
The problem...
I've got a method of injecting code from ring0 at process or thread start, by intercepting a relevant syscall used during the act of creating an initial or secondary thread. The injected code, which executes immediately before thread start, is run from a mapped user mode MDL, generally in the 140000 memory region, returning to regular code say at 401000.
*In* this injected code I issue an option to send the DebugBreak API to the system debugger. No particular reason, I just wanted to do something funky to see what happened..
When issued from the context of the MDL code, running at 140000, DebugBreak wil NOT trigger the default JIT debugger. If DebugBreak is written into regular code it does, as it's supposed to.
If the process was *initially* loaded by a debugger, and not necessarily the system JIT debugger, the DebugBreak in the MDL code is recognized and is properly handled by the system (the attached debugger handles the INT3). (As does having Softice active with I3HERE ON).
What I'm trying to ascertain is *why* DebugBreak executed at 140000 in an MDL does not trigger a JIT debugger. The MDL *is* in the same context as the thread, but it's ignored. (I issue a MessageBox before DebugBreak is used, so the process now sits in an initial "GUI" thread, for whatever relevance there may be to win32k.sys/SSDT shadow table).
I'm wondering if it's as simple as an address check (>400000) in some naughty bit of kernel code, either the INT3 handler or UnhandledExceptionFilter, or it's some other restriction inherent in the system.
One thing I haven't tried, and might shed some light, is to try to execute DebugBreak from some memory injected in a more traditional way, such as CreateRemoteThread, where the memory is allocated by VirtualAllocEx, instead of an ntoskrnl handled MDL mapping.
In any case, a little holiday reversing mystery to ponder for those not overly inebriated...
Cheers,
Kayaker
Chances are I might find a few more clues to this with some hands on tracing, but I throw this out as a general interest question.
A ring 3 debugger like Olly or MSDev can be registered as a JIT (Just in Time) debugger which will trigger on DebugBreak API calls (basically an embedded INT3). The default JIT debugger is registered under
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
"Debugger"="\"C:\\Program Files\\Microsoft Visual Studio\\Common\\MSDev98\\Bin\\msdev.exe\" -p %ld -e %ld"
References to AeDebug leads to UnhandledExceptionFilter in kernel32.dll. The answer to my question may be buried in UnhandledExceptionFilter, or it might be lower, deeper and earlier in how the system handles DebugBreak. I figure there are others who have reversed UnhandledExceptionFilter in greater detail, for other reasons, and might have some clues.
The problem...
I've got a method of injecting code from ring0 at process or thread start, by intercepting a relevant syscall used during the act of creating an initial or secondary thread. The injected code, which executes immediately before thread start, is run from a mapped user mode MDL, generally in the 140000 memory region, returning to regular code say at 401000.
*In* this injected code I issue an option to send the DebugBreak API to the system debugger. No particular reason, I just wanted to do something funky to see what happened..
When issued from the context of the MDL code, running at 140000, DebugBreak wil NOT trigger the default JIT debugger. If DebugBreak is written into regular code it does, as it's supposed to.
If the process was *initially* loaded by a debugger, and not necessarily the system JIT debugger, the DebugBreak in the MDL code is recognized and is properly handled by the system (the attached debugger handles the INT3). (As does having Softice active with I3HERE ON).
What I'm trying to ascertain is *why* DebugBreak executed at 140000 in an MDL does not trigger a JIT debugger. The MDL *is* in the same context as the thread, but it's ignored. (I issue a MessageBox before DebugBreak is used, so the process now sits in an initial "GUI" thread, for whatever relevance there may be to win32k.sys/SSDT shadow table).
I'm wondering if it's as simple as an address check (>400000) in some naughty bit of kernel code, either the INT3 handler or UnhandledExceptionFilter, or it's some other restriction inherent in the system.
One thing I haven't tried, and might shed some light, is to try to execute DebugBreak from some memory injected in a more traditional way, such as CreateRemoteThread, where the memory is allocated by VirtualAllocEx, instead of an ntoskrnl handled MDL mapping.
In any case, a little holiday reversing mystery to ponder for those not overly inebriated...

Cheers,
Kayaker