Log in

View Full Version : Inside SetUnhandledExceptionFilter


evilcry
July 26th, 2008, 03:06
Hi,

SetUnhandledExceptionFilter()
is frequently used as Anti Debug Trick, especially in Malware Applications. Around here there are various plugins for Olly that allows the Reverser to trasparently debug this kind of protection, so there is not a real necessity add other words about the mere practical part of trick overcoming.

Due to the fact that today, too many young reversers uses a ton of plugins anti - anti - xxx without knowing how internally they works, I decided to expose here a little summary of SetUnhandledExceptionFilter Internal characteristics.

First of all, what SetUnhandledExceptionFilter is? according to MSDN documentation:

Enables an application to supersede the top-level exception handler of each thread of a process.

After calling this function, if an exception occurs in a process that is not being debugged, and the exception makes it to the unhandled exception filter, that filter will call the exception filter function specified by the lpTopLevelExceptionFilter parameter.


And this is the Syntax:

Code:
LPTOP_LEVEL_EXCEPTION_FILTER WINAPI SetUnhandledExceptionFilter(
__in LPTOP_LEVEL_EXCEPTION_FILTER lpTopLevelExceptionFilter
);

lpTopLevelExceptionFilter is a pointer to top-level exception filter function that will be called whenever the UnhandledExceptionFilter function gets control, and the process is not being debugged. A value of NULL for this parameter specifies default handling within UnhandledExceptionFilter.

Usually, in absence of an UnhandledExceptionFilter the topmost handler called when an uncatched exception occours, is the default one provided by Windows Itself, the classical MessageBox that advices the user that an Unhandled Exception has occured.

But Windows allow programs to use custom Handlers for UnhandledException. The core of the trick is here, if the application is NOT debugged, the application is able to call the Custom Handler, but if the application IS debugged the Custom Handler will be never called.

The possibility of cognitive differentiation make obviously able the target application to apply a series of countemeasures against debugging, from detection to code hidding.

Just remember that due to the architecture of Windows Exception Handling, in every case is called UnhlandledExceptionFilter() function, and this will our point of attack (for anti - anti dbg trick).

This is the general inner meccanism of SetUnhandledExceptionFilter(), going more deep we observe the call stack of the first thread of any Win32 application, we can see that execution in every case is reported to BaseProcess, here the pseudo definition:

Code:
VOID BaseProcessStart( PPROCESS_START_ROUTINE pfnStartAddr )
{
__try
{
ExitThread( (pfnStartAddr)() );
}
__except( UnhandledExceptionFilter( GetExceptionInformation()) )
{
ExitProcess( GetExceptionCode() );
}
}

The same thing happens for threads, by referencing to BaseThreadStart:

Code:
VOID BaseThreadStart( PTHREAD_START_ROUTINE pfnStartAddr, PVOID pParam )
{
__try
{
ExitThread( (pfnStartAddr)(pParam) );
}
__except( UnhandledExceptionFilter(GetExceptionInformation()) )
{
ExitProcess( GetExceptionCode() );
}
}

All that happens inside BaseProcessStart() and BaseThreadStart() for what previously said, will be passed to the UnhandledExceptionFilter().

It’s now time to see what really is UnhandledExceptionFilter(), according to MSDN:

An application-defined function that passes unhandled exceptions to the debugger, if the process is being debugged. Otherwise, it optionally displays an Application Error message box and causes the exception handler to be executed. This function can be called only from within the filter expression of an exception handler.

Syntax:


Code:
LONG WINAPI UnhandledExceptionFilter(
__in struct _EXCEPTION_POINTERS *ExceptionInfo
);

Became clear that UnhandledExceptionFilter represents the last choise for processing unhandled exceptions, so the Check Debugger Presence surely is located inside this function, let’s see a simplified version of this function:

Code:
LONG UnhandledExceptionFilter( EXCEPTION_POINTERS* pep )
{
DWORD rv;

EXCEPTION_RECORD* per = pep->ExceptionRecord;

if( ( per->ExceptionCode == EXCEPTION_ACCESS_VIOLATION ) &&
( per->ExceptionInformation[0] != 0 ) )
{
rv = BasepCheckForReadOnlyResource( per->ExceptionInformation[1] );

if( rv == EXCEPTION_CONTINUE_EXECUTION )
return EXCEPTION_CONTINUE_EXECUTION;
}

DWORD DebugPort = 0;

rv = NtQueryInformationProcess( GetCurrentProcess(), ProcessDebugPort,
&DebugPort, sizeof( DebugPort ), 0 );


if( ( rv >= 0 ) && ( DebugPort != 0 ) )
{
// Yes, it is -> Pass exception to the debugger
return EXCEPTION_CONTINUE_SEARCH;
}

// Is custom filter for unhandled exceptions registered ?

if( BasepCurrentTopLevelFilter != 0 )
{
// Yes, it is -> Call the custom filter

rv = (BasepCurrentTopLevelFilter)(pep);

if( rv == EXCEPTION_EXECUTE_HANDLER )
return EXCEPTION_EXECUTE_HANDLER;

if( rv == EXCEPTION_CONTINUE_EXECUTION )
return EXCEPTION_CONTINUE_EXECUTION;
}

}

As you can see, inside UnhandledExceptionFilter() is called NtQueryInformationProcess() that has as first parameter our process and next DebugPort, this is done to know if the process is debugged.

All that we have to do to obtain an apparently undebugged process is to modify the first parameter (last pushed at debugging time), in other words we have to change the retur value of GetCurrentProcess() from 0xFFFFFFFF to 0×00000000.

So remember, when you have to overcome a SetUnhandledExceptionFilter() just put a Breakpoint for UnhandledExceptionFilter() and go inside this function to modify the previously exposed parameter

Thanks to Oleg Starodumov for pseudocodes

See you to the next blog post..

deroko
July 27th, 2008, 17:04
neat, I guessed earlier how this trick works, and, as I see from your explanation, my guess was correct but as I use softice for debugging never was interested in detailed exploration. Tnx for detailed explanation

Arcane
July 28th, 2008, 07:55
nice post , this kinda stuff is always handy if for some reasons the plugins dont work..as it does happen

evilcry
July 28th, 2008, 09:08
Thank you people

Soon I'll post the Internals of another, not well known function, that
can be used for anti debug tricks..

Regards,
Evilcry

Arcane
July 28th, 2008, 10:06
sounds cool , perhaps also idea for sub project for the collective series ...since well ..all the Anti-dbg tricks are well..loosely collectet to my awareness and always nice to have a good updatet reference ?

evilcry
July 28th, 2008, 10:44
Could be a nice idea to have a Woodmann's Anti-Dbg Reference one,
there is one well documented and complete on SecurityFocus

http://www.securityfocus.com/infocus/1893